| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-05-16 | |||
| 14:53:23 | sean-k-mooney | yep and im not sure we ever supproted it in osc | |
| 14:53:32 | sean-k-mooney | because we dont want to main that | |
| 14:53:48 | bauzas | huzzah it worked | |
| 14:53:59 | bauzas | upping the registered limit, I mean | |
| 14:54:35 | bauzas | size | 25571885056 | |
| 14:54:38 | bauzas | voila the reason | |
| 14:54:54 | bauzas | 25GB | |
| 15:26:32 | clarkb | zigo: https://docs.yubico.com/hardware/yubihsm-2/hsm-2-user-guide/hsm2-openssl-libp11.html this is for their hsm product but maybe similar for regular yubikeys | |
| 15:27:22 | clarkb | alternatively ssh using the key then change the password and don't bother involving nova at all? | |
| 15:36:42 | zigo | clarkb: There's no such thing as sshd in our windows instances, just cloud-base-init. I'm trying to figure out what format "nova get-password" is giving out, and if I can unencrypt it with my yubikey... | |
| 15:37:44 | clarkb | zigo: got it (I think you can run an sshd on windows now fwiw, but that makes sense). Maybe those yubikey hsm docs are the clue you need for using the hardware to decrypt | |
| 15:38:05 | zigo | Thanks. I'm reading it. | |
| 15:38:06 | dansmith | yep sshd on windows is trivial now | |
| 15:38:10 | dansmith | built-in almost | |
| 15:42:40 | opendevreview | Merged openstack/nova master: Make python 3.12 unit and functional voting https://review.opendev.org/c/openstack/nova/+/919767 | |
| 16:54:39 | bauzas | melwitt: I want to demonstrate VGPU quota usage on my demo next week | |
| 16:55:02 | bauzas | when setting unifiedlimits driver in nova.conf, which services are using it ? n-sch and n-api right? | |
| 16:55:29 | bauzas | none from the conductor or the compute ? | |
| 16:56:01 | melwitt | bauzas: I think only n-api. I'll double check | |
| 16:56:24 | bauzas | I quickly looked at the code, I only found the scheduler and the api were importing nova.quota | |
| 16:58:44 | melwitt | bauzas: there's a "recheck" in nova-conductor that enforces "strict quota" like races | |
| 16:59:06 | bauzas | then I need to also update nova_cell1.conf | |
| 17:00:41 | bauzas | Unable to initialize OpenStackSDK session: An auth plugin is required to determine endpoint URL: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin | |
| 17:00:48 | bauzas | humpf, I probably need to set something else | |
| 17:01:29 | bauzas | melwitt: do you know what I missed ? ^ | |
| 17:01:56 | bauzas | (that's the API exception I get when wanting to create a new instance) | |
| 17:02:02 | melwitt | looking.. | |
| 17:02:48 | bauzas | oh, I have nothing | |
| 17:02:55 | bauzas | devstack doesn't automatically set it | |
| 17:04:00 | melwitt | yeah.. maybe something switched to using the sdk since then? I don't remember seeing anything like this | |
| 17:04:18 | melwitt | I need to lookup what's needed for sdk. though it seems surprising it wouldn't be done automatically in devstack like you said | |
| 17:08:46 | melwitt | oh huh that's actually coming from oslo.limit. so likely something needed in an [oslo_limit] conf section. still looking | |
| 17:09:03 | bauzas | yeah, still getting the api exception | |
| 17:09:21 | bauzas | after restarting n-api (and setting [keystone] section)à | |
| 17:10:04 | melwitt | https://docs.openstack.org/oslo.limit/latest/user/usage.html#configuration | |
| 17:10:09 | bauzas | https://docs.openstack.org/oslo.limit/latest/user/usage.html#configuration | |
| 17:10:13 | bauzas | heh | |
| 17:11:14 | melwitt | I'm just confused how/where this got missed. and I'll will add stuff to the quota docs | |
| 17:12:17 | melwitt | ok, here it is in devstack, it should have configured it for you https://github.com/openstack/devstack/blob/85b8d2ccab53bb2b38090c8801404072fae2f7ab/lib/nova#L817 | |
| 17:12:26 | dansmith | do we not have a job with unified limits enabled? | |
| 17:12:32 | melwitt | (I'll still add to docs) | |
| 17:12:39 | melwitt | we do, it's on in nova-next | |
| 17:13:13 | dansmith | ack, so it must be in devstack unless we're hacking it all on in our job config right? | |
| 17:14:02 | melwitt | yeah I just linked the devstack stuff above | |
| 17:14:25 | melwitt | bauzas maybe didn't do the config through devstack or something (?) | |
| 17:14:27 | dansmith | I know, I'm just talking about the assertion of it being missed or not supported | |
| 17:14:28 | dansmith | right | |
| 17:14:33 | melwitt | NOVA_USE_UNIFIED_LIMITS=True should do the stuff | |
| 17:14:52 | melwitt | oh, sure | |
| 17:15:42 | bauzas | no, I just modified nova.conf directly | |
| 17:16:27 | bauzas | I don't want to set quotas for all resources but only VGPU | |
| 17:16:51 | bauzas | I know it will mean that all the quotas limits will be unlimitied :) | |
| 17:17:16 | melwitt | ok cool. I'll propose a patch to add the oslo_limit config info to the docs, that's what I missed | |
| 17:18:02 | melwitt | bauzas: make sure you set a limit for DISK_GB otherwise it will treat it as 0 limit | |
| 17:18:27 | melwitt | that's the only new "core" quota resource | |
| 17:18:59 | bauzas | melwitt: oh ok | |
| 17:19:20 | melwitt | oh wait, actually you have to set all of them bc you didn't do in devstack. but you can just copy what's in devstack | |
| 17:20:01 | melwitt | https://github.com/openstack/devstack/blob/85b8d2ccab53bb2b38090c8801404072fae2f7ab/lib/nova#L789-L812 | |
| 17:20:20 | melwitt | and then this to make the nova service user able to read the quotas https://github.com/openstack/devstack/blob/85b8d2ccab53bb2b38090c8801404072fae2f7ab/lib/nova#L827-L829 | |
| 17:24:21 | bauzas | yup, I'm done now, I just need to tweak the registered limits | |
| 17:24:24 | bauzas | Project 1471d08833d141c583b5f04344476ebd is over a limit for [Resource class:DISK_GB is over limit of 0 due to current usage 80 and delta 40, Resource class:MEMORY_MB is over limit of 0 due to current usage 16384 and delta 8192, Resource class:VCPU is over limit of 0 due to current usage 2 and delta 1, Resource class:VGPU is over limit of 0 due to current usage 2 and delta 1, Resource servers is over limit of 0 due to current usa | |
| 17:24:26 | bauzas | ge 2 and delta 1] (HTTP 403) (Request-ID: req-980c41e8-6f18-4e12-984b-7eb5daa7d9e8) | |
| 17:24:53 | bauzas | so, I not only need to add a limit for DISK_GB but also the other classes | |
| 17:29:09 | melwitt | yes. any class that's going to be requested has to have a registered limit else oslo.limit treats it as zero | |
| 17:30:02 | bauzas | yup I know it :) | |
| 17:30:04 | melwitt | (which is why we were discussing at the PTG about how to make an unlimited option in nova) | |
| 17:30:27 | bauzas | yeah my brain fried | |
| 17:31:13 | bauzas | sometimes my brain can't reconcile two things :) | |
| 17:31:13 | melwitt | :) | |
| 17:35:08 | bauzas | yay that works | |
| 17:35:31 | bauzas | I have everything now for my demo, I just need to record it :) | |
| 17:35:51 | bauzas | my next nights => bye bye | |
| 17:36:01 | melwitt | \o/ | |
| 17:36:05 | bauzas | thanks melwitt for the quick resolution and the help | |
| 17:36:38 | bauzas | CERN will be more happy to test unified limits for VGPU if they haven't done it yet :) | |
| 17:36:52 | melwitt | np. you can ping me if you have questions or run into any other issues | |
| 17:37:10 | melwitt | that would be super :) | |
| 17:37:18 | bauzas | no, unified limits is waaaay simplier that f**** nvidia kernel usage :) | |
| 17:38:04 | melwitt | haha! yeah, I guess if you can nvidia then you can do anything 😆 | |
| 17:39:11 | bauzas | at least now I know more about cuda and numba :) | |
| 17:39:47 | bauzas | like the fact we need to use a specific cuda release related to the mdev type | |
| 17:40:23 | bauzas | (and the kernel nvidia version) | |
| 18:11:46 | sean-k-mooney | i mean its not surpsing the have spit there mdev by vdi vs comptue usage in there skus forever | |
| #openstack-nova - 2024-05-17 | |||
| 02:25:47 | opendevreview | melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913 | |
| 03:32:53 | opendevreview | melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913 | |
| 04:05:34 | opendevreview | melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913 | |
| 04:26:56 | opendevreview | melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913 | |
| 07:41:22 | opendevreview | Robert Hoffmann proposed openstack/nova master: Fix backups hanging in creating state https://review.opendev.org/c/openstack/nova/+/919923 | |
| 10:05:31 | opendevreview | Robert Hoffmann proposed openstack/nova master: Fix backups hanging in creating state https://review.opendev.org/c/openstack/nova/+/919923 | |
| 13:13:30 | opendevreview | Mauricio Faria de Oliveira proposed openstack/nova master: libvirt: Filter out disabled guest CPU features in _get_cpu_feature_traits() https://review.opendev.org/c/openstack/nova/+/919949 | |
| 14:10:22 | opendevreview | Andrew Bonney proposed openstack/nova master: [WIP] Set port dns_name when unshelving instances https://review.opendev.org/c/openstack/nova/+/919961 | |
| 17:56:14 | opendevreview | sean mooney proposed openstack/nova master: [WIP] ensure correct cleanup of multi-attach volumes https://review.opendev.org/c/openstack/nova/+/916322 | |
| 17:56:14 | opendevreview | sean mooney proposed openstack/nova master: add functional repoducer for bug 2048837 https://review.opendev.org/c/openstack/nova/+/919979 | |
| 17:57:09 | sean-k-mooney | i spend the better part of 6 hours trying to get that repoduce to pass ^ and then i realsied the reason i could not get it to pass was my fix didnt work... | |
| 17:57:29 | sean-k-mooney | on the plus side i now have a repoducer for the orginal bug... | |
| 17:59:39 | melwitt | too real | |
| 18:05:19 | sean-k-mooney | melwitt: you dont happen to knwo if we stash if a volume is a multi attach volume in the bdms do you | |
| 18:07:03 | melwitt | sean-k-mooney: there isn't a dedicated field for it that I see. I dunno if it could possibly be in the connection_info though | |
| 18:08:18 | sean-k-mooney | ack no worries | |
| 18:09:20 | sean-k-mooney | it is in the conection info yes | |