| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-07-18 | |||
| 16:42:00 | sean-k-mooney | liekk is the plan ot create a url router within the metadata code to fetch only the conent for that file via decompoing things into lots of funcitons | |
| 16:42:51 | sean-k-mooney | or do some kind of properlty based laze loading on a class | |
| 16:44:19 | sean-k-mooney | there is really just https://review.opendev.org/c/openstack/nova-specs/+/922201/9/specs/2024.2/approved/lazy-metadata-loading.rst#79 | |
| 16:53:38 | dansmith | sean-k-mooney: yeah, if there's a race in the boot, add floating, cloud-init sequence it could be wrong | |
| 17:08:13 | sean-k-mooney | so to me the metadata api comes under the banner of "no more proxy apis" in nova | |
| 17:08:55 | sean-k-mooney | i.e. its not a data socue that shoudl be proxiying up to date info form other service its just a cached view of some useful info | |
| 17:09:22 | sean-k-mooney | so the idea that applciations are writen to poll it at a rate | |
| 17:09:29 | dansmith | idk, I don't think that's reasonable if there is literally no other way to get the info and we're holding the guest in our hand | |
| 17:09:35 | sean-k-mooney | that is enough to cause load issues is kind of bizar to me | |
| 17:09:43 | dansmith | like, if we start provisioning them with credentials and a directory, then I'm all for it | |
| 17:09:59 | dansmith | (provisioning with credentials that can be renewed I should say) | |
| 17:10:23 | sean-k-mooney | ya so at some point i think that should be done like how pods in k8s get service credital tokens to be able to talk to k8s | |
| 17:10:46 | sean-k-mooney | but short term if we can optimise it then ok im just concerned by the lack of detial on how | |
| 17:10:48 | dansmith | right | |
| 17:10:53 | dansmith | people have been asking for that for a long time | |
| 17:11:12 | opendevreview | Sylvain Bauza proposed openstack/nova master: cpu: Only check governor type on online cores https://review.opendev.org/c/openstack/nova/+/924427 | |
| 17:11:17 | sean-k-mooney | ya i think its a reason able thing to od at least if the user opts into it | |
| 17:11:33 | sean-k-mooney | but its not what the spec is adressing | |
| 17:12:10 | bauzas | sean-k-mooney: I had to drop due to a bugfix, but please just capture your concerns on that lazy-loading metadata spec so you can move on | |
| 17:12:28 | bauzas | and we'll revisit that spec at the PTG | |
| 17:16:55 | sean-k-mooney | bauzas: i rased most of them in the spec already, mainily i dont know how they are planning to implemnte the lazy loading and im concured it will break cloud-init or cirros-init due to the fact they dont retry by default and some recuest take longer then other to formulate | |
| 17:17:04 | bauzas | ++ | |
| 17:17:27 | sean-k-mooney | bauzas: bascily cloud-init and cirros-init today only retry the first request and assuem the rest will be fast and work first try by default | |
| 17:17:39 | sean-k-mooney | cloud init cna be configured to retry its just not the default | |
| 17:18:13 | sean-k-mooney | so if this is on by defualt im concerned it woudl regression previous bugs we had where vms would not get an ssh key inejcted ectra | |
| 17:20:14 | dansmith | sean-k-mooney: yeah, that's exactly my concern about making this less obvious by having some requests work and some not because they're all independent on the backend | |
| 17:20:22 | dansmith | I missed your comment about that but +100 | |
| 17:20:49 | dansmith | I think if we want to defeat caching in certain circumstances we *should* make cloud-init declare when it wants uncached data or not, | |
| 17:20:53 | dansmith | and otherwise we're just guessing | |
| 17:21:06 | dansmith | but splitting this into N backend requests doesn't seem to improve things at all to me | |
| 17:26:02 | sean-k-mooney | i was wondering if there was soem way to use etag or just support triggerting the cache invalidation form the clinet side i.e. cluod init | |
| 17:26:11 | sean-k-mooney | or curl | |
| 17:26:32 | sean-k-mooney | i dont know what the standard convetion is for that at a http level but im sure there is one | |
| 17:30:16 | sean-k-mooney | sound like there is a semi standardisaed HTTP PURGE method that cloudflare, varnish and squid all support to do this | |
| 17:31:31 | dansmith | cache-control, I mentioned it in my review | |
| 17:31:49 | sean-k-mooney | cache-contol is slightly different | |
| 17:32:07 | sean-k-mooney | that says if caching layers are allowed to cache it or not and for how long | |
| 17:32:22 | dansmith | it's usable on request as well | |
| 17:32:25 | sean-k-mooney | that works with etag to allow the caching server to know if the content changed | |
| 17:32:31 | sean-k-mooney | oh really | |
| 17:32:39 | dansmith | https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control | |
| 17:32:46 | sean-k-mooney | so you can send a cache-contole no-store or similr | |
| 17:32:49 | dansmith | max-age, no-cache, etc would be perfect here I think | |
| 17:33:03 | sean-k-mooney | ya i tought that was responce only | |
| 17:33:09 | sean-k-mooney | but if it can be in the request | |
| 17:33:11 | sean-k-mooney | then yes | |
| 17:34:28 | sean-k-mooney | ya so that seams quite reasonabel to look into | |
| 17:56:16 | opendevreview | Merged openstack/nova-specs master: Propose a new API microversion for a spice-direct console. https://review.opendev.org/c/openstack/nova-specs/+/915190 | |
| 20:40:14 | opendevreview | melanie witt proposed openstack/nova master: nova-manage: Add flavor scanning to migrate_to_unified_limits https://review.opendev.org/c/openstack/nova/+/924110 | |
| 23:40:47 | opendevreview | melanie witt proposed openstack/nova master: nova-manage: Add flavor scanning to migrate_to_unified_limits https://review.opendev.org/c/openstack/nova/+/924110 | |
| #openstack-nova - 2024-07-19 | |||
| 06:39:47 | mikal | bauzas: I am fine with working on adding tests too. Without Kerbside deployed it would still be meaningful to have a functional test which called the console create call to get an auth token and then looked up connection details from that. | |
| 06:40:14 | mikal | One wart there is that the openstacksdk doesn't actually support the console auth token API at the moment, so I need to add that too unless I am going to write a different client for that test. | |
| 07:49:59 | bauzas | mikal: fine by me, we'll discuss this on the implementation then | |
| 08:52:37 | bauzas | sean-k-mooney: when you're up, could you please +2/+W https://review.opendev.org/c/openstack/os-traits/+/908966 ? | |
| 08:52:46 | bauzas | then I'll create a os-traits release patch | |
| 08:54:27 | sean-k-mooney | bauzas: done | |
| 08:54:31 | bauzas | ++ | |
| 08:54:39 | sean-k-mooney | i have been thinking about the cpu power managment | |
| 08:54:56 | sean-k-mooney | i have a simple quick fix and hten a longer term fix | |
| 08:54:58 | bauzas | all approved specs are now in the dalmatian etherpad | |
| 08:55:08 | bauzas | and Launchpad is now correct | |
| 08:55:14 | bauzas | sean-k-mooney: kk, tell me | |
| 08:55:28 | bauzas | I just need to go to a cycling shop in 30 mins | |
| 08:55:46 | sean-k-mooney | the simple quick fix is when nova-compute is stopped we can just power on all core | |
| 08:56:33 | sean-k-mooney | and when we start up we can just cache the core info with the runonce decorator before we turn off any of the cores | |
| 08:57:34 | bauzas | it would be a behaviour change, but why not | |
| 08:58:24 | sean-k-mooney | the longer term soltuion is we may want to add a local key value store or some other form of data persitince | |
| 08:59:21 | sean-k-mooney | we have had a few feature lately where having a very simple key value store avaibel o nthe compute like https://docs.python.org/3/library/dbm.html | |
| 08:59:27 | sean-k-mooney | would have been useful | |
| 08:59:40 | sean-k-mooney | so we shoudl disucss that at some point | |
| 09:00:14 | bauzas | for remembering the governor strategy ? | |
| 09:00:27 | sean-k-mooney | there are several other good option like sqlite | |
| 09:00:31 | sean-k-mooney | bauzas: not just for that | |
| 09:00:42 | sean-k-mooney | for data persitance on the comptue node locally in general | |
| 09:01:05 | bauzas | then this is a way larger context but I don't disagree | |
| 09:01:05 | sean-k-mooney | bauzas: for example for storing information about the image in the image cache | |
| 09:01:25 | bauzas | I just think this is useless for just caching the governor state | |
| 09:01:26 | sean-k-mooney | bauzas: right that why i said long term for that approhc | |
| 09:01:49 | sean-k-mooney | bauzas: right i never mention those becaus im not suggestin we do | |
| 09:02:29 | sean-k-mooney | what we need to cache is strictly the numa info for the cores before we turn them off so that the numa toplgoy blob has the correct numa node and socket | |
| 09:02:35 | bauzas | what we can also do is when the instance is about to spawn, we can doublecheck whether the core is poweroff if this is asking to get a governor | |
| 09:02:47 | bauzas | to set* | |
| 09:02:54 | sean-k-mooney | bauzas: this is not related to the govenor | |
| 09:03:05 | sean-k-mooney | im not talkign about that bug im talking about the other one | |
| 09:03:26 | sean-k-mooney | the one where libvirt returns incrroet data when the core is off | |
| 09:03:34 | sean-k-mooney | which breaks numa affinity | |
| 09:03:37 | bauzas | hah | |
| 09:04:12 | sean-k-mooney | bauzas: i think im ok with the patch you propsed yested for the govoner bug | |
| 09:04:21 | sean-k-mooney | at least for now | |
| 09:06:54 | bauzas | we can hold a little bit the resolution | |
| 09:07:09 | bauzas | as people want | |
| 09:07:38 | bauzas | honestly, that simple series became way more tricky than expected | |
| 09:07:55 | bauzas | because the kernel doesn't act like I was expecting | |
| 09:08:07 | bauzas | I just wish this was more documented | |
| 09:08:43 | opendevreview | Merged openstack/os-traits master: Add a new trait for AMD SEV-ES https://review.opendev.org/c/openstack/os-traits/+/908966 | |
| 09:10:28 | gibi | turning on cpus during nova-compute shutdown is leaky, what if the hypervisor is rebooted out of the blue? | |
| 09:13:11 | gibi | if we want to go this direction the it is safer to do the following in init_host: turn on all dedicated cores, collect governor data, run off all unused dedicated cores. | |
| 09:13:19 | opendevreview | Fabian Wiesel proposed openstack/nova-specs master: Lazy Metadata Loading in order to Reduce Server Load https://review.opendev.org/c/openstack/nova-specs/+/922201 | |
| 09:15:49 | bauzas | I need to leave now but I'm cool with discussing this later on this afternoon | |
| 09:16:23 | sean-k-mooney | gibi: if the hyperiofr is reboot all the cores would be one but we coudl turn them all on, gather the numa info once and then turn them off | |