Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-07-18
17:17:39 sean-k-mooney cloud init cna be configured to retry its just not the default
17:18:13 sean-k-mooney so if this is on by defualt im concerned it woudl regression previous bugs we had where vms would not get an ssh key inejcted ectra
17:20:14 dansmith sean-k-mooney: yeah, that's exactly my concern about making this less obvious by having some requests work and some not because they're all independent on the backend
17:20:22 dansmith I missed your comment about that but +100
17:20:49 dansmith I think if we want to defeat caching in certain circumstances we *should* make cloud-init declare when it wants uncached data or not,
17:20:53 dansmith and otherwise we're just guessing
17:21:06 dansmith but splitting this into N backend requests doesn't seem to improve things at all to me
17:26:02 sean-k-mooney i was wondering if there was soem way to use etag or just support triggerting the cache invalidation form the clinet side i.e. cluod init
17:26:11 sean-k-mooney or curl
17:26:32 sean-k-mooney i dont know what the standard convetion is for that at a http level but im sure there is one
17:30:16 sean-k-mooney sound like there is a semi standardisaed HTTP PURGE method that cloudflare, varnish and squid all support to do this
17:31:31 dansmith cache-control, I mentioned it in my review
17:31:49 sean-k-mooney cache-contol is slightly different
17:32:07 sean-k-mooney that says if caching layers are allowed to cache it or not and for how long
17:32:22 dansmith it's usable on request as well
17:32:25 sean-k-mooney that works with etag to allow the caching server to know if the content changed
17:32:31 sean-k-mooney oh really
17:32:39 dansmith https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
17:32:46 sean-k-mooney so you can send a cache-contole no-store or similr
17:32:49 dansmith max-age, no-cache, etc would be perfect here I think
17:33:03 sean-k-mooney ya i tought that was responce only
17:33:09 sean-k-mooney but if it can be in the request
17:33:11 sean-k-mooney then yes
17:34:28 sean-k-mooney ya so that seams quite reasonabel to look into
17:56:16 opendevreview Merged openstack/nova-specs master: Propose a new API microversion for a spice-direct console. https://review.opendev.org/c/openstack/nova-specs/+/915190
20:40:14 opendevreview melanie witt proposed openstack/nova master: nova-manage: Add flavor scanning to migrate_to_unified_limits https://review.opendev.org/c/openstack/nova/+/924110
23:40:47 opendevreview melanie witt proposed openstack/nova master: nova-manage: Add flavor scanning to migrate_to_unified_limits https://review.opendev.org/c/openstack/nova/+/924110
#openstack-nova - 2024-07-19
06:39:47 mikal bauzas: I am fine with working on adding tests too. Without Kerbside deployed it would still be meaningful to have a functional test which called the console create call to get an auth token and then looked up connection details from that.
06:40:14 mikal One wart there is that the openstacksdk doesn't actually support the console auth token API at the moment, so I need to add that too unless I am going to write a different client for that test.
07:49:59 bauzas mikal: fine by me, we'll discuss this on the implementation then
08:52:37 bauzas sean-k-mooney: when you're up, could you please +2/+W https://review.opendev.org/c/openstack/os-traits/+/908966 ?
08:52:46 bauzas then I'll create a os-traits release patch
08:54:27 sean-k-mooney bauzas: done
08:54:31 bauzas ++
08:54:39 sean-k-mooney i have been thinking about the cpu power managment
08:54:56 sean-k-mooney i have a simple quick fix and hten a longer term fix
08:54:58 bauzas all approved specs are now in the dalmatian etherpad
08:55:08 bauzas and Launchpad is now correct
08:55:14 bauzas sean-k-mooney: kk, tell me
08:55:28 bauzas I just need to go to a cycling shop in 30 mins
08:55:46 sean-k-mooney the simple quick fix is when nova-compute is stopped we can just power on all core
08:56:33 sean-k-mooney and when we start up we can just cache the core info with the runonce decorator before we turn off any of the cores
08:57:34 bauzas it would be a behaviour change, but why not
08:58:24 sean-k-mooney the longer term soltuion is we may want to add a local key value store or some other form of data persitince
08:59:21 sean-k-mooney we have had a few feature lately where having a very simple key value store avaibel o nthe compute like https://docs.python.org/3/library/dbm.html
08:59:27 sean-k-mooney would have been useful
08:59:40 sean-k-mooney so we shoudl disucss that at some point
09:00:14 bauzas for remembering the governor strategy ?
09:00:27 sean-k-mooney there are several other good option like sqlite
09:00:31 sean-k-mooney bauzas: not just for that
09:00:42 sean-k-mooney for data persitance on the comptue node locally in general
09:01:05 sean-k-mooney bauzas: for example for storing information about the image in the image cache
09:01:05 bauzas then this is a way larger context but I don't disagree
09:01:25 bauzas I just think this is useless for just caching the governor state
09:01:26 sean-k-mooney bauzas: right that why i said long term for that approhc
09:01:49 sean-k-mooney bauzas: right i never mention those becaus im not suggestin we do
09:02:29 sean-k-mooney what we need to cache is strictly the numa info for the cores before we turn them off so that the numa toplgoy blob has the correct numa node and socket
09:02:35 bauzas what we can also do is when the instance is about to spawn, we can doublecheck whether the core is poweroff if this is asking to get a governor
09:02:47 bauzas to set*
09:02:54 sean-k-mooney bauzas: this is not related to the govenor
09:03:05 sean-k-mooney im not talkign about that bug im talking about the other one
09:03:26 sean-k-mooney the one where libvirt returns incrroet data when the core is off
09:03:34 sean-k-mooney which breaks numa affinity
09:03:37 bauzas hah
09:04:12 sean-k-mooney bauzas: i think im ok with the patch you propsed yested for the govoner bug
09:04:21 sean-k-mooney at least for now
09:06:54 bauzas we can hold a little bit the resolution
09:07:09 bauzas as people want
09:07:38 bauzas honestly, that simple series became way more tricky than expected
09:07:55 bauzas because the kernel doesn't act like I was expecting
09:08:07 bauzas I just wish this was more documented
09:08:43 opendevreview Merged openstack/os-traits master: Add a new trait for AMD SEV-ES https://review.opendev.org/c/openstack/os-traits/+/908966
09:10:28 gibi turning on cpus during nova-compute shutdown is leaky, what if the hypervisor is rebooted out of the blue?
09:13:11 gibi if we want to go this direction the it is safer to do the following in init_host: turn on all dedicated cores, collect governor data, run off all unused dedicated cores.
09:13:19 opendevreview Fabian Wiesel proposed openstack/nova-specs master: Lazy Metadata Loading in order to Reduce Server Load https://review.opendev.org/c/openstack/nova-specs/+/922201
09:15:49 bauzas I need to leave now but I'm cool with discussing this later on this afternoon
09:16:23 sean-k-mooney gibi: if the hyperiofr is reboot all the cores would be one but we coudl turn them all on, gather the numa info once and then turn them off
09:16:27 sean-k-mooney when we start
09:16:47 sean-k-mooney so yes im fine with that in init_host
09:17:44 sean-k-mooney so the toploty info of a core (socket, numa node, cluster, die) shoudl not change while the agent is running
09:17:53 sean-k-mooney that would iply the phsical hardware changed
09:18:40 gibi yeah I agree that we should be able to cache the topology at startup and not re-query it.
09:19:03 sean-k-mooney so if we can ensure we are collecting good data, (by turning on all the cores we manage), we can look that up once and stop doing it every 5 mins for no real value
09:20:04 sean-k-mooney i think in the short term that is the solution to the bug,
09:20:50 sean-k-mooney in the long term as i said to bauzas i think we have usecase wehre not could benifit form a persitient datastore that is local to the nova-compute agent fo some kind
09:21:18 gibi yeah, we can discuss the persistent store for long term.
09:21:18 sean-k-mooney so we have a tool we can use to solve these type of problems in a more robust way
09:22:45 sean-k-mooney too be clear im alo not opposed ot useing the filesytem as a db. that works very well and is simple to debug and understand but i think we shoudl gater some usecase and see why might be more approate and talk about this more before or during the ptg
09:35:43 gibi bauzas: I left a simple request in the power mgmt fix https://review.opendev.org/c/openstack/nova/+/924427
09:40:37 sean-k-mooney gibi: yep i agree with %
09:40:39 sean-k-mooney ^
09:41:43 gibi bauzas: and thanks for quickly jumping on this bug
10:23:13 opendevreview Sylvain Bauza proposed openstack/nova master: cpu: Only check governor type on online cores https://review.opendev.org/c/openstack/nova/+/924427
10:23:20 bauzas sean-k-mooney: gibi: patch updated
10:23:34 bauzas once merged, I'll provide the backports to Antelope
10:37:33 sean-k-mooney the previous results were green and the change is trivial so +2
10:45:34 gibi bauzas: thanks +A
11:32:32 bauzas tkajinam: cores, fwiw os-traits release patch https://review.opendev.org/c/openstack/releases/+/924492
11:56:17 stephenfin gmann: Lemme know when you're around. I'm not sure I understand what you mean in https://review.opendev.org/c/openstack/nova/+/915738
11:56:29 opendevreview Stefan Hoffmann proposed openstack/nova master: wait for ovn network at migration https://review.opendev.org/c/openstack/nova/+/924500

Earlier   Later