Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-01-16
09:56:27 bauzas I think you found the only one missing :(
09:56:37 bauzas (sorry, was in meeting)
09:57:37 gibi OK
09:57:40 gibi I will backport it
09:58:54 bauzas thanks
10:02:16 bauzas Uggla: are you talking of hypervisor_hostname field ?
10:03:06 bauzas if so, this is virt-dependent
10:04:15 Uggla yep, in fact it seems we have users blocked by the patched introduced by Dan to check rename compute. They registered their nodes with the short name and libvirt seems to provide the FQDN and thus making the comparison to fail and block the compute startup.
10:05:35 bauzas Uggla: libvirt populates this field by https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L9611
10:06:21 bauzas which itself calls libvirt with https://github.com/openstack/nova/blob/master/nova/virt/libvirt/host.py#L1065
10:06:53 Uggla @bauzas, yep I looked at the methods doc --> This returns a system hostname on which the hypervisor is running (based on the result of the gethostname system call, but possibly expanded to a fully-qualified domain name via getaddrinfo)
10:07:49 Uggla but I guess that's not libvirt which changed it's behavior.
10:08:08 bauzas yeah, just checked the libvirt release notes
10:08:22 bauzas this is possibly something that the operator did which broke
10:08:37 Uggla The question was more personal to understand what libvirt is doing.
10:08:53 bauzas and what dansmith writes actually prevents that, which is definitely what :)
10:08:56 bauzas we want :)
10:09:21 bauzas accidental hostname rewrites break placement and many other things
10:09:49 Uggla It seams the operator upgraded to 2023.1 and he is blocked by the check.
10:10:25 Uggla because short name != fqdn
10:10:57 bauzas that's good that they're blocked then
10:11:27 Uggla I guess it should not be blocked if the compute were registered wit the fqdn previously.
10:12:15 Uggla but I'm not sure we have guideline that enforce this ?
10:12:46 Uggla And how the operator ccan get out this issue without db surgery ?
10:13:35 bauzas nova only registers the nodename
10:13:46 bauzas that means that *something* changed it
10:13:52 gibi bauzas: there is another non backported fix for power mgmt: https://review.opendev.org/c/openstack/nova/+/885352 you implemented this on master but not backported. Therefore I see the bug in 2023.1 but did not noticed the master fix and implemented another fix that is now backported https://review.opendev.org/c/openstack/nova/+/903169 . Unfortunately
10:13:55 bauzas and now nova refuses to start
10:13:59 gibi https://review.opendev.org/c/openstack/nova/+/903169 has a bug in it as it expecting a wrong exception type
10:14:18 gibi bauzas: I will do some reverts and backports to clean this up
10:14:23 bauzas doh
10:14:40 bauzas gibi: apologies for the mess
10:15:08 bauzas I should create bugfixes with the same gerrit topic than the feature
10:15:08 gibi no worries I made my one fair share of mess in it
10:15:26 bauzas instead of using the bug number topic
10:15:40 gibi I prefer bug number topics
10:15:57 bauzas yeah I understand
10:16:02 bauzas this is a tradeoff
10:16:18 bauzas then I should mention some changeid in the commit msg, so gerrit could possibly link it
10:17:02 bauzas tbc, I don't trust my memory and that one is another evidence
10:17:38 bauzas the series was small and there were only two bugfixes, despite that I failed to remember about that
10:17:52 bauzas my brain is so fcked
10:19:43 opendevreview Merged openstack/nova stable/zed: Do not untrack resources of a server being unshelved https://review.opendev.org/c/openstack/nova/+/904375
10:24:40 gibi bauzas: I don't know how to avoid this via automation
10:25:22 gibi elodilles: do I need to backport a revert, or I can do a revert on the stable branch independently of a revert on master?
10:26:55 gibi elodilles: i.e can I do git revert on stable, or do I need to do git cherry-pick -x on the revert commit from master to stable?
10:32:58 gibi elodilles: I will go with the cherry-pick as I affraid of the stable backport script will not allow an independent revert :/
10:33:30 Uggla @bauzas you said we store only the shortname for compute name in the db right ?
10:33:37 bauzas nope
10:33:49 bauzas I said we store what libvirt gives us
10:34:24 bauzas if after an upgrade libvirt is giving us something else, then now with 2023.1 we fail
10:34:56 bauzas previously, we were blindly recreating duplicate resource providers leading to inconsistencies
10:35:43 Uggla yep it means that when the compute was registered he got the shortname and after upgrade or something else libvirt supply the fqdn.
10:36:21 Uggla @bauzas, note I understand the patch goal.
10:47:33 elodilles gibi: we usually simply backport reverts if it is needed all the way from master branch to multiple stable branches. if only needed for some older branches, then [stable-only] is fine. and of course if the bug is superimportant highly critical ultimate fix is needed, then also mark them as [stable-only] and merge them parallel in all stable branches :)
10:48:17 elodilles these are my memories from the past ^^^ :)
10:49:56 gibi not super critical
10:50:09 gibi going with the cherry-picks
10:50:42 opendevreview Balazs Gibizer proposed openstack/nova master: Revert "[pwmgmt]ignore missin governor when cpu_state used" https://review.opendev.org/c/openstack/nova/+/905671
10:51:02 opendevreview Balazs Gibizer proposed openstack/nova stable/2023.2: Revert "[pwmgmt]ignore missin governor when cpu_state used" https://review.opendev.org/c/openstack/nova/+/905672
10:51:03 opendevreview Balazs Gibizer proposed openstack/nova stable/2023.2: cpu: make governors to be optional https://review.opendev.org/c/openstack/nova/+/905673
10:51:16 opendevreview Balazs Gibizer proposed openstack/nova stable/2023.1: Revert "[pwmgmt]ignore missin governor when cpu_state used" https://review.opendev.org/c/openstack/nova/+/905674
10:51:17 opendevreview Balazs Gibizer proposed openstack/nova stable/2023.1: cpu: fix the privsep issue when offlining the cpu https://review.opendev.org/c/openstack/nova/+/905675
10:51:18 opendevreview Balazs Gibizer proposed openstack/nova stable/2023.1: cpu: make governors to be optional https://review.opendev.org/c/openstack/nova/+/905676
10:51:36 gibi elodilles, bauzas, sean-k-mooney ^^
10:52:52 gibi it is complicated as on master we only need to revert, on 2023.2 we revert and backport the proper fix from master, on 2023.1 we revert, backport a related fix from 2023.2 and then backport the original good fix from master->2023.2
10:52:57 bauzas gibi: ++
11:06:59 Uggla do we agree that there is no "proper" way to rename a compute node. And that we need to stop the compute service, remove the compute from the configuration, and restart the compute to have it registered with the new name ?
11:07:00 zigo kashyap: For Ubuntu stuff, please ask jamespage or coreycb_ please, I have no idea about #ubuntu-kernel on Libera at all, and I honestly don't know anything about the Ubuntu kernel in general.
11:07:46 kashyap zigo: Thans for the contacts :) In the past I had a couple of chats on #ubuntu-kernel, while it used to be still on FN :)
11:12:15 jamespage kashyap: lacking some context - just reading the etherpad linked above somewhere
11:12:59 kashyap jamespage: Hi, I didn't ping you with full context yet. :)
11:13:31 kashyap jamespage: It's about a certain class of kernel panics we're seeing in the upstream CI. An example: https://bugs.launchpad.net/nova/+bug/2018612
11:16:07 kashyap Admittedly (a) these are difficult to reproduce locally; and (b) distro kernel maintainers (understandably) might not have time to debug "unsupported distros" (as evidenced above).
11:16:30 kashyap I need to step out to get some lunch; need to drop here. (I don't have an IRC bouncer anymore)
11:16:36 jamespage ack
11:19:41 auniyal in cinderfixture for attachment_update https://review.opendev.org/c/openstack/nova/+/658904/4/nova/tests/fixtures.py#1997
11:19:41 auniyal why this volume_id must be self.MULTIATTACH_VOL (the sent volume is a dynamically created new volume, ) MULTIATTACH_VOL is a contant
11:19:53 auniyal gibi ^
11:23:44 auniyal I am trying to to create a server from snapshot-image and while updating attachment in cinder its failing at this step - can someone please tell why this is wrtten and what I can I do to use MULTIATTACH_VOL.
11:24:04 auniyal so attachment can get updated
11:33:05 opendevreview Merged openstack/nova master: Adds server show in helpers https://review.opendev.org/c/openstack/nova/+/893584
12:00:36 opendevreview Amit Uniyal proposed openstack/nova master: Fixes bug 2048184 https://review.opendev.org/c/openstack/nova/+/904817
12:00:37 opendevreview Amit Uniyal proposed openstack/nova master: Updates glance fixture for create image https://review.opendev.org/c/openstack/nova/+/905684
12:00:37 opendevreview Amit Uniyal proposed openstack/nova master: WIP: snapshot tests https://review.opendev.org/c/openstack/nova/+/905685
13:36:04 bauzas Uggla: in our OSP product, we have some solution for renaming a compute by scaling in and the out the computes
14:42:24 SvenKieske Hey, has anyone ever thought about enabling "core scheduling" support by default? It's in libvirt since v8.9.0 but I can't find any information regarding openstack implementation status: https://www.libvirt.org/news.html#v8-9-0-2022-11-01
14:46:58 SvenKieske when searching for this in nova I also found this gem, which seems not to be tracked anymore (last activity 2019, but bug was never closed): https://bugs.launchpad.net/nova/+bug/1417975
14:49:24 opendevreview Amit Uniyal proposed openstack/nova master: WIP: snapshot tests https://review.opendev.org/c/openstack/nova/+/905685
14:56:09 opendevreview Amit Uniyal proposed openstack/nova master: Updates glance fixture for create image https://review.opendev.org/c/openstack/nova/+/905684
14:56:10 opendevreview Amit Uniyal proposed openstack/nova master: Fixes bug 2048184 https://review.opendev.org/c/openstack/nova/+/904817
14:56:10 opendevreview Amit Uniyal proposed openstack/nova master: WIP: snapshot tests https://review.opendev.org/c/openstack/nova/+/905685
15:12:02 bauzas SvenKieske: that sounds a new feature request, if you want to use it with nova
15:13:18 gibi dansmith melwitt bauzas : a fresh kernel panic on master https://review.opendev.org/c/openstack/nova/+/905671
15:23:28 dansmith gibi: that's similar to the one that we opened against the ubuntu team and they asked us to repro on a newer kernel, so that's ... "good"
15:31:49 SvenKieske bauzas: yeah sure, I'm just a little baffled that I seem to be the first person to mention this feature request, as this is a rather useful feature, from a security perspective.
15:47:10 kashyap gibi: Hi, dansmith tells me you've ran into another kernel panic. Got a log link?
15:47:25 dansmith kashyap: https://review.opendev.org/c/openstack/nova/+/905671
15:48:01 dansmith kashyap: also for future reference: https://meetings.opendev.org/irclogs/
15:48:20 kashyap dansmith: Hi; thanks!

Earlier   Later