| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-02-05 | |||
| 19:22:33 | sean-k-mooney | so we can change it in one of the many jobs teh nova-ovs-hybrid-plug job | |
| 19:22:51 | dansmith | yeah a test at the end is good to me.. even if we can't land it with the rest, being able to see it working (or almost working or whatever) is data for review | |
| 19:23:33 | sean-k-mooney | mikal: i would modify this https://github.com/openstack/nova/blob/master/.zuul.yaml#L144 | |
| 19:23:46 | melwitt | mikal: tempest config are set in https://github.com/openstack/nova/blob/d267ede98f1b0aa25630b1efe07db5a8b478d473/.zuul.yaml#L136 for example | |
| 19:23:55 | sean-k-mooney | we use it to mainly test with ml2/ovs but its a good candiate for non standard configs | |
| 19:23:55 | mikal | Ok, so the moving parts are: add a test to tempest; add plumbing in devstack to enable that test; fiddle with nova gate jobs to use a devstack with the spice test turned on? | |
| 19:24:40 | sean-k-mooney | ya i might be able to help by putting up a poc patch of just enabling spice in that job | |
| 19:25:08 | sean-k-mooney | you coudl modify it and make it depend on your tempest patch when that is ready | |
| 19:25:10 | mikal | Ok, well I'll take a swing at this today but its sounding like more than I'll get done in a single day to be honest. | |
| 19:29:33 | opendevreview | Merged openstack/nova master: [Trivial] Fix the typo error https://review.opendev.org/c/openstack/nova/+/937744 | |
| 19:50:32 | opendevreview | sean mooney proposed openstack/nova master: Disable the heal instance info cache periodic task https://review.opendev.org/c/openstack/nova/+/939476 | |
| 19:51:08 | sean-k-mooney | bauzas: i did a spell check pass on ^ otherwise it the same as the last version | |
| 20:25:51 | opendevreview | Merged openstack/nova master: Cleanup RBAC jobs in check/gate pipeline https://review.opendev.org/c/openstack/nova/+/940737 | |
| 20:36:43 | opendevreview | sean mooney proposed openstack/nova master: [WIP] move nova0ovs-hybrid-plug to deploy with spice https://review.opendev.org/c/openstack/nova/+/940835 | |
| 20:45:19 | opendevreview | Doug Goldstein proposed openstack/nova stable/2024.2: ironic: Fix ConflictException when deleting server https://review.opendev.org/c/openstack/nova/+/940836 | |
| 21:02:02 | sean-k-mooney | mikal: https://review.opendev.org/c/openstack/nova/+/940835 might work but i expect it to fail, if it does i think https://review.opendev.org/c/openstack/devstack/+/940838 might fix it. im going to finish for today but if you want to rebase those patches, addign a depend on and setting NOVA_SPICE_ENABLED later feel free. | |
| 21:03:25 | sean-k-mooney | doing a quick search with hound i dont think we have any job anywhere that enables spice so i dont know if there is anything else broken | |
| 21:04:11 | sean-k-mooney | s/any job/any devstack job/ | |
| 22:10:31 | mikal | sean-k-mooney: thanks, I'll take a look soon. | |
| #openstack-nova - 2025-02-06 | |||
| 04:49:04 | chungwon | Hello, can you review this? | |
| 04:49:57 | chungwon | review.opendev.org/c/openstack/nova/+/939929 | |
| 05:21:32 | opendevreview | suiong ng proposed openstack/nova stable/2024.1: ironic: Fix ConflictException when deleting server https://review.opendev.org/c/openstack/nova/+/940846 | |
| 06:42:46 | opendevreview | Michael Still proposed openstack/nova master: libvirt: Add extra spec for sound device. https://review.opendev.org/c/openstack/nova/+/926126 | |
| 06:42:47 | opendevreview | Michael Still proposed openstack/nova master: libvirt: Add extra specs for USB redirection. https://review.opendev.org/c/openstack/nova/+/927354 | |
| 06:42:47 | opendevreview | Michael Still proposed openstack/nova master: Protect older compute managers from sound model requests. https://review.opendev.org/c/openstack/nova/+/940770 | |
| 06:42:48 | opendevreview | Michael Still proposed openstack/nova master: Don't calculate the minimum compute version repeatedly. https://review.opendev.org/c/openstack/nova/+/940848 | |
| 06:46:08 | opendevreview | Michael Still proposed openstack/nova master: libvirt: direct SPICE console object changes https://review.opendev.org/c/openstack/nova/+/926876 | |
| 06:46:09 | opendevreview | Michael Still proposed openstack/nova master: libvirt: allow direct SPICE connections to qemu https://review.opendev.org/c/openstack/nova/+/924844 | |
| 06:46:09 | opendevreview | Michael Still proposed openstack/nova master: libvirt: direct SPICE console database changes https://review.opendev.org/c/openstack/nova/+/926877 | |
| 06:59:09 | mikal | I have updated the tracking etherpad at https://etherpad.opendev.org/p/nova-2025.1-status with a brain dump of the current state of the various SPICE VDI patches. | |
| 08:25:37 | opendevreview | benlei proposed openstack/nova master: Abort live migration task when stop nova compute service https://review.opendev.org/c/openstack/nova/+/938223 | |
| 08:50:56 | artem_vasilyev | Hey, could you review a small linter and test fixes for macOS support, please https://review.opendev.org/c/openstack/nova/+/937727 | |
| 10:20:15 | bauzas | dansmith: Uggla: thanks for the reviews on https://review.opendev.org/c/openstack/nova/+/940642 | |
| 10:27:26 | Uggla | bauzas, you are welcome. | |
| 10:48:26 | sean-k-mooney | bauzas: i just left my comment too i dont partically like the approch you took | |
| 10:48:43 | sean-k-mooney | it seems complex for what your trying to do | |
| 10:57:21 | bauzas | sean-k-mooney: thanks, I'll look over them after lunch | |
| 10:58:07 | bauzas | sean-k-mooney: your idea of getting all the props by a object property is actually a good one | |
| 10:59:43 | sean-k-mooney | i ocationally have them | |
| 11:00:21 | bauzas | for your algorithm with union/intersection, let's discuss this later, as I need to explain correctly why it wasn't working when I tested them | |
| 11:00:28 | sean-k-mooney | i check ovo and it does not have a funciton for that but they do it often internally. i woudl consider adding set_filed ot novas base object class | |
| 11:01:14 | sean-k-mooney | sure. form my perspecive the cardinatly shoudl not matter | |
| 11:01:47 | sean-k-mooney | i.e. if 2 isntance request the image property on one host adn one instnace request it on anohter shoudl not affect the weight | |
| 11:02:00 | sean-k-mooney | i think you are trying to also include that by using counter right? | |
| 11:02:58 | sean-k-mooney | we can chat later but i was intentionly trying not to have the number of vms on a host affect teh weighing of the host | |
| 11:03:21 | bauzas | I tested it, and the problem is that for example, if you have a host that has already 2 instances and one other that only has 1 using the prop, then it would prefer the second | |
| 11:04:08 | bauzas | so I preferred to just return the number of how many asked properties that are alrady used in the host | |
| 11:04:20 | bauzas | this is simplier to calculate and understand | |
| 11:04:45 | sean-k-mooney | i think that is more complex | |
| 11:04:54 | bauzas | also, I saw with devstack that when you create an instance, you get more props that the one you asked | |
| 11:04:56 | sean-k-mooney | and harder ot understand | |
| 11:05:14 | sean-k-mooney | you do yes we set some like the machine type on boot | |
| 11:05:19 | sean-k-mooney | or config drive | |
| 11:05:36 | bauzas | indeed, but you haven't asked for it, that's my point | |
| 11:05:39 | sean-k-mooney | based on the host config. we will be adding the vtpm secrete type | |
| 11:05:53 | sean-k-mooney | you have not but we shoudl not exclude them | |
| 11:06:15 | bauzas | anyway, I need to go off | |
| 11:06:19 | sean-k-mooney | ack | |
| 12:20:41 | opendevreview | sean mooney proposed openstack/nova master: Dont deploy n-spice on compute nodes. https://review.opendev.org/c/openstack/nova/+/940873 | |
| 12:20:41 | opendevreview | sean mooney proposed openstack/nova master: [WIP] move nova-ovs-hybrid-plug to deploy with spice and fix qxl default https://review.opendev.org/c/openstack/nova/+/940835 | |
| 13:15:49 | opendevreview | suiong ng proposed openstack/nova master: Fix parameter order in add_instance_info_to_node https://review.opendev.org/c/openstack/nova/+/939411 | |
| 13:53:54 | opendevreview | sean mooney proposed openstack/nova master: allow discover host to be enabled in multiple schedulers https://review.opendev.org/c/openstack/nova/+/938523 | |
| 13:54:31 | sean-k-mooney | gibi: i just dropped the job change form ^ | |
| 13:55:48 | sean-k-mooney | if you have time to look at https://review.opendev.org/c/openstack/nova/+/939476 as well it would be nice to land that | |
| 13:59:20 | sean-k-mooney | ... | |
| 13:59:27 | sean-k-mooney | mikal: "libvirt.libvirtError: unsupported configuration: spice graphics are not supported with this QEMU" | |
| 13:59:58 | sean-k-mooney | it looks like ubuntu 24.04 didnt just remove qxl they fully removed spice supprot like rhel9 | |
| 14:00:32 | sean-k-mooney | i think, debian still supprots it | |
| 14:01:21 | sean-k-mooney | so i could move that job to debian but if centos/rhel 9 and ubuntu 24.04 have fully drop spice that proably our last option to test it | |
| 14:01:58 | sean-k-mooney | kolla uses debian continer on all distos now so i woudl guess that is why its workign for you in kolla-ansible | |
| 14:02:24 | sean-k-mooney | that our your still using ubuntu 22.04? | |
| 15:04:13 | ykarel | Hi is the issue a known one where test fails randomly with libvirt.libvirtError: Failed to terminate process 70698 with SIGKILL: Device or resource busy ? | |
| 15:04:46 | ykarel | https://0c63ab9652170854bf26-a09d1a3317eb4b9b558e42ad19c25861.ssl.cf2.rackcdn.com/940474/1/gate/neutron-ovs-tempest-multinode-full/5184c7f/testr_results.html | |
| 15:04:46 | ykarel | like seen in | |
| 15:04:51 | ykarel | https://97ad4d1320a89f3380ef-01f5fc3a5734547a13a0f54725d40b32.ssl.cf5.rackcdn.com/936364/4/gate/nova-multi-cell/3003ee6/testr_results.html | |
| 15:06:23 | ykarel | found a quite old related patch https://review.opendev.org/c/openstack/nova/+/639091 but that was targetting a libvirt version and it got removed from nova now as that old versions no longer supported | |
| 15:35:42 | sean-k-mooney | ykarel: rerely teh qemu monitor process that is responisble for respondind to the howdown of the guest can become unresponsive | |
| 15:36:38 | sean-k-mooney | but htis is not realy a nova issue its more of a libvirt/qemu one | |
| 15:36:55 | sean-k-mooney | nova should not need to actuly retry the destroy at all | |
| 15:38:52 | ykarel | sean-k-mooney, ohk so it's a known issue? what is the mitigation for this then? | |
| 15:39:05 | ykarel | the operator/user have to retry the operation? | |
| 15:39:10 | sean-k-mooney | not really | |
| 15:39:32 | sean-k-mooney | its a very rare failure mode in qemu to exit gracefully when ask | |
| 15:39:51 | sean-k-mooney | libvirt tried to fix this by sendign sigkill instead of just sigterm | |
| 15:39:59 | sean-k-mooney | so the os would evenutally reap the process | |
| 15:40:45 | sean-k-mooney | if sigkill does not work there nothing nova can do to force it to exit really | |
| 15:42:00 | ykarel | ohkk got it, but why nova cannot do retry like done in above patch if response is ebusy ? | |
| 15:43:51 | sean-k-mooney | libvirt is ment to do it internally | |
| 15:45:31 | ykarel | ohkk but as per logs thats not helping as there are failures in CI | |
| 15:46:05 | ykarel | ohh or you mean libvirt should do it but it's not doing it and libvirt have to be fixed for that? | |
| 15:46:27 | sean-k-mooney | its either not doing it or it is doing it and qemu is still not exiting | |
| 15:47:52 | ykarel | ohkk no idea then where the issue is, i will start with nova bug and other points could be checked as part of it | |
| 15:48:01 | ykarel | thx sean-k-mooney for sharing all the insights | |
| 15:48:13 | sean-k-mooney | i dont think you shoudl start with a nova bug | |
| 15:48:43 | sean-k-mooney | i also think we sthil have some rety logic because _destory does call itesl https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1612 | |
| 15:48:49 | ykarel | nova people may know more about libvirt/qemu bits so can judge more then me so i thought to start with it :) | |
| 15:51:11 | sean-k-mooney | i would not personaly consider it a nova bug but other might disagree | |
| 15:52:45 | sean-k-mooney | the orgianl issue was not a nova bug it was a workaround for a libvirt/qemu one | |
| 15:55:03 | sean-k-mooney | ykarel: as far as i can see we actuly retry infinetly https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1603 | |
| 15:56:56 | sean-k-mooney | im not a huge fan of who this is written but in destroy we are using loopingcall form oslo.service to execute _wait_for_destroy which checks if the instnace is stil runing. if it sets kwargs['is_running'] = True | |