| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-08-29 | |||
| 11:41:43 | sean-k-mooney | if you spcify the mtu in the xml it changes form 32bit to 64bit | |
| 11:41:59 | sean-k-mooney | which is also nto supproted by qemu during a live migration | |
| 11:42:19 | sean-k-mooney | so if the xml does nto have an mtu we cant add it durign a live migration | |
| 11:42:32 | gibi | I see, so this is the case ^^ | |
| 11:42:42 | gibi | nova tries to add it | |
| 11:43:14 | sean-k-mooney | that why we orginally started down the line of grab the mtu form the port and store it then generate the new config and restore the mtu | |
| 11:43:33 | sean-k-mooney | gibi: yep so we fixed this once before after i acindeltely broke vexhost | |
| 11:45:32 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/553072 | |
| 11:45:54 | sean-k-mooney | that was the change to set the mtu in the first palace (its also neede for jumbo frames) | |
| 11:46:12 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1747496 | |
| 11:47:05 | sean-k-mooney | that shoudl result in ports being attached with an mtu as well | |
| 11:47:39 | sean-k-mooney | gibi: the issue they are reproting in https://bugs.launchpad.net/nova/+bug/2080531 | |
| 11:47:42 | opendevreview | Merged openstack/placement master: Remove the job with Ubuntu 22.04 https://review.opendev.org/c/openstack/placement/+/958820 | |
| 11:47:50 | sean-k-mooney | i think artom had satred to fxi a while ago | |
| 11:47:54 | sean-k-mooney | let me see if i can find that | |
| 11:48:30 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Move regex helpers to utils https://review.opendev.org/c/openstack/nova/+/958745 | |
| 11:48:31 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Simplify API version check for flavor description https://review.opendev.org/c/openstack/nova/+/958747 | |
| 11:48:31 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Clean up flavors tests https://review.opendev.org/c/openstack/nova/+/958746 | |
| 11:48:32 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Remove dead fields from flavors response https://review.opendev.org/c/openstack/nova/+/958749 | |
| 11:48:32 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add ability to filter flavors by name https://review.opendev.org/c/openstack/nova/+/958748 | |
| 11:48:33 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Fix typo https://review.opendev.org/c/openstack/nova/+/958855 | |
| 11:48:33 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Restrict additional query string arguments https://review.opendev.org/c/openstack/nova/+/958750 | |
| 11:48:34 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Add missing test coverage https://review.opendev.org/c/openstack/nova/+/958857 | |
| 11:48:34 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add runtime check for query additionalProperties https://review.opendev.org/c/openstack/nova/+/958856 | |
| 11:48:36 | opendevreview | Stephen Finucane proposed openstack/nova master: WIP: api: Add runtime check for general additionalProperties https://review.opendev.org/c/openstack/nova/+/958858 | |
| 11:48:46 | sean-k-mooney | gibi: https://review.opendev.org/c/openstack/nova/+/791553/1 | |
| 11:49:05 | sean-k-mooney | so we have a regression test https://review.opendev.org/c/openstack/nova/+/791235/3 | |
| 11:49:11 | sean-k-mooney | then then the poc to fix it | |
| 11:49:48 | sean-k-mooney | and there as a seocnd attept to fix it in https://review.opendev.org/c/openstack/nova/+/852367/1 | |
| 11:50:29 | sean-k-mooney | gibi: these all kind of stalled out partly because of review banwith | |
| 11:51:53 | gibi | I'm not sure if simply checking the MTU does not change will help if we don't even have the MTU on the source host to begin with. So probably multiple fixes are needed. Anyhow I will let the downstream reported file a bug on us to get motivation to fix it | |
| 11:52:12 | gibi | s/reported/reporter/ | |
| 11:52:15 | sean-k-mooney | ack | |
| 11:52:20 | sean-k-mooney | ya so there are a few issues | |
| 11:52:34 | sean-k-mooney | i might actilly have an old pathc for the mtu not being set | |
| 11:54:36 | sean-k-mooney | i think the best way forward is 1 letg create some functional repoduces for the diffent senairos 2 then fix the mtu setting on attach and then on migrate as followups | |
| 11:55:21 | gibi | yepp I agree | |
| 11:55:37 | sean-k-mooney | the patch you linked migth fix part of it i woudl jsut need to trace throught he code to see why | |
| 11:56:13 | sean-k-mooney | its not obvious that it should | |
| 11:58:40 | sean-k-mooney | by the way we also knwo that there is a bug related to this that applie to sriov port as well | |
| 11:59:32 | sean-k-mooney | basiclly wehn testing seriov live mgirtion i belvie we have to use 2 sriov prot (one mac vtap and one direct) to make it work instaed of ovs and macvtapo | |
| 12:00:13 | sean-k-mooney | i dont exactly recall the details but it has somethign to do with the mtu i think as well but we could check with james parker and they might have a link | |
| 12:00:24 | sean-k-mooney | we had ot work aroudn it in our ci | |
| 12:02:37 | gibi | uuu a workarond in CI sounds interesting | |
| 12:03:15 | gibi | the new report upstream involves SRIOV ports | |
| 12:03:31 | gibi | I guess | |
| 12:03:32 | sean-k-mooney | oh so ya | |
| 12:03:49 | sean-k-mooney | am if you have an ovs prot and a sriov prot in direct mode | |
| 12:04:16 | sean-k-mooney | live migration does not work because of somethign ot do with the mtu it hkn | |
| 12:04:34 | sean-k-mooney | i debuged it once but again dint have tiem to go fix it | |
| 12:06:17 | sean-k-mooney | ok reading the bug again ya this is the same issue we see in ci | |
| 12:07:38 | sean-k-mooney | but the netwrok data in the metadata api is not alwasy the same as in the network info cache | |
| 12:08:51 | sean-k-mooney | ill try and look at this review again properly next week | |
| 12:09:05 | sean-k-mooney | this is the first item i have seen that bug or patch | |
| 12:11:13 | gibi | OK. thanks | |
| 12:13:21 | sean-k-mooney | im partly nerd snips because i was going to start on sumething new but you pinged me before i start on it | |
| 12:14:45 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Remove unnecessary action method prefix https://review.opendev.org/c/openstack/nova/+/958863 | |
| 12:14:45 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Correct version for rebuild response https://review.opendev.org/c/openstack/nova/+/958862 | |
| 12:14:46 | opendevreview | Stephen Finucane proposed openstack/nova-specs master: Repropose remove-os-volumes_boot-api https://review.opendev.org/c/openstack/nova-specs/+/958865 | |
| 12:14:46 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Deprecate os-volumes_boot API https://review.opendev.org/c/openstack/nova/+/958864 | |
| 12:14:57 | gibi | sean-k-mooney: sorry for that. This is not urgent so feel free to punt it for a while | |
| 12:15:57 | opendevreview | Stephen Finucane proposed openstack/nova-specs master: Repropose flavor-search-by-name spec https://review.opendev.org/c/openstack/nova-specs/+/958866 | |
| 12:16:22 | sean-k-mooney | im not complaining but im going to spend 15 mins considerign the patch then move on. ill come back to it on monday | |
| 12:17:39 | sean-k-mooney | i think the change they are propsoing is safe to proceed with | |
| 12:18:58 | sean-k-mooney | i am not conviced it will fix all 2-3 of the related cases i am vagely aware of but i dont think it will make any of those harder to fix later | |
| 12:19:08 | opendevreview | Merged openstack/nova-specs master: Create specs directory for 2026.1 Gazpacho https://review.opendev.org/c/openstack/nova-specs/+/956166 | |
| 12:19:20 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Disable VMCoreInfo device for SEV-encrypted instances https://review.opendev.org/c/openstack/nova/+/958868 | |
| 12:20:56 | sean-k-mooney | gibi: i feel like it need a bit more testing but im tempeted to pull this into one of my devstacks and try it and maybe jsut take a good enough is better then perfect approch in this case | |
| 12:21:47 | sean-k-mooney | gibi: in other words we can proably proceed with this before rc1 but i want to poke at it a bit before then | |
| 12:22:10 | opendevreview | Stephen Finucane proposed openstack/nova-specs master: Repropose openapi spec for Gazpacho https://review.opendev.org/c/openstack/nova-specs/+/958869 | |
| 12:33:21 | gibi | sean-k-mooney: OK. Thanks | |
| 12:35:22 | stephenfin | gmaan: For next cycle, but you'll probably like https://review.opendev.org/c/openstack/nova/+/958856/ and https://review.opendev.org/c/openstack/nova/+/958858/ | |
| 12:35:42 | stephenfin | one less thing for you to worry about during reviews 😄 | |
| 12:37:40 | sean-k-mooney | gibi: i basiclly want to see if i can replciate this with 2 ovs network with differnt mtu or if it actully need ovs + sriov to see if we can maybe test this in tempest and if it obviously fixes the issue for me locally | |
| 12:39:05 | sean-k-mooney | gibi: i dont curertnly have sriov hardware avaibel to thise this simiply so i need to create 2 devstack vms using igb and set it up for live migration ectra | |
| 12:39:12 | sean-k-mooney | so i wont get to thtat today | |
| 12:39:55 | opendevreview | Takashi Kajinami proposed openstack/nova stable/2025.1: libvirt: Disable VMCoreInfo device for SEV-encrypted instances https://review.opendev.org/c/openstack/nova/+/958874 | |
| 12:44:52 | gibi | sean-k-mooney: sync with Arnau from neutron they migh have envs as this was reported from running neutron's whitebox against antelope | |
| 13:03:37 | sean-k-mooney | ack i mostly want to see if we can test this edge case in the 1st party ci going forward via tempest without the sriov depency described in the bug | |
| 13:04:11 | sean-k-mooney | if it can happen anytime you have a vm with 2 networks with diffent mtus for exampel we can easilly add a senario test in neutron to do that | |
| 13:05:45 | sean-k-mooney | like in theory if usign the neutron whitebox tempest plugin found it downstream that same test should have repodcuded it on master right | |
| 13:08:58 | gibi | yepp that theory make sense | |
| 13:09:33 | gibi | there was also recent changes in that repo about the exact same test case so it might pass on master but not on older whitebox versions | |
| 13:42:06 | dansmith | gibi: woohoo, thanks | |
| 13:43:00 | amorin | hey team, the call to attach a volume on nova is done through rpc. During this, the wsgi worker keep the connection open (it's a synchronous call). Do you know why it's a sync call and not an async one? | |
| 13:43:10 | amorin | see here https://opendev.org/openstack/nova/src/commit/07ab08aa69ae39644c276a0907759175f8936239/nova/compute/api.py#L5127 | |
| 13:43:19 | amorin | and there https://opendev.org/openstack/nova/src/commit/07ab08aa69ae39644c276a0907759175f8936239/nova/compute/api.py#L5150 | |
| 13:45:58 | dansmith | amorin: probably so that we delete the BDM if the attach fails | |
| 13:47:54 | amorin | hum, damn | |
| 13:49:18 | amorin | what if we send the attach action to the compute, answer to the API request, and then delegate all the hard work to conductor / compute? | |
| 13:54:31 | dansmith | I mean, anything is possible, but this used to be a situation where you got the device name back when you attached, which is not possible otherwise | |
| 13:54:47 | dansmith | why are you asking? I'm guessing you've got attach requests stacked up or something? | |
| 13:54:54 | amorin | exactly :) | |
| 13:54:59 | amorin | exhausting my apache2 workers | |
| 13:55:07 | amorin | and finally nova was answering 503 | |
| 13:55:20 | amorin | due to the fact that one of my ceph cluster was not healthy | |
| 13:55:37 | amorin | I am trying to figure out a way to avoid such behavior | |
| 13:56:39 | dansmith | in that case the attach is going to fail regardless right? | |
| 13:56:57 | dansmith | is the problem that compute gets totally hung on ceph, or that it waits too long for ceph? | |
| 13:57:02 | amorin | yes, it fails after the RPC timeout | |