| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-01-16 | |||
| 14:13:31 | bauzas | gibi: cool, I'll look | |
| 14:19:49 | opendevreview | Bogdan Dobrelya proposed openstack/nova master: Run nova-next without periodic cache healing https://review.opendev.org/c/openstack/nova/+/939453 | |
| 14:23:08 | opendevreview | Merged openstack/nova-specs master: Fix a line that was not removed before merging the specification https://review.opendev.org/c/openstack/nova-specs/+/939380 | |
| 14:24:53 | opendevreview | Merged openstack/os-traits master: Add 'HW_PCI_LIVE_MIGRATABLE' https://review.opendev.org/c/openstack/os-traits/+/939264 | |
| 15:04:08 | gibi | Uggla: ^^ you can propose a os-traits release now :) | |
| 17:08:33 | Uggla | gibi, yep I will do that. | |
| 17:46:14 | melwitt | sean-k-mooney, stephenfin: fyi the revert of the libvirt tpool proxy patch merged and I have reproposed the patch with unit tests added here https://review.opendev.org/c/openstack/nova/+/939317 if you might have a chance to re-review at your convenience | |
| 17:46:28 | stephenfin | already done | |
| 17:46:44 | melwitt | thanks stephenfin | |
| 17:47:04 | sean-k-mooney | cool ill take a look shortly | |
| 17:47:23 | sean-k-mooney | im runnging the funtional test locally and my laptop is a littel laggy right now | |
| 17:48:17 | melwitt | thanks stephenfin | |
| 17:48:21 | melwitt | argh | |
| 17:48:23 | melwitt | thanks sean-k-mooney | |
| 17:49:04 | sean-k-mooney | load average of 20 oin 8cores.16 thread... its fine.... | |
| 17:49:46 | sean-k-mooney | 22, the fact its going up is definally not an issue, nope nope nope :) | |
| 18:06:36 | melwitt | 😳 | |
| 18:08:25 | opendevreview | sean mooney proposed openstack/nova master: disable the heal instance info cache periodic https://review.opendev.org/c/openstack/nova/+/939476 | |
| 18:10:48 | sean-k-mooney | melwitt: two questions, since we are redoing https://review.opendev.org/c/openstack/nova/+/939317 do we want a release note too? second question is i have readded +2 but do we want to wait for dan to review before +w | |
| 18:11:40 | sean-k-mooney | melwitt: there isnt anything required of the operator so we dont technially need a release not i was just wondering if you think its important enough to call out as a fixes note or not | |
| 18:15:42 | sean-k-mooney | the upstream bug does not have an improants set althoguh the downstream ones you linked seam to be high/medium | |
| 18:15:55 | sean-k-mooney | so it might be worth callign out its up to you however | |
| 18:16:36 | melwitt | sean-k-mooney: hm, I hadn't considered that. I'm not sure how commonly people hit the issue but if they do hit it, I would consider it a pretty significant issue and worth a reno. and yeah probably a good idea to have dansmith look at it before +Wing | |
| 18:17:19 | dansmith | yeah it's probably worth one because if people don't know why their stuff is locking up, they're not going to be looking for something specific | |
| 18:17:46 | dansmith | since these tend to hang us in the middle of something that doesn't go into the logs as "about to do something that might hang, if so, check like 123 in the file" | |
| 18:18:43 | melwitt | makes sense. I'll write one up and push an update | |
| 18:49:26 | artom | dansmith, remind me, what was your patch that enabled direct LUN passthrough? | |
| 18:49:44 | artom | (What I really want to know is in what release it went out, but I can figure that out on my own if I know the patch | |
| 18:49:46 | artom | ) | |
| 18:49:58 | artom | Is it https://review.opendev.org/c/openstack/nova/+/918089? | |
| 18:50:06 | sean-k-mooney | there are two patches | |
| 18:50:29 | sean-k-mooney | but that is one of them | |
| 18:50:35 | dansmith | yeah that's the second one I think | |
| 18:50:59 | sean-k-mooney | you can see them in the serice https://review.opendev.org/c/openstack/nova/+/918470/2 is the other | |
| 18:51:30 | sean-k-mooney | this depend on another serise however | |
| 18:51:42 | sean-k-mooney | which was the device alisa feature | |
| 18:52:08 | sean-k-mooney | https://review.opendev.org/q/topic:%22bp/libvirt-dev-alias%22 | |
| 18:53:07 | sean-k-mooney | @artom: tldr the lun type cant have a serial embeded os we need to be able to detach/attach volumes based on the alias instead | |
| 18:53:24 | artom | I remember the alias -> direct LUN passthrough dependency chain | |
| 18:53:49 | artom | This is just to answer an email internally - tl;dr, it's not in our released product yet | |
| 18:54:02 | sean-k-mooney | correct | |
| 19:03:17 | sean-k-mooney | the detach via alias change set was considred an upstream freature to use the new libvirt api, that merged in 2024.1(caracal) the lun type suport is considered a bugfix and was merged in 2024.2 dalmatin | |
| 19:03:29 | sean-k-mooney | so tha tcould be backported ot caracal upstream if really wanted | |
| 19:13:52 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Wrap un-proxied listDevices() and listAllDevices() https://review.opendev.org/c/openstack/nova/+/939317 | |
| 19:18:19 | gmann | bauzas: hi, can you check this, let me know exactly where (ML or meeting) I need to propose exception if needed https://review.opendev.org/c/openstack/nova-specs/+/937650 | |
| 19:24:21 | opendevreview | sean mooney proposed openstack/nova master: disable the heal instance info cache periodic https://review.opendev.org/c/openstack/nova/+/939476 | |
| 19:49:22 | opendevreview | Mengyang Zhang proposed openstack/nova master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova/+/939490 | |
| 20:13:28 | opendevreview | Mengyang Zhang proposed openstack/nova master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova/+/939310 | |
| 20:15:27 | opendevreview | Mengyang Zhang proposed openstack/nova master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova/+/939490 | |
| 20:26:30 | opendevreview | Mengyang Zhang proposed openstack/nova-specs master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova-specs/+/932653 | |
| 20:43:19 | opendevreview | Mengyang Zhang proposed openstack/nova-specs master: Add Burst Length Support to Cinder QoS https://review.opendev.org/c/openstack/nova-specs/+/932653 | |
| #openstack-nova - 2025-01-17 | |||
| 11:08:12 | opendevreview | Stephen Finucane proposed openstack/nova master: libvirt: Wrap un-proxied listDevices() and listAllDevices() https://review.opendev.org/c/openstack/nova/+/939317 | |
| 13:54:48 | gibi | folks, what is a good way to change a multinode zuul job (nova-next) to have a different placement config than default? I cannot just set vars:devstack_local_conf as that fails as it tries to be applied on all nodes and computes has no placement service. Setting it is a group-vars that controller only does not work either as then I lose what is already set via vars. | |
| 13:54:56 | gibi | context https://review.opendev.org/c/openstack/nova/+/937275 | |
| 13:55:48 | gibi | do I need to duplicate the settings from vars to group-vars? | |
| 13:56:18 | dougszu | Has anyone seen ephemeral volumes corrupted after a VM is migrated? I see Nova create the filesystem on the backing disk file during instance create, but then regenerate the backing disk on the destination HV. It leads to the UUID changing in the backing disk file, and the VM sees the filesystem as corrupted (for a subset of filesystems which are good spotting the anomaly). | |
| 13:56:50 | dougszu | *good at spotting | |
| 13:57:02 | dougszu | happy to create a bug and work on it, if there isn't something already | |
| 14:04:18 | opendevreview | Balazs Gibizer proposed openstack/nova master: [CI][nova-next]test with placement ac breadth-first https://review.opendev.org/c/openstack/nova/+/937275 | |
| 14:20:29 | sean-k-mooney | gibi: so i can think of a few hacks but i tought we fixed this in the past | |
| 14:21:05 | sean-k-mooney | gibi: there was a patch propsoed before to make sure hta tbefore the post-config code ran it would do a make -p on the partent path | |
| 14:21:50 | sean-k-mooney | the hack is in the devstack_local_conf section you could jsut add that | |
| 14:21:59 | sean-k-mooney | sorry mkdir -p | |
| 14:24:57 | gibi | sean-k-mooney: still there is no placement.conf to edit on the computes so mkdir -p alone is not enough | |
| 14:25:21 | sean-k-mooney | dougszu: so nova will redownload the backing image form glance and copy it form the souce if the image is not deleted | |
| 14:25:29 | sean-k-mooney | *is deleted | |
| 14:25:47 | sean-k-mooney | gibi: the file shoudl nto be required to exist | |
| 14:25:51 | sean-k-mooney | it shoudl create it if its not there | |
| 14:26:13 | sean-k-mooney | gibi: at least that what my memory says is the expected behavior | |
| 14:26:25 | sean-k-mooney | im trying to find this in the devstack repo now | |
| 14:26:44 | dougszu | sean: what I don't see it the attempt to download the backing file - perhaps it is a misconfiguration. I've filed this with steps: https://bugs.launchpad.net/nova/+bug/2095173 | |
| 14:28:09 | sean-k-mooney | oh sorry you mean ephmeral storage provisin via the flavor | |
| 14:28:31 | sean-k-mooney | so for live migration we copy the data | |
| 14:28:38 | sean-k-mooney | for cold migration we dont as far as i recall | |
| 14:29:17 | sean-k-mooney | i woudl need to check but we do not expect the content of the ephemral disks to be preseved for a number of operations | |
| 14:29:49 | sean-k-mooney | resize is oen of them i dont know about cold-migration. live migration is intended to preserve the data | |
| 14:30:31 | dougszu | Thanks - I see the ephemeral data copied for both live and cold. The bug affects both | |
| 14:31:08 | sean-k-mooney | so the content o fthe backing file sholud not really effect the resulting image since its ment to be blank | |
| 14:31:20 | sean-k-mooney | but i think i know why this is happening | |
| 14:31:54 | dougszu | Yeah, so I think you can configure it that way - to not create a FS on the ephemeral drive. But if you do tell Nova to create FS, it will create it on the backing file. | |
| 14:31:54 | sean-k-mooney | i think we are currently formating the backign file if you have not disabeld that | |
| 14:32:19 | sean-k-mooney | dougszu: right os we have been disucssint removign the ablity for nova to format the ephemral storage | |
| 14:32:49 | sean-k-mooney | we planned to disabel it by default and then deprecate it but i dont think we have doen that yet | |
| 14:33:05 | sean-k-mooney | for the ephmeral disk there is no backign file in glance | |
| 14:33:38 | sean-k-mooney | so the only optionon cold or live migrate woudl be to copy it from the souce node otherwise we would geneate a new one on the dst with no refernce to the old one a diffent uuid as a result | |
| 14:33:48 | dougszu | curious -yeah, that would work, it forces the FS into the top layer | |
| 14:33:54 | dougszu | and fixes the issue | |
| 14:34:07 | dougszu | but not for existing VMs :D | |
| 14:34:23 | gibi | sean-k-mooney: so you suggest that devstack_local_conf post-config should work even if the referenced service is not deployed and the config file is not there as it i) creates the missing dir ii) generates a new file with the post-config content if the file does not exits? | |
| 14:34:27 | sean-k-mooney | right so evernayully we also dont want to have backign files for test disks ata ll | |
| 14:34:52 | sean-k-mooney | the fact they have them is a sideffect of hwo the qcow dirver is currently impelmetned and not really intentional | |
| 14:35:17 | sean-k-mooney | gibi: that is how i think it was orgianly inteded to work yes | |
| 14:35:33 | sean-k-mooney | or at least that is how i would have implemtned it | |
| 14:35:47 | dougszu | You also can't copy the backing file from the source node. Eg. if you had two identical VMs on different HVs, and migrated them both to another HV, they would fight over the backing file. | |
| 14:36:08 | sean-k-mooney | dougszu: you can but you need ot have a diffent name for it | |
| 14:36:20 | gibi | sean-k-mooney: ack, I can also try to dig devstack... | |
| 14:36:25 | gibi | based on thsi | |
| 14:36:27 | gibi | this | |
| 14:37:06 | dougszu | yeah - agree on that - different name would work | |
| 14:37:21 | sean-k-mooney | dougszu: but ya we dont generate a repoducable file system on the ephmale backign file today as far as im aware. | |