| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-08-13 | |||
| 16:45:24 | artom | OK, that keeps thing tidy | |
| 16:47:03 | gibi | artom: at a first read what you proposed seems OK to me. I need to look at it once more with a fresh set of eyes probably after we have some test results too | |
| 16:47:25 | artom | For sure :) Good to know that nothing obvious stands out | |
| 16:47:43 | artom | And there's a whitebox DNM patch running, so that should weed out any issues | |
| 16:53:52 | sean-k-mooney | artom: you proably should not be touching cores outside fo cpu_dedicated_set | |
| 16:54:39 | sean-k-mooney | we only powermanage cores in cpu_dedicated_set and its a hard error for cores in cpu_shared_set to be offline | |
| 16:55:03 | sean-k-mooney | but any cores out side of those two sets are valid for the admin to ower on/off as they desire | |
| 16:55:05 | artom | Yeah... I guess the only edge case where it might make sense is if the host has been trained, and nova-compute restarted to change the cpu_dedicated_set | |
| 16:55:08 | sean-k-mooney | as nova is not ment to use them | |
| 16:55:17 | artom | s/trained/drained/ | |
| 16:55:40 | artom | And so some of the offline cores could be outside the _current_ dedicated set, but part of the _previous_ dedicated set | |
| 16:55:48 | sean-k-mooney | well not you missing the fact that core outside fo cpu_shared_set adn cpu_dedicated_set can be used for other usecases | |
| 16:56:43 | sean-k-mooney | artom: so it would be incorrect for use to turn on any cores not in the currnt cpu_dedicated_set | |
| 16:57:11 | sean-k-mooney | as long as the offline cores are not added to cpu_shared_set we dont care if they were formally in cpu_dedicated_set | |
| 16:57:28 | artom | We do if they're offline and we read their socket as 0 :) | |
| 16:57:34 | sean-k-mooney | no we dont | |
| 16:57:40 | sean-k-mooney | because nova will never use them | |
| 16:58:23 | artom | Oh... yeah? NUMA affinity but with shared CPUs is not a thing? | |
| 16:58:43 | sean-k-mooney | it is but we have a sperat check for cpu_shared_set to ensure they are online | |
| 16:58:59 | sean-k-mooney | we dont allow you to have any offlien cores in cpu_shared_set | |
| 16:59:25 | artom | Ah ok, in that case it's even tidier, I don't need the new power_up_unconditionally() | |
| 17:00:34 | sean-k-mooney | right you can resue the existing _power_up although you might want to drop the _ or add a "public" wraper | |
| 17:03:28 | artom | Yeah | |
| 17:48:58 | opendevreview | Merged openstack/nova master: Functional test test_boot_reschedule_with_proper_pci_device_count https://review.opendev.org/c/openstack/nova/+/760354 | |
| 18:09:42 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: Work around lbvirt reporting offline cores as being on socket 0 https://review.opendev.org/c/openstack/nova/+/926218 | |
| 18:33:02 | opendevreview | sean mooney proposed openstack/nova master: Skip new image format tests https://review.opendev.org/c/openstack/nova/+/926215 | |
| #openstack-nova - 2024-08-14 | |||
| 07:16:18 | opendevreview | Tobias Urdin proposed openstack/nova master: Fix retried resize operation failing with FileNotFound https://review.opendev.org/c/openstack/nova/+/926251 | |
| 07:29:02 | opendevreview | Tobias Urdin proposed openstack/nova master: libvirt: Fix retried resize operation failing with FileNotFound https://review.opendev.org/c/openstack/nova/+/926251 | |
| 13:10:10 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | |
| 15:39:40 | opendevreview | Viktor Křivák proposed openstack/nova master: Ensure claims to placement are freed after evacuate complete Improve resource_tracker delete_allocation_for_evacuated_instance to handle non-cached compute hosts Add test to prevent regression https://review.opendev.org/c/openstack/nova/+/926292 | |
| 16:07:09 | opendevreview | Markus Hentsch proposed openstack/nova master: Use os-brick for volume attachment passphrases https://review.opendev.org/c/openstack/nova/+/926326 | |
| 16:09:29 | opendevreview | Elod Illes proposed openstack/python-novaclient master: DNM: gate health test https://review.opendev.org/c/openstack/python-novaclient/+/926334 | |
| 17:02:49 | sean-k-mooney | gibi: i think responding to https://review.opendev.org/c/openstack/nova/+/710848 ate what little brain power i had for today but i tried to find where the save was happeing im just not sure which path it is | |
| 17:03:11 | sean-k-mooney | i think its the call to the conductor that trigering the save with the refeshed content | |
| 17:03:47 | sean-k-mooney | artom: fyi ^ | |
| 17:04:26 | sean-k-mooney | we could try moving the refresh to here https://review.opendev.org/c/openstack/nova/+/710848/6/nova/compute/manager.py#2371 | |
| 17:04:53 | sean-k-mooney | which i think would result in it being saved here instead https://review.opendev.org/c/openstack/nova/+/710848/6/nova/compute/manager.py#2260 | |
| 17:05:28 | sean-k-mooney | which would be better i think | |
| 17:05:42 | sean-k-mooney | can you try that locally | |
| 17:05:52 | artom | sean-k-mooney, I saw the comments but since I didn't write the actual patch I have to start from scratch to figure out what's going on | |
| 17:06:41 | sean-k-mooney | ok im going to finish in 5 mins but ill try to just move that locally and see if it passes your test case | |
| 17:37:02 | sean-k-mooney | ... we have a seperate sqlalchemy 2.0 issue | |
| 17:44:18 | opendevreview | sean mooney proposed openstack/nova master: test fixing this when allocation fails. https://review.opendev.org/c/openstack/nova/+/926350 | |
| 17:44:34 | sean-k-mooney | artom: ^ passes locally if i run the funciontal tests | |
| 17:44:40 | sean-k-mooney | but the serise also need a rebase | |
| 17:45:18 | sean-k-mooney | ... | |
| 17:45:30 | sean-k-mooney | or you know i could not do that on patchset 6... | |
| 17:45:49 | artom | Yeah, that relation chain is... special :) | |
| 17:48:20 | opendevreview | sean mooney proposed openstack/nova master: test fixing this when allocation fails. https://review.opendev.org/c/openstack/nova/+/926350 | |
| 17:49:41 | sean-k-mooney | ok that should be better | |
| 17:50:07 | sean-k-mooney | gibi: ^ i think that is a better fix | |
| 17:50:29 | sean-k-mooney | ... or it would be if it didnt fail | |
| 17:53:46 | sean-k-mooney | ok im not sure why that didnt work but it did not so im just not goign to look at that today | |
| 17:54:38 | opendevreview | Merged openstack/nova master: Skip new image format tests https://review.opendev.org/c/openstack/nova/+/926215 | |
| 17:55:04 | sean-k-mooney | oh actully i think i do no | |
| 17:55:14 | sean-k-mooney | that not where the excption is being raised | |
| 17:59:10 | opendevreview | sean mooney proposed openstack/nova master: test fixing this when allocation fails. https://review.opendev.org/c/openstack/nova/+/926350 | |
| 17:59:25 | sean-k-mooney | ^ actully passes locally | |
| 18:06:41 | sean-k-mooney | artom: gibi i have left some comments to explain why i think this also work and is cleaner but also why the patch i have is still not ideal | |
| #openstack-nova - 2024-08-15 | |||
| 01:24:41 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | |
| 01:26:30 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Launch instances with SEV-ES memory encryption https://review.opendev.org/c/openstack/nova/+/926106 | |
| 08:40:05 | gibi | sean-k-mooney: artom: I'm still in the process of understanding the logic of the relationship between the instance.pci_devices and the pci_tracker state. At some point the acconting of the two got out of sync and I think that is the place we need to fix | |
| 08:40:15 | gibi | the refresh is just a band aid | |
| 09:03:18 | zigo | bauzas: Hi there! Are you aware of this problem? https://bugs.launchpad.net/nova/+bug/2077070 | |
| 09:03:37 | zigo | Is Sylvain in holidays btw? :) | |
| 09:13:55 | frickler | isn't all of france in holiday during august? but this sounds related to what sean-k-mooney and gibi were discussing earlier at least in as much as pci devices are affected | |
| 09:44:36 | gibi | sean-k-mooney: artom: https://review.opendev.org/c/openstack/nova/+/710848/13#message-354946be57b39eb88b67872230c226361d8a3559 this is where I am at. The resource trackker correctly removes the device from the instance.pci_devices list during the aborting of the claim, but that update is lost when we return to the compute manager | |
| 09:49:49 | sean-k-mooney[m] | zigo is the compute node marked as deleted | |
| 09:49:53 | sean-k-mooney[m] | we dont actully delete the row | |
| 09:50:16 | gibi | sean-k-mooney: artom: yeah Claim ctor clones the instance https://github.com/openstack/nova/blob/a7c82399b237aff44810ac76b0c4d10416c3bdf9/nova/compute/claims.py#L65-L66 | |
| 09:51:14 | sean-k-mooney[m] | gibi: im going to gab a coffee but ill be back in 5 mins and ill take a look | |
| 10:01:33 | sean-k-mooney | ok actull | |
| 10:03:34 | gibi | abort claim logic working on a copy of the instance so the cleanup of the instance is lost when nova returns to the compute manager | |
| 10:04:04 | sean-k-mooney | do you want to to jump on a google meet and debug this in 10 mins. that sound plausible yes looking at the clone. | |
| 10:04:16 | gibi | sure I can | |
| 10:04:27 | sean-k-mooney | i belive we do that to make sure we dont modify the instnace object during the claim | |
| 10:04:43 | sean-k-mooney | btu i guess we did in one path and didnt in another | |
| 10:04:48 | sean-k-mooney | and the two are out of sync | |
| 10:05:13 | gibi | should we wait for artom? | |
| 10:05:19 | gibi | maybe he is interested too | |
| 10:05:21 | sean-k-mooney | we can | |
| 10:05:52 | gibi | then I will grab a quick lunch, and let you look at what I found in the meantime | |
| 10:06:04 | gibi | and we can jump on a call when artom is arong | |
| 10:06:07 | gibi | arond | |
| 10:06:11 | sean-k-mooney | i didnt sleep well last night (didnt get to sleep until 4am) so im just startign my day now so wanted to skim my email first while having my morning coffee | |
| 10:06:22 | sean-k-mooney | so we can chat when they are onlien | |
| 10:06:33 | gibi | OK | |
| 10:15:11 | sean-k-mooney | frickler: and yes that is a differnt topic. we have a very old bug, with an eaislly old bugfix that "works" but its fixing the sideefect rather then the cause and we are tryign to under stand what is the cuase and how to move forward | |
| 10:16:05 | zigo | sean-k-mooney: I'm not sure, I haven't checked for that, you're probably right that it just stays as deleted, but my bug still stand. | |
| 10:16:30 | sean-k-mooney | zigo: i have not looked at your bug yet :) | |
| 10:17:04 | sean-k-mooney | openstack compute service delete <uuid> is deleting the comptue service not the compute node by the way | |
| 10:17:18 | sean-k-mooney | it may do one as a side effect of the other but those are two logically differnt things | |
| 10:17:59 | sean-k-mooney | zigo: are there any instance on the compute node at the time your deleting the compute service | |
| 10:18:10 | sean-k-mooney | espically any that are using pci devices | |
| 10:18:43 | zigo | sean-k-mooney: I had a case where that was the issue, but in another case, no, there was no instance at all, even less with a PCI device attached. | |
| 10:20:31 | sean-k-mooney | zigo so reinstalling a node because of a hardware issue would not normally invovle deleting the comppute service | |
| 10:20:35 | sean-k-mooney | is there a reason you did that | |
| 10:20:54 | sean-k-mooney | im not saying we should not look into the clean up | |
| 10:21:12 | zigo | It's just our procedure to clean-up nova, cinder and neutron services when we decomission a node... | |
| 10:21:17 | sean-k-mooney | but the procudure yoru takign is not what i would expect/recommend if you are replacing a node bug keeping the host name | |