| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-10-21 | |||
| 18:16:58 | sean-k-mooney | the issue is we cant do that with nested vir | |
| 18:17:08 | sean-k-mooney | i.e. we can only test it properly with baremental compute nodes | |
| 18:17:28 | sean-k-mooney | if you passhtough afully gpu to a vm you cant then use vgpus | |
| 18:17:44 | JayF | darn, too bad we don't know how to provision baremetal /s | |
| 18:18:01 | JayF | seriously though, having to provision baremetal is very tough in CI, it's part of why Ironic 3rd party CIs died out | |
| 18:18:04 | JayF | just a little high touch | |
| 18:18:25 | fungi | well, yeah they were talking about nova baremetal api to boot machines with vgpu-capable resources | |
| 18:18:31 | fungi | (i.e. ironic) | |
| 18:19:23 | fungi | but i agree with you about the whole safety challenge with providing root access to physical machines for untrusted job payloads | |
| 18:19:41 | JayF | it's not just a safety challenge, it's a challenge-challenge | |
| 18:19:58 | JayF | many real-world failure modes which needs human hands, or at least, non-automated intervention to fix | |
| 18:20:26 | sean-k-mooney | the tldr is when you passthough a fully pci device in qemu it removes all the sriov capablities because qemu did not have a way to make that work | |
| 18:20:58 | sean-k-mooney | for what its worth we are lookign to see if we can use our internal hardware in teh future to test specifi upstream changes | |
| 18:21:50 | sean-k-mooney | that woudl do a baremteally deployment of a nova compute (using ironic) with a manual triger to basicaly say "this is safe to test" | |
| 18:21:53 | sean-k-mooney | as a third party ci | |
| 18:22:52 | JayF | I think there's a lot of untapped value in that idea of on-demand testing | |
| 18:23:09 | JayF | we can *usually* pinpoint when we're getting close to an edge case that might need revalidation | |
| 18:23:20 | sean-k-mooney | but that is why we are also trying to figure out how to test without real hardware using the sample kernel dirvers for a baseline of coverage | |
| 18:24:05 | sean-k-mooney | for example we are enableing igb vif model in nova so tthat in the future we can use that to test sriov | |
| 18:24:40 | sean-k-mooney | if our providers can provide neutron ports with igb vif model in the future we can fully test sriov in the 1st party ci | |
| 18:25:16 | sean-k-mooney | unfrogually it needs a new enough nova/qemu but by 2026.1 we might be in a position to do that | |
| 18:25:32 | sean-k-mooney | *unfortunetly | |
| 19:04:37 | opendevreview | Artom Lifshitz proposed openstack/nova master: pwr mgmt: power up instances's CPUs upon init https://review.opendev.org/c/openstack/nova/+/932926 | |
| 19:54:32 | opendevreview | sean mooney proposed openstack/nova master: Add repoducer test for bug 2074219 https://review.opendev.org/c/openstack/nova/+/932930 | |
| 19:54:33 | opendevreview | sean mooney proposed openstack/nova master: Fix detaching devices by alias with mdevs https://review.opendev.org/c/openstack/nova/+/932931 | |
| 19:55:09 | sean-k-mooney | JayF: johnthetubaguy: fungi: ^ that is the actual correct fix and what i was asking for in that review | |
| 20:13:03 | fungi | awesome! | |
| #openstack-nova - 2024-10-22 | |||
| 06:10:40 | opendevreview | Aleksandr Chudinov proposed openstack/nova master: Fix instance snapshot min disk size after instance resize https://review.opendev.org/c/openstack/nova/+/931737 | |
| 07:24:01 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:24:06 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:25:48 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:26:51 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:29:06 | opendevreview | Takashi Kajinami proposed openstack/nova master: Drop unused parameter type validation method https://review.opendev.org/c/openstack/nova/+/909189 | |
| 08:09:16 | elodilles | bauzas: auniyal: Uggla: if you have some time could you please review (release liaison approve) the antelope final stable release? https://review.opendev.org/c/openstack/releases/+/932406 | |
| 08:14:01 | mikal | sean-k-mooney: I've reproposed https://review.opendev.org/c/openstack/nova-specs/+/932387 (spice-direct spec), just wondering if there's anything else I should do apart from put it in the status etherpad? | |
| 08:17:14 | sean-k-mooney[m] | mikal: you updated the history section so that all that was needed | |
| 08:21:40 | sean-k-mooney[m] | gibi: ill fix the typos in https://review.opendev.org/c/openstack/nova/+/932931/1 once i have had coffee and am actually awake. thanks for the review | |
| 08:22:11 | sean-k-mooney[m] | mikal: i assume the rest of the pataches are ready for review? | |
| 08:24:38 | sean-k-mooney[m] | im not seeing any merge conflicts or direct requests form mel that need to be addressed but also have not looked closely at the current state | |
| 08:55:23 | gibi | sean-k-mooney[m]: sure, just ping me after the respin and I will re-add my +2 | |
| 09:02:18 | gibi | bauzas: when you are up, I'm wondering how much quorum the nova room will have on wednesday during the global eventlet removal discussion? | |
| 09:02:33 | bauzas | I'm already up :) | |
| 09:02:38 | gibi | o/ :) | |
| 09:02:49 | bauzas | gibi: that's a good question | |
| 09:03:06 | bauzas | that discussion will trample our own sessions | |
| 09:03:06 | gibi | I would like to join to the eventlet removal discussion at least | |
| 09:03:24 | bauzas | I was thinking it could be on Monday | |
| 09:03:25 | gibi | but other cores might have the same plans | |
| 09:03:26 | bauzas | :( | |
| 09:03:35 | bauzas | let's discuss then the agenda today | |
| 09:03:44 | bauzas | I'm currently looking at the etherpad | |
| 09:03:49 | gibi | yeah I think the deeper dive will be on wednesday, yesterday we only covered high level things | |
| 09:03:56 | bauzas | and I'll try to provide an agenda | |
| 09:04:03 | gibi | cool, thanks | |
| 09:04:18 | gibi | I have no other limitation during the week as far as I see | |
| 09:10:57 | bauzas | gibi: my own limitation will be on Friday due to the TC sessions :( | |
| 09:11:06 | bauzas | hence the agenda proposal | |
| 09:22:11 | bauzas | gibi: I think we have a serious problem that we need to solve about Wed | |
| 09:41:54 | sean-k-mooney | bauzas: gibi: i dont know if we want to discuss this at the ptg https://bugs.launchpad.net/nova/+bug/2076089 but it would be good if other could confirm or refute my assertion that we shoudl not be abel to force boot a new instance on a disabled compute | |
| 09:42:17 | bauzas | sean-k-mooney: the agenda is now fully packed tbh | |
| 09:42:22 | sean-k-mooney | as far as i am aware we made an intentional decsion to prevent this | |
| 09:42:32 | sean-k-mooney | bauzas: that fine its a bug so we can discuss it async | |
| 09:42:34 | bauzas | the eventlet-removal session just ate all the left time | |
| 09:42:40 | sean-k-mooney | i just want to spread awareness of it | |
| 09:44:02 | sean-k-mooney | im basically asking for a sanity check to ensure im not off base when i say this is not a bug. we can put it on the next team meeting adgenda too if we like | |
| 09:44:09 | bauzas | well | |
| 09:44:22 | bauzas | the forced AZ hack was the usual way for ops to test new computes | |
| 09:44:36 | sean-k-mooney | right but not disabled hosts | |
| 09:44:45 | bauzas | no, even disabled ones | |
| 09:44:56 | sean-k-mooney | that was definetly a bug then | |
| 09:45:09 | sean-k-mooney | i.e that that ever worked | |
| 09:45:21 | bauzas | yeah, for me it looks to me a regression | |
| 09:45:39 | sean-k-mooney | im saying we should not allow that to happen and that i dont condier this a regression | |
| 09:46:01 | sean-k-mooney | if the az hack worked in the past tha that was a bug IMO | |
| 09:46:09 | bauzas | because when you want to verify that your new compute works fine, then you just use the AZ hack for it | |
| 09:46:36 | bauzas | you just make sure that when you create the new compute, it's disabled first | |
| 09:46:54 | sean-k-mooney | your you regerster it as disabeld and add it to a host aggreate restricted to your tenent and then enable it | |
| 09:48:23 | bauzas | hum, no, in general you just disable the host, and then you create new instances using the AZ hack | |
| 09:48:39 | sean-k-mooney | bauzas: the az hack is not ment to force a host bypasssing all schdulign | |
| 09:48:41 | bauzas | once you're sure it works, then you delete the instances and the enable the host, that's it | |
| 09:49:06 | bauzas | sean-k-mooney: trust me, this is why I say this is a hack :) | |
| 09:49:33 | sean-k-mooney | bauzas: right so i object to reintroducing that broken behavior if its what we had in the past | |
| 09:49:39 | sean-k-mooney | as i said i condier it a bug | |
| 09:50:00 | sean-k-mooney | so if we want to fix it i want to be conviced why that is accapable to bypass placment | |
| 09:50:54 | bauzas | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L593 | |
| 09:51:11 | sean-k-mooney | the reporter confirmed the existing behavior (where its rejected) exists as far back as zed | |
| 09:51:18 | sean-k-mooney | meaning that all stable release have the current behavior | |
| 09:52:00 | sean-k-mooney | bauzas: right i have used that in the past but that was nto inteded to ignore if its disabeld. | |
| 09:53:30 | bauzas | that's because the forced host values are not checking the filters | |
| 09:53:46 | bauzas | but... the pre-filters actually check them | |
| 09:53:55 | sean-k-mooney | the prefilters run before the filters | |
| 09:53:55 | bauzas | hence the change | |
| 09:54:00 | bauzas | correct, hence the bug | |
| 09:54:05 | sean-k-mooney | yep a change that was made in train | |
| 09:54:18 | sean-k-mooney | as a result of a spec that was intened to prevent schduling to disable compute | |
| 09:54:31 | sean-k-mooney | to me this was somethign we agree to block as part of the spec | |
| 09:54:40 | sean-k-mooney | so not a bug as a result | |
| 09:56:07 | sean-k-mooney | we also added the ablity to request a host without the az hack in train for what its worth | |
| 09:56:48 | sean-k-mooney | this is the spec https://specs.openstack.org/openstack/nova-specs/specs/train/implemented/pre-filter-disabled-computes.html | |
| 10:01:16 | bauzas | I just think we forgot about this UX... | |