| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-10-21 | |||
| 17:34:53 | sean-k-mooney | we hope the mtty supprot will give a baseline of coverage for things like this | |
| 17:35:15 | sean-k-mooney | johnthetubaguy: https://review.opendev.org/q/topic:%22mtty_support%22 | |
| 17:36:13 | sean-k-mooney | we need https://review.opendev.org/c/openstack/nova/+/898100/4 first ot allow using virtual devices | |
| 17:42:12 | opendevreview | Artom Lifshitz proposed openstack/nova master: pwr mgmt: power up instances's CPUs upon init https://review.opendev.org/c/openstack/nova/+/932926 | |
| 17:42:30 | sean-k-mooney | fungi: the orgianl confugion arount "we dont support your usecase" came from the fact they fix the bug by adding the infra required to be able to detach an mdev via the alisa, without stating that they were actully trying to detach a volume and this just so happend to fail when the vm also had an mdev | |
| 17:43:26 | sean-k-mooney | they had a limited explcation of that in the bug but not in the commit message | |
| 18:15:02 | fungi | sean-k-mooney: thanks for the clarification, makes sense | |
| 18:16:00 | fungi | also worth noting, there was a chameleon cloud representative at oid-na who was talking about the possibility of making vgpu resources available for upstream testing. not sure where that will go, but something to consider | |
| 18:16:58 | sean-k-mooney | the issue is we cant do that with nested vir | |
| 18:17:08 | sean-k-mooney | i.e. we can only test it properly with baremental compute nodes | |
| 18:17:28 | sean-k-mooney | if you passhtough afully gpu to a vm you cant then use vgpus | |
| 18:17:44 | JayF | darn, too bad we don't know how to provision baremetal /s | |
| 18:18:01 | JayF | seriously though, having to provision baremetal is very tough in CI, it's part of why Ironic 3rd party CIs died out | |
| 18:18:04 | JayF | just a little high touch | |
| 18:18:25 | fungi | well, yeah they were talking about nova baremetal api to boot machines with vgpu-capable resources | |
| 18:18:31 | fungi | (i.e. ironic) | |
| 18:19:23 | fungi | but i agree with you about the whole safety challenge with providing root access to physical machines for untrusted job payloads | |
| 18:19:41 | JayF | it's not just a safety challenge, it's a challenge-challenge | |
| 18:19:58 | JayF | many real-world failure modes which needs human hands, or at least, non-automated intervention to fix | |
| 18:20:26 | sean-k-mooney | the tldr is when you passthough a fully pci device in qemu it removes all the sriov capablities because qemu did not have a way to make that work | |
| 18:20:58 | sean-k-mooney | for what its worth we are lookign to see if we can use our internal hardware in teh future to test specifi upstream changes | |
| 18:21:50 | sean-k-mooney | that woudl do a baremteally deployment of a nova compute (using ironic) with a manual triger to basicaly say "this is safe to test" | |
| 18:21:53 | sean-k-mooney | as a third party ci | |
| 18:22:52 | JayF | I think there's a lot of untapped value in that idea of on-demand testing | |
| 18:23:09 | JayF | we can *usually* pinpoint when we're getting close to an edge case that might need revalidation | |
| 18:23:20 | sean-k-mooney | but that is why we are also trying to figure out how to test without real hardware using the sample kernel dirvers for a baseline of coverage | |
| 18:24:05 | sean-k-mooney | for example we are enableing igb vif model in nova so tthat in the future we can use that to test sriov | |
| 18:24:40 | sean-k-mooney | if our providers can provide neutron ports with igb vif model in the future we can fully test sriov in the 1st party ci | |
| 18:25:16 | sean-k-mooney | unfrogually it needs a new enough nova/qemu but by 2026.1 we might be in a position to do that | |
| 18:25:32 | sean-k-mooney | *unfortunetly | |
| 19:04:37 | opendevreview | Artom Lifshitz proposed openstack/nova master: pwr mgmt: power up instances's CPUs upon init https://review.opendev.org/c/openstack/nova/+/932926 | |
| 19:54:32 | opendevreview | sean mooney proposed openstack/nova master: Add repoducer test for bug 2074219 https://review.opendev.org/c/openstack/nova/+/932930 | |
| 19:54:33 | opendevreview | sean mooney proposed openstack/nova master: Fix detaching devices by alias with mdevs https://review.opendev.org/c/openstack/nova/+/932931 | |
| 19:55:09 | sean-k-mooney | JayF: johnthetubaguy: fungi: ^ that is the actual correct fix and what i was asking for in that review | |
| 20:13:03 | fungi | awesome! | |
| #openstack-nova - 2024-10-22 | |||
| 06:10:40 | opendevreview | Aleksandr Chudinov proposed openstack/nova master: Fix instance snapshot min disk size after instance resize https://review.opendev.org/c/openstack/nova/+/931737 | |
| 07:24:01 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:24:06 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:25:48 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:26:51 | opendevreview | Takashi Kajinami proposed openstack/nova master: Deprecate [wsgi] secure_proxy_ssl_header https://review.opendev.org/c/openstack/nova/+/817927 | |
| 07:29:06 | opendevreview | Takashi Kajinami proposed openstack/nova master: Drop unused parameter type validation method https://review.opendev.org/c/openstack/nova/+/909189 | |
| 08:09:16 | elodilles | bauzas: auniyal: Uggla: if you have some time could you please review (release liaison approve) the antelope final stable release? https://review.opendev.org/c/openstack/releases/+/932406 | |
| 08:14:01 | mikal | sean-k-mooney: I've reproposed https://review.opendev.org/c/openstack/nova-specs/+/932387 (spice-direct spec), just wondering if there's anything else I should do apart from put it in the status etherpad? | |
| 08:17:14 | sean-k-mooney[m] | mikal: you updated the history section so that all that was needed | |
| 08:21:40 | sean-k-mooney[m] | gibi: ill fix the typos in https://review.opendev.org/c/openstack/nova/+/932931/1 once i have had coffee and am actually awake. thanks for the review | |
| 08:22:11 | sean-k-mooney[m] | mikal: i assume the rest of the pataches are ready for review? | |
| 08:24:38 | sean-k-mooney[m] | im not seeing any merge conflicts or direct requests form mel that need to be addressed but also have not looked closely at the current state | |
| 08:55:23 | gibi | sean-k-mooney[m]: sure, just ping me after the respin and I will re-add my +2 | |
| 09:02:18 | gibi | bauzas: when you are up, I'm wondering how much quorum the nova room will have on wednesday during the global eventlet removal discussion? | |
| 09:02:33 | bauzas | I'm already up :) | |
| 09:02:38 | gibi | o/ :) | |
| 09:02:49 | bauzas | gibi: that's a good question | |
| 09:03:06 | gibi | I would like to join to the eventlet removal discussion at least | |
| 09:03:06 | bauzas | that discussion will trample our own sessions | |
| 09:03:24 | bauzas | I was thinking it could be on Monday | |
| 09:03:25 | gibi | but other cores might have the same plans | |
| 09:03:26 | bauzas | :( | |
| 09:03:35 | bauzas | let's discuss then the agenda today | |
| 09:03:44 | bauzas | I'm currently looking at the etherpad | |
| 09:03:49 | gibi | yeah I think the deeper dive will be on wednesday, yesterday we only covered high level things | |
| 09:03:56 | bauzas | and I'll try to provide an agenda | |
| 09:04:03 | gibi | cool, thanks | |
| 09:04:18 | gibi | I have no other limitation during the week as far as I see | |
| 09:10:57 | bauzas | gibi: my own limitation will be on Friday due to the TC sessions :( | |
| 09:11:06 | bauzas | hence the agenda proposal | |
| 09:22:11 | bauzas | gibi: I think we have a serious problem that we need to solve about Wed | |
| 09:41:54 | sean-k-mooney | bauzas: gibi: i dont know if we want to discuss this at the ptg https://bugs.launchpad.net/nova/+bug/2076089 but it would be good if other could confirm or refute my assertion that we shoudl not be abel to force boot a new instance on a disabled compute | |
| 09:42:17 | bauzas | sean-k-mooney: the agenda is now fully packed tbh | |
| 09:42:22 | sean-k-mooney | as far as i am aware we made an intentional decsion to prevent this | |
| 09:42:32 | sean-k-mooney | bauzas: that fine its a bug so we can discuss it async | |
| 09:42:34 | bauzas | the eventlet-removal session just ate all the left time | |
| 09:42:40 | sean-k-mooney | i just want to spread awareness of it | |
| 09:44:02 | sean-k-mooney | im basically asking for a sanity check to ensure im not off base when i say this is not a bug. we can put it on the next team meeting adgenda too if we like | |
| 09:44:09 | bauzas | well | |
| 09:44:22 | bauzas | the forced AZ hack was the usual way for ops to test new computes | |
| 09:44:36 | sean-k-mooney | right but not disabled hosts | |
| 09:44:45 | bauzas | no, even disabled ones | |
| 09:44:56 | sean-k-mooney | that was definetly a bug then | |
| 09:45:09 | sean-k-mooney | i.e that that ever worked | |
| 09:45:21 | bauzas | yeah, for me it looks to me a regression | |
| 09:45:39 | sean-k-mooney | im saying we should not allow that to happen and that i dont condier this a regression | |
| 09:46:01 | sean-k-mooney | if the az hack worked in the past tha that was a bug IMO | |
| 09:46:09 | bauzas | because when you want to verify that your new compute works fine, then you just use the AZ hack for it | |
| 09:46:36 | bauzas | you just make sure that when you create the new compute, it's disabled first | |
| 09:46:54 | sean-k-mooney | your you regerster it as disabeld and add it to a host aggreate restricted to your tenent and then enable it | |
| 09:48:23 | bauzas | hum, no, in general you just disable the host, and then you create new instances using the AZ hack | |
| 09:48:39 | sean-k-mooney | bauzas: the az hack is not ment to force a host bypasssing all schdulign | |
| 09:48:41 | bauzas | once you're sure it works, then you delete the instances and the enable the host, that's it | |
| 09:49:06 | bauzas | sean-k-mooney: trust me, this is why I say this is a hack :) | |
| 09:49:33 | sean-k-mooney | bauzas: right so i object to reintroducing that broken behavior if its what we had in the past | |
| 09:49:39 | sean-k-mooney | as i said i condier it a bug | |
| 09:50:00 | sean-k-mooney | so if we want to fix it i want to be conviced why that is accapable to bypass placment | |
| 09:50:54 | bauzas | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L593 | |
| 09:51:11 | sean-k-mooney | the reporter confirmed the existing behavior (where its rejected) exists as far back as zed | |
| 09:51:18 | sean-k-mooney | meaning that all stable release have the current behavior | |
| 09:52:00 | sean-k-mooney | bauzas: right i have used that in the past but that was nto inteded to ignore if its disabeld. | |
| 09:53:30 | bauzas | that's because the forced host values are not checking the filters | |
| 09:53:46 | bauzas | but... the pre-filters actually check them | |
| 09:53:55 | bauzas | hence the change | |
| 09:53:55 | sean-k-mooney | the prefilters run before the filters | |