Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-04-03
00:45:36 mnaser i'll add it to my tomorrow list!
00:46:25 cardoe hmm. That's gonna be a tough one to solve cause config_drive is generated before the update_port_postcommit() which will resolve that deferred allocation no?
02:36:00 opendevreview Mohammed Naser proposed openstack/nova master: Add reproducer for bug#2106073 https://review.opendev.org/c/openstack/nova/+/946221
02:37:33 mnaser https://bugs.launchpad.net/nova/+bug/2106073 - revised the bug, seems to be affecting nova at wide since it feeds in ports without ip allocation (and doesnt wait for it ip_allocation=deferred) and pushed another repro..
03:14:35 opendevreview Yaguang Tang proposed openstack/nova stable/2024.1: Fix deepcopy usage for BlockDeviceMapping in get_root_info https://review.opendev.org/c/openstack/nova/+/926696
03:35:03 opendevreview melanie witt proposed openstack/nova master: Call volume detach rollback API if detach fails https://review.opendev.org/c/openstack/nova/+/880399
03:35:03 opendevreview melanie witt proposed openstack/nova master: Reproducer for bug 2016173 https://review.opendev.org/c/openstack/nova/+/946222
03:52:56 opendevreview Masahito Muroi proposed openstack/nova master: Add check_instance_state validation to lock API https://review.opendev.org/c/openstack/nova/+/946223
04:14:22 opendevreview Masahito Muroi proposed openstack/nova master: Add check_instance_state validation to lock API https://review.opendev.org/c/openstack/nova/+/946223
08:46:07 opendevreview Masahito Muroi proposed openstack/nova master: Add check_instance_state validation to lock API https://review.opendev.org/c/openstack/nova/+/946223
09:15:54 opendevreview Masahito Muroi proposed openstack/nova master: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/939658
09:18:35 masahito gmann: sean-k-mooney: ack. thanks for the review. updated the commit. https://review.opendev.org/c/openstack/nova/+/939658
10:47:00 sean-k-mooney mnaser: that not new
10:47:10 sean-k-mooney mnaser: there is already a older bug for that
10:48:39 sean-k-mooney mnaser: well or at least there have been some patches floating around for that issue for a while
10:50:45 sean-k-mooney mnaser: ah i twas a downstream only bug report https://bugzilla.redhat.com/show_bug.cgi?id=2276701
10:51:02 sean-k-mooney which got moved to here https://issues.redhat.com/browse/OSP-33489
10:51:22 sean-k-mooney ill add the upstream tracker to that
10:52:15 sean-k-mooney sorry https://issues.redhat.com/browse/OSPRH-13144
13:18:59 mnaser sean-k-mooney: thanks for all those details
13:19:19 mnaser I wonder if can get away with deferring config drive creation till we get vif plugged event
13:19:32 sean-k-mooney for ironci we cant
13:19:42 sean-k-mooney for libvirt i think yes
13:19:54 sean-k-mooney the file just need to exist before we start the vm
13:20:01 sean-k-mooney so we can do that later
13:20:28 mnaser So then the only way we have for this in ironic would be some sort of polling for ports if they're ip address deferred
13:20:37 sean-k-mooney well we can look at the ironic situration but the ironci driver is "special"
13:21:29 mnaser It feels ugly but I guess we can just loop over the vifs
13:21:38 sean-k-mooney for ironic we are not really generting a config drive. what we are doign is generating the content we woudl provide and then passing that to there rest api
13:21:43 mnaser And check if any ports have deferred IP allocation, if so, wait for it to appear
13:22:08 sean-k-mooney its possibel we could add a call back ot update ironic if the metadata changes
13:22:13 sean-k-mooney but i dont knwo fi that woudl work
13:22:42 sean-k-mooney ideally you would use the metadat service instead of config drive with ironic if you need it to be up to date
13:22:57 mnaser That might be a lot of complexity and also still prone to race conditions cause then what will ironic do if it's mid provision and it changes?
13:23:05 sean-k-mooney ill quickly take a look at the driver and see if we can generate it slightly later
13:23:15 mnaser Yeah -- my issue is that I'm using bonds
13:23:22 mnaser AFAIK I need config drive for those
13:23:34 sean-k-mooney well bonds are not supprot in nova or neutron
13:23:47 mnaser Ironic port groups?
13:23:48 sean-k-mooney and its implemtned i think in ironci directly via port groups?
13:23:57 sean-k-mooney ya prot groups are not a thing in nova or neutron
13:24:07 sean-k-mooney its a purely ironic only network construct
13:24:28 mnaser Yeah and the only place that l
13:24:42 mnaser "Bond" network data is generated is config drive
13:25:04 mnaser So otherwise if I use metadata ill have to do this manually after system boot
13:25:34 sean-k-mooney so ironic has it own enrechment of the stand metadta https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L1045-L1106
13:25:37 sean-k-mooney for the network info
13:25:59 sean-k-mooney which is all custom for the ironic integration and not part of how this normally works
13:26:42 mnaser The enrichment relies on the get network metadata returning the right thing and that doesn't do that so that's how we end up where we are :p
13:27:39 sean-k-mooney right os most if not all other dirver use the commeon code that is shared with the api for generating this metadtaa
13:27:50 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/api/metadata/base.py#L112
13:29:04 sean-k-mooney so they generate there enriched config driver here when calling provision https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L1239-L1258
13:29:05 mnaser I think attach_interface is how ports are plugged?
13:30:06 sean-k-mooney so port bidnign is intened to happen in the compute manger and that is when the ip should be assigend
13:30:31 sean-k-mooney and then port pluggin is where we are ment to talk to the network backend to make sure the ports are conencted on the swtich
13:30:34 sean-k-mooney virutal or otherwise
13:30:58 sean-k-mooney and tha tis when we are ment to waith for the network-vif-plugged evnet
13:31:42 sean-k-mooney plug_vifs in the ironic driver is a noop
13:31:44 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L1567-L1638
13:32:42 sean-k-mooney that is breakign the virt dirver contract by how its delegating the attachment
13:33:07 dansmith sean-k-mooney: juicy OTU series just waiting for +Ws :) https://review.opendev.org/c/openstack/nova/+/944148
13:34:10 sean-k-mooney dansmith: yes... im aware. im slightly anooyed at you for askign for an os-traits release when i ask not to do that unlie we landed teh vdi traits but its on my list to look at before the ptg
13:34:52 mnaser Now I'm wondering where does it even get the ports from... Unless it doesn't and ironic does that stuff behind the scenes?
13:35:07 sean-k-mooney mnaser: so attach_interface is ment to be for adding/removing port after the "server" is created
13:35:21 mnaser That's what I thought...
13:35:30 dansmith sean-k-mooney: you did? I'm sorry
13:35:49 mnaser So that means the neutron vif plug happens when the spawn is called
13:35:50 sean-k-mooney dansmith: its fine, release are cheap
13:36:00 sean-k-mooney mnaser: ya internaly in ironic
13:36:16 mnaser So I think it might be impossible to get configdrive then
13:36:28 dansmith sean-k-mooney: yeah, I honestly never would have though it would be worth ever holding a release to batch up more than one
13:37:10 mnaser Because the plug happens after we call ironic to spawn with configdrive included.
13:37:17 mnaser I see what you mean about the whole callback thing now
13:37:20 mnaser Uuuuuugh
13:37:22 sean-k-mooney dansmith: well last time i ask peole said they woudl take a look at the traits patch and didnt. gibi did look this time to give them credit
13:37:51 sean-k-mooney mnaser: this is one of the large pices of technial debt with the ironic driver
13:37:59 sean-k-mooney its doign several things its not allowed to do
13:38:05 dansmith sean-k-mooney: link me
13:38:10 sean-k-mooney including changign the vnic type ot baremetal
13:38:33 sean-k-mooney dansmith: its no kind of blockd on use approving the specless blueprint which si now a ptg topic but sure on sec
13:39:21 dansmith sean-k-mooney: okay well if I had realized I would have argued not to block my thing on another blocked thing, especially since "releases are cheap" :)
13:39:25 sean-k-mooney dansmith: its https://review.opendev.org/c/openstack/os-traits/+/940418
13:40:08 sean-k-mooney dansmith: we can wait till melwitt has a chance ot respond or till the ptg
13:40:26 sean-k-mooney dansmith: anyway ill try and get to the OTU device tommorw or this evening
13:40:27 dansmith yeah, sounds like that's the best idea
13:41:01 sean-k-mooney mnaser: i think port bidning is still done in nova.
13:41:33 sean-k-mooney mnaser: if we can confirm that we may be able ot tweak plug_vifs to refesh the cache or soemthign like that to get the ip in teh defered case
13:41:46 sean-k-mooney mnaser: but its kind of building one hack on top of anohter
13:42:09 sean-k-mooney unless we can do that in a common way in the compute manger of coruse
13:43:00 mnaser hmm yea
13:43:49 mnaser Do other drivers rely on compute manager to bind the ports?
13:44:01 sean-k-mooney yes
13:44:08 sean-k-mooney virt drivers are not allowed to do it
13:44:19 sean-k-mooney tehy can do port pluging
13:44:33 sean-k-mooney btu that a diffent thing
13:45:19 sean-k-mooney as i said the ironic dirver or ironci its self is doing thing that its not technailly allowed to do like modifyiing cidner volume attachmetn or neutron ports
13:45:24 mnaser I see, so then technically we should have a bound port at some point when we generate the config drive
13:45:51 sean-k-mooney i dont thinke we will
13:46:10 mnaser Like I know this sounds stupid but as a workaround for me now a sleep and a refresh of network info can potentially make the ip arrive?
13:46:13 sean-k-mooney because im not sure any of the prot groups stuff for the bonds will be done when we bind but i might be wrong

Earlier   Later