Earlier  
Posted Nick Remark
#openstack-nova - 2021-12-08
10:47:55 slaweq I don't know about shelve and unshelve
10:47:56 gibi that is almost like a new boot
10:48:02 gibi but from a previous snapshot
10:48:25 gibi there is a new scheduling, a new port biding a new vif plug and spawn
10:48:52 slaweq so it should be sent when plug is completed
10:49:04 slaweq in such case
10:49:06 gibi yes, that is how I would expect it (and how nova expects it)
10:49:27 slaweq please move that bug to neutron then, I will check in our logs what happened there
10:49:31 opendevreview mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935
10:49:43 slaweq and thx for checking that
10:49:45 gibi slaweq: ok, I summarized this in the bug so you have the logs with timestamps from nova
10:49:51 slaweq ++
10:51:36 gibi I haven't looked at the other case in the same bug the tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert
10:51:45 gibi but I can do that too
12:12:59 sean-k-mooney slaweq: unshelve should be the same as first boot
12:13:00 lyarwood does anyone know when/where during a spawn that we would expect the metadata service to have details on a given instance ready to serve up?
12:14:34 lyarwood I'm looking at a CI failure downstream where cloud-init gets Connection refused everytime it tries to pull from the api but I can't see errors in the actual service logs
12:19:12 gibi lyarwood: I have limited knowledge too, but the request is goes from the guest, goes to the neutron-metadata service via the guest's network and the neutron forwards the request to nova-metadata
12:23:13 lyarwood thanks I think I was missing the neutron part
12:24:12 sean-k-mooney ill get the code one sec
12:24:32 sean-k-mooney lyarwood: it will always be ready before we call libvirt
12:25:20 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L168
12:25:28 sean-k-mooney that is where we generate the network metadata
12:26:15 sean-k-mooney which is called form the init of InstanceMetadata https://github.com/openstack/nova/blob/052cf963583ab7c6bbe4fcbf7bfe69f8f6733bdb/nova/api/metadata/base.py#L176
12:27:49 lyarwood ACK thanks
12:28:14 lyarwood looks like an issue before that as the neutron-metadata-agent on the compute isn't even seeing the requests
12:28:29 lyarwood I guess with plugging but AFAICT from the compute logs that worked
12:29:53 sean-k-mooney the neutorn-metadata-agent is typeiclay not on the compute but on the contler
12:30:09 sean-k-mooney it runs in either the router or dhcp agent network namespaces
12:30:14 sean-k-mooney depening on your config
12:30:27 sean-k-mooney ovn works slightly differntly and i think it does it on each compute
12:30:43 sean-k-mooney but the agent there is jsut calling the nova metadta api
12:31:06 sean-k-mooney the api i belive generates the instnace mentadta directly itself and caches it
12:32:48 sean-k-mooney for libvirt we generate it here by the way as part of creating the config drive https://github.com/openstack/nova/blob/e537d90d6fc0977742f7126c3f8cfef6bf8b2a15/nova/virt/libvirt/driver.py#L4782
12:36:10 sean-k-mooney lyarwood: are we using memcache downstream
12:36:25 sean-k-mooney we had to enable it upstream to make the ci stable
12:36:40 sean-k-mooney without it differrnt request could go to differnt api workers
12:36:57 sean-k-mooney cloud-init will only retry the first request
12:37:09 sean-k-mooney and after that it assuem the data is avaiable and does not retry them
12:37:28 sean-k-mooney if you hit a different worker and the node is under powered that can lead to timeouts
12:37:47 sean-k-mooney or failure to retive the data as genertatd the metadata is actully quite expensive
12:56:39 lyarwood sorry had to drop quickly for lunch
12:57:51 lyarwood this is part of a queens to train upgrade job downstream and it looks like the neutron metadata services are running on the computes
12:58:04 lyarwood I think this is failing post upgrade to train but it's super confusing
12:58:08 lyarwood jenkins--
13:00:45 lyarwood ah no it's pre upgrade on queens
13:09:23 gryf asfdsdf
13:11:57 kashyap gryf: Thanks; hope you don't change your password :D
13:13:57 gryf kashyap, not really, just the ssh connection hiccup ;)
13:14:27 kashyap Heh, was just trolling ya
13:16:05 gryf I figured :)
13:22:18 pslestang ccccccdlucfhelkhccngbvunedtddhhlecgrlcdverhc
13:22:40 pslestang ouup's sorry
13:22:49 pslestang hello by the way!
13:22:52 kashyap That's a YubiKey :)
13:22:58 pslestang ;-)
13:23:10 gryf that's looks definitely like a sort of pass ;P
13:23:25 kashyap pslestang: There's a setting to actually make YubiKey to _not_ send the key press automatically
13:23:45 kashyap Lemme post my notes. I have a Nano
13:25:23 pslestang kashyap: I do not have a press button, my yubikey is near the enter key and when I touch the edge of the enter key ccccccdlucfhekeerjhdnrejghllkrdvdnjnjnijknbt
13:25:44 pslestang you see waht I mean
13:25:58 gryf pslestang, just change usb port :)
13:26:05 pslestang already done
13:26:09 gryf :D
13:27:11 kashyap pslestang: What I'm saying is that once you touch it, it automatically sends return keypress. That way you can still touch it while you're active on this channel, and it won't post it here
13:27:24 kashyap See what I mean?
13:27:30 kashyap pslestang: This one - https://kashyapc.fedorapeople.org/YubiKey-Nano-Config-for-Return-Key.html
13:30:31 pslestang kashyap: thx for sharing this, it will definitely be part of my setup
13:31:09 kashyap When I said "That way ..." above, I meant to say _once_ you use the above config. :)
13:33:49 pslestang kashyap: just applied, works great, thx!
13:34:05 kashyap Cool.
13:40:07 kashyap pslestang: (I had broken formatting in the above page; fixed it now. I see that you were able to see through it)
13:43:49 pslestang kashyap: yep I automatically 's/ /</br>/' whith my eyes in the corresponding lines :)
14:29:59 gsantos Hello, folks. I'm the owner of https://review.opendev.org/c/openstack/nova/+/815373 , which was just merged, and I was wondering if a backport of this fix to the previous branches would be feasible. I'm looking at Ussuri, specifically.
14:31:12 lyarwood gsantos: I would think so but it's conditional on the version of libvirt used right?
14:37:40 gsantos Yes, that is a concern. It needs at least libvirt v4.3.0, and the minimum libvirt version for Ussuri seems to be 4.0.0
14:40:07 lyarwood gsantos: kk that's easy enough to handle in the backport however
14:46:38 gsantos Great! So, in order to do that, do I cherry pick (and resolve conflicts for) this commit to the xena, wallaby, victoria and ussuri branches?
14:52:30 lyarwood gsantos: Correct, https://docs.openstack.org/project-team-guide/stable-branches.html#processes
14:52:50 lyarwood gsantos: and add in the libvirt version check once our MIN_LIBVIRT_VERSION dips below 3.2.0
14:52:55 lyarwood sorry 4.3.0
14:57:32 opendevreview Alexey Stupnikov proposed openstack/nova master: Test aborting queued live migration https://review.opendev.org/c/openstack/nova/+/776250
15:06:00 opendevreview mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935
15:12:47 opendevreview mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935
15:15:52 EugenMayer when getting `--os-compute-api-version 2.26 or greater is required to support the --not-tag option` while using xena, is that expected?
15:20:42 EugenMayer interestingly, that is only the case for the openstack cli, using nova list --not-tags works without issues
15:21:17 opendevreview Merged openstack/nova-specs master: Allow project admin to list hypervisors https://review.opendev.org/c/openstack/nova-specs/+/793011
15:33:47 gsantos lyarwood: I will try to make these cherry-picks today. Thank you!
15:39:40 sean-k-mooney lyarwood: if you have time can you look at https://review.opendev.org/c/openstack/nova/+/820531 i think the patch looks ok but it would be good to have your input. its pretty small
15:59:09 lyarwood sean-k-mooney: yeah that doesn't look correct
15:59:17 lyarwood I'll write up some notes after some downstream calls
16:00:35 sean-k-mooney ack
16:08:26 opendevreview Merged openstack/nova master: Reattach mdevs to guest on resume https://review.opendev.org/c/openstack/nova/+/815373
16:54:22 EugenMayer Trying to us `--not-tags` with the openstack cli - tells me that my API version '--os-compute-api-version 2.26 or greater is required to support the --not-tag option' - is this expected with xena? seems like https://docs.openstack.org/releasenotes/nova/en_GB/xena.html tells me, that xena has v2.90. - any hints?
16:56:43 melwitt EugenMayer: it is expected, you have to pass --os-compute-api-version 2.26 with the command else by default OSC uses the oldest available microversion 2.1. it does not default to latest microversion
17:03:40 EugenMayer ah ! thank you sir!
19:54:21 eandersson Anyone know if the bug with orphaned neutron ports when deleting a VM on a offline compute fixed in newer versions of OpenStack? Also not super sure if it is a Neutron or Nova bug :p
20:55:18 opendevreview Gustavo Santos proposed openstack/nova stable/xena: Reattach mdevs to guest on resume https://review.opendev.org/c/openstack/nova/+/821126
#openstack-nova - 2021-12-09
08:02:40 gibi o/ morning nova
08:03:08 gibi sean-k-mooney, bauzas, gmann: yesterday I forgot to read back on the instance_action proposal from Tuesday. Now I did

Earlier   Later