| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-10-22 | |||
| 17:33:00 | bauzas | I'm just sad to hear that the next PTG will still be virtual, but that's life | |
| 17:33:27 | bauzas | I'm just exhausted and I miss our whiteboards and hallway discussions | |
| 17:34:00 | bauzas | but I'll open a beer to enjoy the last day of the PTG as if it was physical :) | |
| 19:22:37 | mnaser_ | hm | |
| 19:22:52 | mnaser_ | anyone would have an idea as to why metadata service seems to be stalling / very slow | |
| 19:23:25 | mnaser_ | 2021-10-22 19:23:13.422 11 INFO nova.metadata.wsgi.server [req-f65469d1-4082-4b9e-ab3b-3c710c1f0366 - - - - -] 10.30.107.187,10.101.2.141 "GET /latest/meta-data/block-device-mapping/root HTTP/1.1" status: 200 len: 148 time: 25.8355188 | |
| 19:23:43 | mnaser_ | it almost feels like all 'requests' stall for a bit and then they all burst out at once | |
| 19:23:47 | mnaser_ | memcache is fine, mysql is fine | |
| 19:24:06 | mnaser_ | plenty of conductors | |
| 19:25:30 | mnaser_ | 2021-10-22 19:25:15.880 13 INFO nova.metadata.wsgi.server [-] 192.168.1.5,10.101.2.142 "GET /openstack HTTP/1.1" status: 200 len: 235 time: 5.0295317 | |
| 19:25:35 | mnaser_ | especailly this, it seems very 5s-y | |
| 19:46:53 | clarkb | mnaser_: I've noticed that on a couple of devstack jobs recently too but haven't had a chance to dig into it beyond noting that was the error returned by tempest | |
| 19:47:13 | clarkb | specifically some tests says metadata service fails to respond in time and the test times out | |
| 20:44:05 | prometheanfire | nova fails with the new oslo-concurrency 4.5.0 https://zuul.opendev.org/t/openstack/build/9a385f3324fb46a3abf6257a09020d38 | |
| 20:45:07 | prometheanfire | looks like an extra value was used (blocking)? | |
| #openstack-nova - 2021-10-25 | |||
| 07:22:03 | opendevreview | Wenping Song proposed openstack/osc-placement master: Remove usage of six https://review.opendev.org/c/openstack/osc-placement/+/815223 | |
| 07:26:41 | opendevreview | Wenping Song proposed openstack/osc-placement master: Remove usage of six https://review.opendev.org/c/openstack/osc-placement/+/815223 | |
| 08:08:03 | opendevreview | Wenping Song proposed openstack/nova master: Cleanup guest process when vm evacuate failed and then deleted https://review.opendev.org/c/openstack/nova/+/814746 | |
| 08:23:23 | bauzas | good morning Nova | |
| 08:25:40 | gibi | bauzas: o/ | |
| 13:00:19 | artom_ | johnthetubaguy[m], TheJulia, so here's an idea (to follow up on the node reassignment hash ring changed thing) | |
| 13:00:27 | artom_ | Move the whole thing to the conductor | |
| 13:02:16 | artom_ | We would in effect be moving virt-driver-specific code into the conductor, but it sure would make everything else easier... | |
| 13:51:51 | kashyap | clarkb: Very cool; this is merged - https://review.opendev.org/c/openstack/devstack/+/815020 :) | |
| 15:18:05 | opendevreview | Erlon R. Cruz proposed openstack/nova master: Fix pre_live_migration rollback https://review.opendev.org/c/openstack/nova/+/815324 | |
| 15:18:45 | opendevreview | Erlon R. Cruz proposed openstack/nova master: Fix pre_live_migration rollback https://review.opendev.org/c/openstack/nova/+/815324 | |
| 15:37:51 | kashyap | clarkb: Responded on the list. Hope that answers your questions. (Even if it's not fully satisfactory) | |
| 15:37:58 | clarkb | kashyap: it does, thanks | |
| 15:38:23 | clarkb | kashyap: also https://bugzilla.redhat.com/show_bug.cgi?id=2010058 seems to be at least part of our fedora problems | |
| 16:54:38 | opendevreview | Merged openstack/nova master: Reproducer unit test for bug 1934094 https://review.opendev.org/c/openstack/nova/+/812646 | |
| 17:42:06 | gibi | melwitt, sean-k-mooney[m]: I think https://review.opendev.org/c/openstack/nova/+/814036 is the way to go. My alternative trial to kill leaking eventlet failed in https://review.opendev.org/c/openstack/nova/+/815017 | |
| 18:27:11 | TheJulia | artom: hmm, I don't know the nova conductor code path well enough to really know for sure. That feels like we're just shifting a point of failure, but at the same time the compute hosts are basically stateless and just fulfill a role... | |
| 18:30:17 | artom | TheJulia, it might be workable if we build a decent abstraction around it... like, the conductor gets the list of running compute services, calls them and passes the list, the services+driver combo do the hash ring thing and return the result to the conductor, the conductor the saves that | |
| 18:30:29 | artom | Really rough initial first idea | |
| 18:33:09 | TheJulia | artom: I guess the question I would ahve is what spured the thought to think to move it | |
| 18:35:48 | artom | TheJulia, well the current problem is that in your proposed patch we'd be modifying instance.host and potentially doing it while something else is accessing it. Except we can't use the lock that everything else uses because we have no guarantee that we're on the same compute host | |
| 18:35:58 | artom | And the lock is host-local | |
| 18:37:12 | TheJulia | Well, it is a single field update on a row, the db should lock it for the transaction unless sqlachemy's save is update all fields or insert/delete | |
| 18:37:29 | TheJulia | and *if* something should happen where there is a split perception, it should reconcile | |
| 18:37:34 | TheJulia | on the net run | |
| 18:37:49 | TheJulia | s/net/next | |
| 18:39:19 | artom | I... guess? In any case, there be dragons | |
| 18:39:32 | TheJulia | there are dragons today :) | |
| 18:39:40 | TheJulia | and they think we are tasty with ketchup | |
| 19:10:35 | opendevreview | mitya-eremeev-2 proposed openstack/nova master: Close Glance image if downloading failed. https://review.opendev.org/c/openstack/nova/+/815347 | |
| 19:37:20 | opendevreview | Gustavo Santos proposed openstack/nova master: Reattach mdevs to guest on resume https://review.opendev.org/c/openstack/nova/+/815373 | |
| #openstack-nova - 2021-10-26 | |||
| 08:27:06 | bauzas | folks, the PTG summary email was sent yesterday, if I made any mistake, lemme know | |
| 08:40:11 | gibi | bauzas: I left a question you might be interested in in https://review.opendev.org/c/openstack/nova/+/810220/9/nova/compute/manager.py#1436 | |
| 08:40:44 | gibi | bauzas: about the PTG mail, did we agreed changing the OWNER trait name from OWNER_CYBORG to OWNED_BY_CYBORG? I don't remember' | |
| 08:41:12 | bauzas | gibi: ah, no worries | |
| 08:41:20 | bauzas | I don't want to bikeshed about the naming | |
| 08:41:46 | bauzas | if we want to use 'OWNER_project', OK, that said I'll provide a comment for it | |
| 08:42:46 | gibi | bauzas: I think the name was OWNER_project before the PTG but your mail now suggest otherwise and I don't know that this change is intentional or not | |
| 08:45:19 | bauzas | gibi: the etherpad says "OWNER_CYBORG" | |
| 08:45:56 | bauzas | gibi: my summary was for operations explaining about what we discussed, but the name is not an issue | |
| 08:45:58 | jkulik | bauzas: thank you very much for the summary. very helpful | |
| 08:47:18 | gibi | bauzas: OK, so that was just an honest mistak in your mail. I will reply sortly to fix that | |
| 08:47:31 | bauzas | k | |
| 09:00:58 | gibi | bauzas: can I get a quick approval on the re proposed pps spec? https://review.opendev.org/c/openstack/nova-specs/+/814531 I also refreshed the remaining implementation patch which is only the nova-manage support https://review.opendev.org/c/openstack/nova/+/802060/17 | |
| 09:01:36 | gibi | and I have a related bugfix https://review.opendev.org/c/openstack/nova/+/811396 | |
| 09:02:32 | bauzas | gibi: I could diff but I trust, what's the diff between the xena and the yoga specs ? | |
| 09:02:43 | bauzas | trust you* | |
| 09:05:25 | gibi | bauzas: the main change is that from upgrade perspective we dont need to care about the older nova newer neutron case | |
| 09:05:48 | gibi | as nova merged the support for the new api extension in xena while neutron did not merged the api extension in yoga | |
| 09:06:02 | gibi | so the nova xena - neutron yoga case does not need special handling any more | |
| 09:06:12 | gibi | but it really only impacts the neutron impl | |
| 09:06:30 | gibi | as now they don't need to make the extension optional | |
| 09:06:39 | bauzas | yup, that's what I see | |
| 09:07:35 | gibi | that what I remember :) | |
| 09:08:21 | gibi | since xena M3 I had also time to propose a set of tempest test for the pps feature so I'm pretty confident it works | |
| 09:09:20 | gibi | if you are interested https://review.opendev.org/q/project:openstack/tempest+status:open+topic:bp/qos-minimum-guaranteed-packet-rate | |
| 09:12:06 | gibi | gmann: when you are up, to be able to move forward with the pps feature in neutron, we need to land the tempest test adaptation for the new API extension. So this is ready https://review.opendev.org/c/openstack/neutron/+/805637 but it needs https://review.opendev.org/c/openstack/tempest/+/809168/ first. If you have time I appreaciate some review on the tempest side | |
| 09:45:38 | opendevreview | Merged openstack/nova-specs master: Re-propose QoS minimum guaranteed packet rate https://review.opendev.org/c/openstack/nova-specs/+/814531 | |
| 10:23:18 | opendevreview | Takashi Kajinami proposed openstack/nova master: Clean up allocations left by evacuation when deleting service https://review.opendev.org/c/openstack/nova/+/778696 | |
| 10:26:52 | opendevreview | Takashi Kajinami proposed openstack/nova master: Clean up allocations left by evacuation when deleting service https://review.opendev.org/c/openstack/nova/+/778696 | |
| 13:12:40 | opendevreview | Wenping Song proposed openstack/nova master: Log error msg when delete resource provider failed https://review.opendev.org/c/openstack/nova/+/815498 | |
| 13:13:59 | opendevreview | Balazs Gibizer proposed openstack/nova master: Add a WA flag waiting for vif-plugged event during reboot https://review.opendev.org/c/openstack/nova/+/813419 | |
| 13:46:12 | gmann | gibi: sure, will review after my breakfast. | |
| 13:47:51 | gibi | gmann: thanks! | |
| 14:00:29 | bauzas | what a joy, 30 new bugs | |
| 14:00:37 | bauzas | +17 | |
| 14:01:03 | gibi | somebody was busy reporting bugs why we had the PTG | |
| 15:16:30 | bauzas | gibi: can you please help me on this one ? https://bugs.launchpad.net/nova/+bug/1948776 | |
| 15:17:32 | bauzas | looks to me legit | |
| 15:18:45 | sean-k-mooney | im not sure that is a bug so much as an oversite | |
| 15:19:05 | sean-k-mooney | it will be a notificaiton object chnage if we start sending it right | |
| 15:19:11 | sean-k-mooney | so it wont be backporatable | |
| 15:19:30 | sean-k-mooney | but the locked state could be included going forward | |
| 15:20:04 | sean-k-mooney | it cannot be added to queens or even xena at this point however | |
| 15:22:57 | sean-k-mooney | bauzas: gibi i triaged that as whishlist and remove the nonstandard tags does that work for ye https://bugs.launchpad.net/nova/+bug/1948776 | |
| 15:23:26 | bauzas | sean-k-mooney: cool, thanks | |
| 15:25:53 | gibi | bauzas, sean-k-mooney: thank I agree with your assesments | |
| 15:26:41 | opendevreview | Gustavo Santos proposed openstack/nova master: Reattach mdevs to guest on resume https://review.opendev.org/c/openstack/nova/+/815373 | |
| 15:50:13 | bauzas | reminder : nova meeting in 10 mins here | |
| 16:00:02 | bauzas | #startmeeting nova | |
| 16:00:02 | opendevmeet | Meeting started Tue Oct 26 16:00:02 2021 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:00:02 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:00:02 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:00:15 | gibi | o/ | |
| 16:00:19 | bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | |