| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-02-07 | |||
| 15:04:10 | bauzas | gibi: any way we could have to introspect in some log what was creating the thread ? | |
| 15:05:01 | bauzas | ChatGPT, maybe you know ? | |
| 15:07:48 | elodilles | :) | |
| 15:08:04 | elodilles | (meanwhile, I'm done with the wiki editing) | |
| 15:10:57 | gibi | bauzas: we can try printing https://github.com/openstack/nova/blob/9bc198e05733c03ba1a40f89cd6a77ab54b7e480/nova/tests/fixtures/notifications.py#L154-L160 to get the name of the testcase that started the eventlet | |
| 15:13:01 | bauzas | gibi: I can write a patch | |
| 15:13:17 | bauzas | given the occurrences, we may have evidences coming up | |
| 15:13:22 | bauzas | sooner than later | |
| 15:14:00 | gibi | yeah lets try that | |
| 15:15:28 | bauzas | gibi: that being said, the thread is maybe not a FakeVersionedNotifier | |
| 15:16:28 | gibi | bauzas: FakeVersionedNotifier was on the receiving end in the past not on the sending side. in the current case the poison is on the receiving side, and the live_mig_abort is on the sending side afaik | |
| 15:16:32 | bauzas | gibi: are you proposing me to add this directly in File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/manager.py", line 8854, in _do_live_migration self.driver.live_migration(context, instance, dest, ? | |
| 15:17:50 | gibi | if we want to print only in true positive cases then add it in /home/zuul/src/opendev.org/openstack/nova/nova/tests/fixtures/nova.py line 1849, | |
| 15:17:59 | gibi | if we want to print in false positive cases too then in /home/zuul/src/opendev.org/openstack/nova/nova/virt/libvirt/driver.py", line 10071, in live_migration_abort | |
| 15:18:54 | gibi | don't call _get_sender_test_case_id just copy the implementation of it | |
| 15:19:36 | bauzas | yup, I see | |
| 15:21:12 | bauzas | gibi: but we want to know the parent, right? | |
| 15:21:35 | gibi | bauzas: print the first id it gets that will be the name of the test case leaked the thread either directly or indirectly | |
| 15:21:56 | gibi | hm, direclty, hence the walking on the parents | |
| 15:22:25 | gibi | so print the first id that will be the eventlet nova spawn or spawn_n started and have the test case id emeded | |
| 15:22:59 | gibi | we walk the parents as eventlets later can spawn other eventlets which we don't control and therefore we cannot propagate the testcase id there | |
| 15:32:37 | opendevreview | Sylvain Bauza proposed openstack/nova master: DNM: Add logging for leaking out the non-poisoned libvirt testcase https://review.opendev.org/c/openstack/nova/+/872975 | |
| 15:32:43 | bauzas | gibi: ^ | |
| 15:32:59 | bauzas | I said DNM but we could merge it | |
| 15:33:18 | bauzas | instead of us rechecking | |
| 15:46:35 | opendevreview | Dan Smith proposed openstack/nova master: Add docs for stable-compute-uuid behaviors https://review.opendev.org/c/openstack/nova/+/872977 | |
| 16:01:03 | bauzas | #startmeeting nova | |
| 16:01:03 | opendevmeet | Meeting started Tue Feb 7 16:01:03 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:01:03 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:01:03 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:01:13 | bauzas | sorry folks, forgot to remind you of the meeting | |
| 16:01:43 | bauzas | who's around ? | |
| 16:01:57 | Uggla | o/ | |
| 16:02:04 | elodilles | o/ | |
| 16:02:44 | bauzas | I guess we can make a soft start | |
| 16:02:55 | bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | |
| 16:03:02 | bauzas | #topic Bugs (stuck/critical) | |
| 16:03:07 | bauzas | #info No Critical bug | |
| 16:03:11 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 28 new untriaged bugs (+1 since the last meeting) | |
| 16:03:15 | bauzas | #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | |
| 16:03:30 | bauzas | Uggla: fancy getting the bug triage baton for this week N | |
| 16:03:32 | bauzas | ? | |
| 16:04:02 | sean-k-mooney | o/ | |
| 16:04:09 | Uggla | I will be out next week so I would rather postponed if possible | |
| 16:04:49 | bauzas | ack, so artom would you want to continue having the triage baton for an extra week ? | |
| 16:04:59 | Uggla | If not I'll try to do my best till the end of the week. | |
| 16:05:13 | artom | Ah, I completely dropped the ball, didn't I? | |
| 16:05:18 | artom | Yeah, I can keep it | |
| 16:05:26 | bauzas | ++ | |
| 16:05:31 | bauzas | artom: no worries | |
| 16:05:37 | bauzas | and thanks | |
| 16:05:41 | gibi | o/ | |
| 16:06:20 | dansmith | o/ | |
| 16:06:28 | bauzas | ok moving on | |
| 16:06:34 | bauzas | #topic Gate status | |
| 16:06:39 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | |
| 16:06:42 | bauzas | new item | |
| 16:06:52 | bauzas | #link https://etherpad.opendev.org/p/nova-ci-failures Etherpad for tracking CI failures | |
| 16:07:14 | gibi | there is a fairly long list but please add to it if you see failures | |
| 16:07:18 | gibi | that are not on the list | |
| 16:07:31 | bauzas | we have some of them are hit us in some place between the chair and the man | |
| 16:08:05 | bauzas | I think the hardest one is at the bottom of the document | |
| 16:08:05 | dansmith | oof yeah | |
| 16:08:23 | bauzas | I had a lovely morning and a half-afternoon spent on that one | |
| 16:08:42 | bauzas | in the context of a soonish feature freeze, more hands are more than welcome | |
| 16:09:01 | dansmith | I wrote the replace_location test, so I can look into that one.. it's a glance test though. I'm sure it's poking some bug in glance, because until I wrote that we didn't really have any tests for that stuff | |
| 16:09:08 | bauzas | because don't expect your patches to be reviewed if most of the cores are having their days spent on fixing CI problems | |
| 16:09:08 | dansmith | but maybe it is resolvable | |
| 16:10:06 | gibi | dansmith: there is https://bugs.launchpad.net/glance/+bug/1999800 and https://bugs.launchpad.net/glance/+bug/2006473 both location tests | |
| 16:10:17 | bauzas | and yeah, I know, debugging a CI failure isn't exactly the best experience you may have of working on an opensource project, but let's be honest and say that's necessary to have an healthy gate | |
| 16:10:26 | gibi | bauzas: +1 | |
| 16:10:35 | dansmith | okay the former is the same as bauzas' one | |
| 16:10:41 | gibi | yeah probably | |
| 16:11:09 | dansmith | yeah from the logs, the test is clearly doing something legit and glance is rejecting it but shouldn't | |
| 16:11:16 | bauzas | gibi: I created https://bugs.launchpad.net/nova/+bug/2004641 but it seems duplicate of https://bugs.launchpad.net/glance/+bug/1999800 | |
| 16:11:27 | dansmith | might be because it fails to talk to the cirros site occasionally, so maybe we can use an openstack infra url instead | |
| 16:11:43 | dansmith | bauzas: indeed | |
| 16:11:52 | sean-k-mooney | i tought we tried to pull those form provider proxies in ci | |
| 16:11:54 | gibi | bauzas: https://bugs.launchpad.net/tempest/+bug/2004641 and https://bugs.launchpad.net/glance/+bug/2006473 are duplicates but https://bugs.launchpad.net/glance/+bug/1999800 is a separate tc | |
| 16:11:55 | bauzas | I can close my one as duplicate | |
| 16:12:06 | dansmith | bauzas: ++ | |
| 16:12:20 | sean-k-mooney | github is more repliable for downlaoding cirrors images by the way then the cirros site | |
| 16:12:22 | bauzas | I just ideally would like to track that bug in our project | |
| 16:13:01 | dansmith | sean-k-mooney: the cirros page just redirects to the github one | |
| 16:13:16 | sean-k-mooney | oh they finally implmetned that | |
| 16:13:29 | dansmith | sean-k-mooney: and we're just using CONF.image.http_image in that test | |
| 16:14:03 | sean-k-mooney | oh this is not the image pulled by https://github.com/openstack/devstack/blob/master/stackrc#L670-L708 | |
| 16:14:05 | dansmith | the github URL is crazy long with tons of tokens and other values after the redirects it does | |
| 16:14:11 | bauzas | gibi: ack will mark your https://bugs.launchpad.net/nova/+bug/2004641 as duplicate of mine, then | |
| 16:14:12 | dansmith | sean-k-mooney: this is a tempest test | |
| 16:14:33 | sean-k-mooney | right the one with the larger image | |
| 16:14:41 | dansmith | no | |
| 16:14:58 | dansmith | gibi: it's the same test case, different behavior, but I'm guessing its the sameish problem | |
| 16:15:37 | bauzas | ok, you know what, I'll add mine in the tracking etherpad, and we'll figure out | |
| 16:15:49 | bauzas | the three of them are set against Glance either way | |
| 16:15:57 | sean-k-mooney | im surpised that the tempest test is not using the one we prestage in the vm but ok | |
| 16:16:48 | sean-k-mooney | i was expecting CONF.image.http_image to be file:///opt/devstack/data/cirros... | |
| 16:17:05 | gibi | dansmith: yeah probably similar root cause | |
| 16:17:06 | dansmith | sean-k-mooney: it can't be because that is specifically for testing fetching an image server-side from http | |
| 16:17:22 | sean-k-mooney | ah thanks i was missing that context | |