| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-10-23 | |||
| 16:17:14 | melwitt | heh | |
| 16:17:20 | melwitt | on it | |
| 16:17:35 | stephenfin | The gate is still hopelessly broken due to this libvirt/eventlet issue but that's one thing off the list | |
| 16:20:00 | melwitt | ++ | |
| 16:20:58 | lyarwood | stephenfin: sorry wasn't watching irc upstream | |
| 16:21:05 | lyarwood | stephenfin: what's the eventlet issue? | |
| 16:21:28 | stephenfin | ||
| 16:21:28 | stephenfin | <stephenfin> the failing one uses eventlet===0.28.0 | |
| 16:21:28 | stephenfin | ||
| 16:21:28 | stephenfin | ||
| 16:21:30 | stephenfin | <stephenfin> so eventlet has borked us \o/ | |
| 16:21:55 | stephenfin | I think it's eventlet to blame, though the diff between those versions doesn't show anything particularly egregious | |
| 16:22:13 | stephenfin | mypy and greenlet were also bumped so they might be more to blame | |
| 16:33:21 | stephenfin | artom++ Good spot on https://review.opendev.org/#/c/758157/. A flaw in my testing | |
| 18:02:42 | openstackgerrit | Merged openstack/nova stable/stein: libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available https://review.opendev.org/747611 | |
| 18:10:05 | melwitt | dansmith: if you're around, fix for the gate is at https://review.opendev.org/759466 | |
| 18:11:04 | dansmith | melwitt: ack | |
| 18:11:35 | melwitt | thanks! | |
| 18:16:15 | johnsom | Yeah, as mentioned above, nova is crashing in the Octavia gates. The traceback does point to eventlet. | |
| 18:16:16 | johnsom | https://zuul.opendev.org/t/openstack/build/2828ba97f9d54d458facd777d91dd36e/log/controller/logs/screen-n-cpu.txt#899 | |
| 18:18:56 | melwitt | urgh. so that's what stephenfin was talking about earlier. I didn't realize it was a 100% rate failure | |
| 18:20:07 | melwitt | looking into it | |
| 18:28:13 | artom | stephenfin, I do what I can:) | |
| 18:46:36 | johnsom | Here is a paste with the packages that are different between the last good run (yesterday) and the failing run today: http://paste.openstack.org/show/799337/ | |
| 18:56:23 | openstackgerrit | melanie witt proposed openstack/nova master: DNM test eventlet 0.26.1 https://review.opendev.org/759506 | |
| 18:56:37 | melwitt | ah, thank you | |
| 19:00:33 | openstackgerrit | melanie witt proposed openstack/nova master: DNM test eventlet 0.26.1 https://review.opendev.org/759506 | |
| 19:34:07 | openstackgerrit | Merged openstack/nova stable/train: compute: Validate a BDMs disk_bus when provided https://review.opendev.org/744552 | |
| 19:34:13 | openstackgerrit | Merged openstack/nova stable/ussuri: Fix invalid assert_has_calls https://review.opendev.org/751752 | |
| 19:36:53 | openstackgerrit | melanie witt proposed openstack/nova master: DNM try using autowrap_names instead of autowrap https://review.opendev.org/759513 | |
| 20:08:31 | openstackgerrit | Merged openstack/nova master: docs: Make JSON valid https://review.opendev.org/759466 | |
| 20:56:29 | openstackgerrit | Billy Olsen proposed openstack/nova stable/ussuri: add functional regression test for bug #1888395 https://review.opendev.org/759522 | |
| 20:56:30 | openstack | bug 1888395 in OpenStack Compute (nova) ussuri "shared live migration of a vm with a vif is broken in train" [High,In progress] https://launchpad.net/bugs/1888395 | |
| 21:10:57 | openstackgerrit | Billy Olsen proposed openstack/nova stable/ussuri: Set migrate_data.vifs only when using multiple port bindings https://review.opendev.org/759151 | |
| 23:22:04 | openstackgerrit | Billy Olsen proposed openstack/nova stable/train: add functional regression test for bug #1888395 https://review.opendev.org/759533 | |
| 23:22:05 | openstack | bug 1888395 in OpenStack Compute (nova) ussuri "shared live migration of a vm with a vif is broken in train" [High,In progress] https://launchpad.net/bugs/1888395 - Assigned to Billy Olsen (billy-olsen) | |
| 23:23:24 | openstackgerrit | Billy Olsen proposed openstack/nova stable/train: Set migrate_data.vifs only when using multiple port bindings https://review.opendev.org/759153 | |
| #openstack-nova - 2020-10-24 | |||
| 01:23:54 | openstackgerrit | Merged openstack/nova master: [doc]: Fix glance image_metadata link https://review.opendev.org/759100 | |
| 01:46:25 | openstackgerrit | Merged openstack/nova master: CinderFixture: Return a driver_type of `fake` within connection_info https://review.opendev.org/758238 | |
| 03:11:16 | openstackgerrit | guolei proposed openstack/nova master: Fix GPU reschedule bug https://review.opendev.org/759544 | |
| 06:12:16 | openstackgerrit | YumengBao proposed openstack/nova-specs master: SRIOV Enabled Nic Support Specification https://review.opendev.org/742785 | |
| 06:56:57 | openstackgerrit | melanie witt proposed openstack/nova master: DNM test eventlet revert from 0.28.0 to 0.26.1 https://review.opendev.org/759506 | |
| #openstack-nova - 2020-10-25 | |||
| 01:19:36 | openstackgerrit | Wenping Song proposed openstack/nova master: Remove redundant code in instance delete https://review.opendev.org/759577 | |
| 03:56:10 | openstackgerrit | Billy Olsen proposed openstack/nova stable/ussuri: add functional regression test for bug #1888395 https://review.opendev.org/759522 | |
| 03:56:11 | openstack | bug 1888395 in OpenStack Compute (nova) ussuri "shared live migration of a vm with a vif is broken in train" [High,In progress] https://launchpad.net/bugs/1888395 - Assigned to Billy Olsen (billy-olsen) | |
| 04:49:52 | openstackgerrit | guolei proposed openstack/nova master: Make guest's cpu pinned more balanced on each numa node https://review.opendev.org/759581 | |
| 14:46:13 | Haunted330 | This channel is closed. You are going to have to leave. | |
| #openstack-nova - 2020-10-26 | |||
| 02:18:43 | openstackgerrit | Billy Olsen proposed openstack/nova stable/ussuri: Set migrate_data.vifs only when using multiple port bindings https://review.opendev.org/759151 | |
| 08:10:29 | bauzas | good morning Nova | |
| 08:57:08 | gibi | bauzas: good mornin | |
| 09:04:21 | bauzas | gibi: well, it's raining, we're on winter TZ, not a lot of sunlight, and I just looked that maybe we should be again on lockdown by Saturday | |
| 09:04:34 | bauzas | :p | |
| 09:04:39 | bauzas | so, a good one ? | |
| 09:04:40 | bauzas | :p | |
| 09:05:30 | bauzas | at least the coffee is good :p | |
| 09:05:39 | gibi | it was raining the whole weekend here and the TZ change hit here too. Our governent now tries not to do anyithing about the virus, so even though the numbers are raising rapidly nobody talks about lockdown | |
| 09:07:00 | bauzas | gibi: nw, I was joking ;-) | |
| 09:07:27 | gibi | it could be worse :) | |
| 09:07:50 | bauzas | indeed ;) | |
| 09:08:13 | kashyap | gibi: Yeah, wholesale lockdown is complex :-( | |
| 09:09:29 | kashyap | gibi: One the one side, it is effective at controlling the spread of virus; on the other, complete sectors of economies become a train wreck that is Jupiter-sized | |
| 09:09:51 | openstackgerrit | Wenping Song proposed openstack/nova-specs master: Support vGPU management by Cyborg https://review.opendev.org/750116 | |
| 09:13:42 | gibi | kashyap: we had a controlled lockdown during spring, we have nothing right now (exepct some lukewarm rule about masks). I'm not an economist but I imagine there is a lot of possible actions between full lockdown and nothing | |
| 09:14:36 | kashyap | gibi: Yeah, I hear you; controlled lockdown makes sense. | |
| 09:14:41 | gibi | e.g. we had controlled timeslots for groceries during sping. | |
| 09:14:47 | gibi | spring | |
| 09:15:10 | frickler | bauzas: gibi: do you have a bug report already for the eventlet update issue (https://review.opendev.org/759506)? seeing this coming up in multiple channels now because it seems to break almost everyone | |
| 09:15:14 | kashyap | (Full lockdown is also a "policital suicide") | |
| 09:16:20 | gibi | frickler: I was on PTO on Friday I need to look back what happened there | |
| 09:16:37 | gibi | kashyap: I agree on the full lockdown thing | |
| 09:17:27 | kashyap | gibi: Right now, here in Belgium is a "controlled lockdown" - all cafes/bars/restaurants closed for a month; no going out from midnight to 05:00 AM (it's even stricter in Brussels); etc. | |
| 09:18:09 | gibi | frickler: we have https://review.opendev.org/#/q/topic:dnm-eventlet-revert+(status:open+OR+status:merged) but I don't see a bugreport | |
| 09:18:57 | openstackgerrit | Wenping Song proposed openstack/nova-specs master: Support vGPU management by Cyborg https://review.opendev.org/750116 | |
| 09:19:12 | gibi | kashyap: that sounsd something I can welcome here too. We have football games with full house here, crazy | |
| 09:19:58 | openstack | Launchpad bug 1901383 in OpenStack Compute (nova) "nova-compute fails with "TypeError: Parameterized generics cannot be used with class" with python 3.6 and libvirt-python 6.8.0" [Critical,New] | |
| 09:19:58 | gibi | frickler: is this the bug? https://bugs.launchpad.net/nova/+bug/1901383 | |
| 09:20:39 | kashyap | gibi: Oh dear ... that "full house" part sounds like a petri dish to let the virus "flourish" | |
| 09:22:02 | frickler | gibi: yes, except it doesn't seem to be related to either py36 nor libvirt, but eventlet, as the jobs are passing with the cap melwitt made | |
| 09:25:13 | gibi | frickler: do you mean https://review.opendev.org/#/c/759552 as the cap? this caps libvirt-python | |
| 09:29:37 | bauzas | frickler: I thought melwitt reported the problem | |
| 09:30:04 | lyarwood | FWIW I'm rasing the libvirt-python issue with danpb downstream now | |
| 09:30:05 | frickler | gibi: oh, indeed, I didn't notice there was a new version | |
| 09:30:24 | openstack | Launchpad bug 1901383 in OpenStack Compute (nova) "nova-compute fails with "TypeError: Parameterized generics cannot be used with class" with python 3.6 and libvirt-python 6.8.0" [Critical,New] | |
| 09:30:24 | bauzas | frickler: gibi: https://bugs.launchpad.net/nova/+bug/1901383 | |
| 09:30:33 | lyarwood | we might want to land the revert anyway while we get a bug filed against libvirt-python etc | |
| 09:30:38 | bauzas | oh man, I'm burned by gibi | |
| 09:32:25 | kashyap | Yeah, perhaps copy/pasting (with credit) what Mel wrote in here - https://gitlab.com/libvirt/libvirt-python/-/issues | |
| 09:57:48 | gibi | lyarwood: I agree | |
| 10:29:13 | lyarwood | gibi: any objections if I pushed an updated PS for the revert? | |
| 10:29:38 | bauzas | lyarwood: I'm also good with | |
| 10:29:56 | lyarwood | ack, lets do that then | |
| 10:32:10 | lyarwood | gibi / bauzas ; okay done - https://review.opendev.org/759552 | |
| 10:32:29 | lyarwood | melwitt: ^ btw once you're online, we are going to push ahead with the revert to unblock the gate | |
| 10:33:56 | bauzas | lyarwood: also, we should make sure that libvirt-python==6.8.0 shouldn't be used in nova | |
| 10:34:30 | lyarwood | bauzas: it's not | |
| 10:34:44 | lyarwood | bauzas: oh wait, you mean an actual startup check? | |
| 10:35:00 | bauzas | no | |
| 10:35:10 | lyarwood | bauzas: ah just a requirements check | |
| 10:35:12 | bauzas | actually https://github.com/openstack/nova/blob/master/requirements.txt | |
| 10:35:15 | lyarwood | yeah | |