| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-25 | |||
| 13:23:56 | gibi | thanks | |
| 13:24:21 | sean-k-mooney1 | i didnt get a responce on irc downstream regarding the live migration timeline ill see if i can get one and let you know | |
| 13:24:37 | sean-k-mooney1 | gibi: there are some patches proposed currently to the qemu dev list | |
| 13:25:02 | sean-k-mooney1 | but im not sure when they expect to finalise the desgin and what version they are targeting | |
| 13:25:19 | gibi | thanks | |
| 13:32:34 | zigo | In Sid, nova-compute tries to connect to /var/run/libvirt/virtqemud-sock instead of /var/run/libvirt/libvirt-sock any idea why? | |
| 13:33:07 | zigo | Then the log says: | |
| 13:33:07 | zigo | DEBUG nova.virt.libvirt.host [-] Connecting to libvirt: qemu:///system _get_new_connection /usr/lib/python3/dist-packages/nova/virt/libvirt/host.py:434 | |
| 13:33:07 | zigo | but it doesn't go any further for me ... | |
| 13:33:44 | zigo | Normally, it should be doing: "Registering for lifecycle events" right? | |
| 13:34:06 | zigo | gibi: Any idea what's going on for me? | |
| 13:35:44 | gibi | zigo: I think nova only configures the url qemu:///system but does not specify the name of the socket | |
| 13:36:02 | zigo | gibi: So that'd be a problem in python3-libvirt then? | |
| 13:36:12 | gibi | I suspect that yes | |
| 13:37:01 | sean-k-mooney1 | gibi: there you go https://blueprints.launchpad.net/nova/+spec/port-scoped-sriov-numa-affinity sorry i forgot to do that before | |
| 13:37:02 | gibi | what is your python3-libvirt version? | |
| 13:37:16 | gibi | sean-k-mooney1: awesome, thanks. I will approve it | |
| 13:38:36 | zigo | I haven't seen instances of a string with "virtqemud-sock" in libvirt, neither in nova, so I really wonder where that is... | |
| 13:41:42 | zigo | sean-k-mooney1: Any idea where the socket path is comming from? | |
| 13:41:54 | zigo | Something IMO needs a patch somewhere ... | |
| 13:43:07 | gibi | zigo: I've just tried in my local Sid with python3-libvirt 6.1.0-1+b3 and it works (I don't have /var/run/libvirt/virtqemud-sock) | |
| 14:00:28 | zigo | gibi: Thanks, I'm not sure what's going on with my setup then, but will try to find out. | |
| 14:01:38 | sean-k-mooney | zigo: sorry was on a 1:1 | |
| 14:01:41 | sean-k-mooney | reading back | |
| 14:02:36 | sean-k-mooney | hum not really familar with virtqemud | |
| 14:02:57 | sean-k-mooney | i would assume that this is contoled by /etc/libvirt/qemu.conf ? | |
| 14:03:13 | nightmare_unreal | sean-k-mooney: i checked other osc-plugins but it does not look like they directly deal with clouds.yaml . I am talking in regards to this : https://storyboard.openstack.org/#!/story/2008553 | |
| 14:04:29 | sean-k-mooney | zigo: https://libvirt.org/daemons.html#modular-driver-daemons | |
| 14:04:41 | sean-k-mooney | zigo: apprently there is a /etc/libvirt/virt${DRIVER}d.conf you can use | |
| 14:05:02 | sean-k-mooney | and https://libvirt.org/daemons.html#modular-sockets | |
| 14:05:05 | sean-k-mooney | covers the sockets | |
| 14:05:19 | zigo | sean-k-mooney: Thanks, will read on. | |
| 14:05:46 | sean-k-mooney | zigo: if i understand thsi properly libvirt now uspport running the virt drivers in indepent deamon form the main libvirt deamon | |
| 14:06:14 | zigo | It's obviously badly configured on my side ... | |
| 14:07:06 | zigo | Though: | |
| 14:07:06 | zigo | # ls /etc/libvirt/virtlo* | |
| 14:07:06 | zigo | /etc/libvirt/virtlockd.conf /etc/libvirt/virtlogd.conf | |
| 14:07:18 | sean-k-mooney | i suspect that it was prviously configured in monolich mode and is not in modular mode | |
| 14:09:06 | sean-k-mooney | it looks like libvirt is planning to eventually remove the monolithic daemon | |
| 14:10:31 | sean-k-mooney | zigo: is debian sid the unstable testing branch of debian | |
| 14:10:50 | sean-k-mooney | bulleye it the latest lts right | |
| 14:11:43 | sean-k-mooney | i suspect that the libvirt version has been bumped in sid and it now default to modular which is causing your current issue | |
| 14:14:35 | zigo | sean-k-mooney: I'm trying to get OpenStack to work in Sid/Unstable because we're going to freeze Bullseye in a few days from now, yes. | |
| 14:14:53 | zigo | And lots of things are broken ... :( | |
| 14:15:03 | sean-k-mooney | ah right well it just got https://tracker.debian.org/news/1223928/libvirt-690-4-migrated-to-testing/ | |
| 14:15:18 | sean-k-mooney | although it had 6.9.0-1 in late november | |
| 14:15:36 | sean-k-mooney | zigo: is bullseye running python 3.9? | |
| 14:15:42 | zigo | Yeah. | |
| 14:15:54 | sean-k-mooney | zigo: i tested with fedroa rawhid in decmeber and we cant run master on it | |
| 14:15:59 | zigo | Eventlet and Amqp broke... | |
| 14:16:02 | sean-k-mooney | yep | |
| 14:16:16 | zigo | I think we got Eventlet back to working, but Amqp still breaks with ssl. | |
| 14:16:31 | sean-k-mooney | there was a new eventlet release on december 13th but not sure its fully working | |
| 14:16:34 | sean-k-mooney | ya | |
| 14:16:36 | zigo | TypeError: _wrap_socket() argument 'sock' must be _socket.socket, not SSLSocket | |
| 14:16:43 | zigo | I'm really not sure how to fix that one... | |
| 14:16:50 | sean-k-mooney | yep the bug is open for a long time for that | |
| 14:16:54 | sean-k-mooney | on sec | |
| 14:17:00 | zigo | Though I'm currently trying to fix the libvirt issue first. | |
| 14:17:02 | sean-k-mooney | there is an issue with dnspython | |
| 14:17:07 | zigo | Oh ... | |
| 14:17:27 | sean-k-mooney | https://github.com/eventlet/eventlet/issues/632 | |
| 14:17:31 | sean-k-mooney | https://github.com/eventlet/eventlet/issues/619 | |
| 14:17:41 | sean-k-mooney | eventlet need dnspyton<2.0.0 | |
| 14:20:11 | sean-k-mooney | https://github.com/eventlet/eventlet/pull/639 partly fixes things but not fully and its not merged | |
| 14:22:30 | zigo | sean-k-mooney: I have the patch already in the Debian package. | |
| 14:23:19 | sean-k-mooney | i see and still no joy | |
| 14:23:43 | zigo | Though do you think this one is an eventlet problem too, rather than in AMQP? | |
| 14:23:43 | zigo | TypeError: _wrap_socket() argument 'sock' must be _socket.socket, not SSLSocket | |
| 14:24:25 | sean-k-mooney | that one im not sure about | |
| 14:24:37 | sean-k-mooney | its a similar error but i have not seen it yet | |
| 14:25:03 | sean-k-mooney | if you find out i would like to know however | |
| 14:25:53 | zigo | :) | |
| 14:26:08 | zigo | One thing at a time, I guess. FIrst, that libvirt issue ... | |
| 14:26:48 | sean-k-mooney | ya i might need to flag the libvirt issue internally too. | |
| 14:27:06 | sean-k-mooney | i dont think we had planned to need to run libvirt in multiple continers | |
| 14:27:16 | sean-k-mooney | or have multiple processes in the libvirt container. | |
| 14:27:25 | sean-k-mooney | beyond virtlogd | |
| 14:32:50 | kashyap | zigo: Modular daemons in libvirt have been around for about a year-and-a-half | |
| 14:34:09 | zigo | Definitively a configuration problem, on my side. At first attempt, nova tries the correct socket path. Then I comment out the libvirtd_opts="-l" to make libvirt work propertly (since it defaults to socket activation), then systemctl start libvritd, and then nova tries the wrong socket path ... | |
| 15:18:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach https://review.opendev.org/c/openstack/nova/+/770246 | |
| 16:32:50 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/c/openstack/nova/+/602432 | |
| 17:15:19 | stephenfin | If anyone wants to make themselves really miserable, take a look at https://review.opendev.org/c/openstack/placement/+/772335/ | |
| 17:15:46 | stephenfin | Somehow, config that I set for oslo.policy is being unset/overridden when I run tests in parallel | |
| 17:16:21 | stephenfin | and I haven't a clue what's bleeding across, despite spending the bulk of the day looking :) | |
| 17:16:31 | stephenfin | stupid global stuff | |
| 17:16:53 | openstackgerrit | Lance Bragstad proposed openstack/placement master: policy: Remove the deprecated 'placement' rule https://review.opendev.org/c/openstack/placement/+/772333 | |
| 17:18:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: libvirt: Define and emit DeviceRemovedEvent and DeviceRemovalFailedEvent https://review.opendev.org/c/openstack/nova/+/749929 | |
| 17:18:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: libvirt: add AsyncDeviceDetachEventsHandler https://review.opendev.org/c/openstack/nova/+/772381 | |
| 17:19:49 | lbragstad | stephenfin i'm not sure if it's related - but implementing some additional gabbi tests for inventories stumped my late last week | |
| 17:20:06 | stephenfin | lbragstad: The one and the same | |
| 17:20:28 | stephenfin | That was an attempted fix for same and it works...if you don't run things in parallel | |
| 17:20:46 | lbragstad | i was trying to implement tests for all the personas in the legacy rbac tests - per gmann's suggestion | |
| 17:20:47 | lbragstad | https://review.opendev.org/c/openstack/placement/+/760238/7 | |
| 17:21:01 | lbragstad | just like what i did here - https://review.opendev.org/c/openstack/placement/+/760240/19/placement/tests/functional/gabbits/resource-provider-legacy-rbac.yaml | |
| 17:21:53 | lbragstad | and in the process of testing inventory updates - the placement API consistently gives me a 409 | |
| 17:22:23 | lbragstad | i tried cleaning up all possible resources in the test and starting from scratch, but nothing worked | |
| 17:22:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: libvirt: allow querying devices from the persistent domain https://review.opendev.org/c/openstack/nova/+/772383 | |
| 17:22:49 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: libvirt: parse alias out from device config https://review.opendev.org/c/openstack/nova/+/772384 | |
| 17:22:56 | stephenfin | lbragstad: Does it work if you run a single test by itself? | |