| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-06 | |||
| 05:07:34 | eandersson | Interesting | |
| 05:07:46 | eandersson | An error'd compute due to quota still takes up an allocation | |
| 05:08:24 | eandersson | Preventing heal_allocation from progressing | |
| 05:15:47 | eandersson | Turns out placement isn't in a great place in that region. Probably why we are hitting errors. | |
| 05:39:50 | eandersson | I wish heal_allocations would have an option to skip failed computes. | |
| 05:40:25 | eandersson | When you are in a bad state you basically need to do some mysql magic to figure out "bad" hosts. Otherwise you are fixing them sequentially. | |
| 08:57:03 | kashyap | gibi_ptg: Good luck :-) | |
| 10:36:12 | openstackgerrit | Boris Bobrov proposed openstack/nova master: Also enable iommu for virtio controllers and video in libvirt https://review.opendev.org/684825 | |
| 10:36:13 | openstackgerrit | Boris Bobrov proposed openstack/nova master: Create a controller for qga when SEV is used https://review.opendev.org/693072 | |
| 12:23:44 | sean-k-mooney | ... so the random number generator has been broken in nova for a while... | |
| 12:24:09 | sean-k-mooney | and by a while i mean since https://github.com/openstack/nova/commit/be548a3396ee5f3a78b2b8b59321eb4c4530f7fb merged in queens | |
| 12:25:31 | sean-k-mooney | the comments is in the wrong place | |
| 12:25:33 | sean-k-mooney | https://github.com/openstack/nova/commit/be548a3396ee5f3a78b2b8b59321eb4c4530f7fb#diff-f4019782d93a196a0d026479e6aa61b1L4037-L4071 | |
| 12:25:57 | sean-k-mooney | after teh refactor the rng is only enabled if you also enable the qemu guest agent | |
| 12:26:39 | sean-k-mooney | before it was eneabled if you enabled it in the image and the host was qemu or kvm | |
| 12:27:52 | sean-k-mooney | kashyap: ^ we proably need to fix that but we talked before about enableing the rng by default. it should be two seperate patches but do you think we shoudl still enable it by default | |
| 12:38:03 | openstack | Launchpad bug 1851497 in OpenStack Compute (nova) "rng is only enabled if qemu guest agent is enabled" [Medium,Triaged] - Assigned to sean mooney (sean-k-mooney) | |
| 12:38:03 | sean-k-mooney | i have filed a bug for it here https://bugs.launchpad.net/nova/+bug/1851497 | |
| 12:54:46 | sean-k-mooney | efried: hi so http://paste.openstack.org/show/785846/ seams to now work. im going to kick it off on a clean vm to test but that stacked and i could boot vms | |
| 13:16:13 | efried | sean-k-mooney: cool beans | |
| 13:21:39 | efried | sean-k-mooney: I just need to update that enable_plugin line for my purposes, right? The rest ought to be unchanged from yesterday? | |
| 13:22:12 | sean-k-mooney | yep | |
| 13:22:41 | sean-k-mooney | ill proably merge the patch today or tomorow but i want to do a clean install first to make sure i have not missed something | |
| 13:23:18 | sean-k-mooney | i need to actully set up a gate job to actully test it rather then manually testing it every time i update it | |
| 13:23:38 | efried | okey. Were you able to repro the last problem I had yesterday where nova thought I didn't have a recent enough qemu? | |
| 13:23:55 | efried | like, it was looking in the wrong place or something | |
| 13:24:23 | sean-k-mooney | not exactly but i noticed i was not unistalling qemu from the package manager before installing it for source | |
| 13:24:27 | sean-k-mooney | so i fixed that | |
| 13:24:33 | sean-k-mooney | i had that for libvirt but not qemu | |
| 13:25:06 | efried | ah | |
| 13:25:12 | efried | sounds like that would probably do it. | |
| 13:25:52 | efried | k, I'm running now. Cleaned first to get rid of compiled libvirt/qemu -- I needed that because of new compile flaggage, rite? | |
| 13:26:01 | efried | for apparmor | |
| 13:26:43 | sean-k-mooney | actully no i have not added that yet | |
| 13:27:07 | sean-k-mooney | but i found the bits i needed to set to disable the use of polkit | |
| 13:27:30 | sean-k-mooney | a clean does not hurt it just will be slower | |
| 13:27:35 | efried | ight. We'll see how this goes :) | |
| 13:29:04 | sean-k-mooney | by the way to go back to normal you just need to unstack then comment out the pluging. it should remove the libvirt and qemu it installed on unstack but wont delete the source in /opt/stack | |
| 13:29:19 | efried | cool | |
| 13:29:23 | efried | I'll never go back to normal | |
| 13:29:26 | efried | This is my new normal | |
| 13:29:41 | sean-k-mooney | :) | |
| 14:00:40 | efried | sean-k-mooney: no idea why this should be failing differently on my otherwise-identical VMs, but one is hitting the libvirt-python install failure again, and the other is hitting the /var/run/libvirt/libvirt-sock ENOENT again. | |
| 14:12:02 | sean-k-mooney | so the libvirt python failure is complaingin about the libvit.so right with pkg-config or somthing like that and the other case libvirt is not running? | |
| 14:13:12 | efried | sean-k-mooney: Maybe I'm not looking at the right patch set. I don't see a delta where you're purging qemu for instance. | |
| 14:14:17 | sean-k-mooney | its here https://review.opendev.org/#/c/693094/3/devstack/libs/libvirt@102 | |
| 14:14:30 | efried | ah, k | |
| 14:15:32 | efried | well, now I got both to fail the same way. The one that was failing with ENOENT is now failing on the libvirt-python thing. | |
| 14:15:52 | efried | I cut it over to USE_PYTHON3 | |
| 14:16:28 | sean-k-mooney | ok. am if you look in /usr/lib is there a libvirt.so | |
| 14:17:05 | efried | yes | |
| 14:17:20 | sean-k-mooney | ok if you unstack it should remove it | |
| 14:17:55 | efried | and that's what I want? | |
| 14:17:58 | sean-k-mooney | if it does not run " sudo rm -f /urs/lib/libvirt* | |
| 14:18:21 | sean-k-mooney | well we want to make sure its the libvirt.so you compiled for a start | |
| 14:18:28 | sean-k-mooney | it shoudl be | |
| 14:18:38 | efried | fwiw the date stamp implies that it is. | |
| 14:19:13 | efried | everything /usr/lib/libvirt* has a now-ish date stamp | |
| 14:19:35 | efried | and yes, unstacking removes everything /usr/lib/libvirt* | |
| 14:19:51 | sean-k-mooney | ok so if you run "pkg-config libvirt ; echo $?" | |
| 14:20:05 | sean-k-mooney | im guessin git now prints a non 0 value right | |
| 14:20:38 | sean-k-mooney | if so can you cd in /opt/stack/libvit and do sudo make install | |
| 14:20:42 | sean-k-mooney | then run it again | |
| 14:21:38 | efried | yes, confirm all of that. | |
| 14:22:10 | sean-k-mooney | so if you run it after you do make install is it 0 or non 0 | |
| 14:22:18 | efried | 0 | |
| 14:22:34 | sean-k-mooney | ok and if you manually to the pip install of libvirt-python? | |
| 14:22:35 | efried | and the checksum of the lib is the same as on the other system where I didn't unstack and manually `make install` | |
| 14:22:48 | sean-k-mooney | ya it should be | |
| 14:23:06 | efried | Found 0 functions in libvirt-override-api.xml | |
| 14:23:06 | efried | Found 475 functions in /usr/share/libvirt/api/libvirt-api.xml | |
| 14:23:06 | efried | /usr/bin/python generator.py libvirt /usr/share/libvirt/api/libvirt-api.xml | |
| 14:23:06 | efried | /usr/bin/pkg-config --print-errors --atleast-version=0.9.11 libvirt | |
| 14:23:06 | efried | same error: | |
| 14:23:07 | efried | virTypedParameterPtr:1 | |
| 14:23:07 | efried | Missing type converters: | |
| 14:23:07 | efried | Generated 385 wrapper functions | |
| 14:23:08 | efried | ERROR: failed virConnectSetIdentity | |
| 14:23:09 | efried | error: command '/usr/bin/python' failed with exit status 1 | |
| 14:24:13 | sean-k-mooney | if you do pip show libvirt-python | |
| 14:24:18 | sean-k-mooney | what version do you see | |
| 14:24:22 | sean-k-mooney | im using 5.7.0 | |
| 14:24:48 | efried | When we hit this yesterday, it went away magically by itself | |
| 14:24:57 | efried | 5.5.0 | |
| 14:25:18 | efried | it's trying to install 5.7.0 that breaks | |
| 14:25:46 | sean-k-mooney | if you try and install 5.5.0 does it work | |
| 14:25:56 | sean-k-mooney | e.g. if you remove it and add it | |
| 14:26:53 | efried | here's a thing: I'm running py2 pip | |
| 14:27:02 | sean-k-mooney | so am i | |
| 14:27:19 | sean-k-mooney | pip version 9.0.3 | |
| 14:27:25 | sean-k-mooney | the one devstack installs | |
| 14:28:16 | efried | if I uninstall 5.5.0 and try to reinstall, it fails (at 5.5.0 and at 5.7.0) the same way | |
| 14:28:44 | efried | also with pip3 | |
| 14:30:07 | efried | what if I tried setting the plugin to build libvirt 5.7.0? | |
| 14:30:18 | sean-k-mooney | oh... | |
| 14:30:29 | sean-k-mooney | yes you could do that | |
| 14:30:52 | sean-k-mooney | but its passing form me because its using the cached wheel | |
| 14:31:04 | sean-k-mooney | if i try to use 5.5.0 i get your error | |
| 14:31:15 | efried | like you already had 5.7.0 installed before you started? | |
| 14:31:18 | sean-k-mooney | actully i get | |