| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-05 | |||
| 22:36:34 | sean-k-mooney | i should not really depend on the transitive depencies like that | |
| 22:37:58 | efried | sean-k-mooney: I have an apparmor_parser in /sbin | |
| 22:38:44 | sean-k-mooney | hum ok it should have remvoed the proifle then but ill document the other way in anycase | |
| 22:39:41 | efried | the file it's referring to doesn't exist fwiw | |
| 22:41:07 | melwitt | mriedem: you were right, that solved the 403. now it's on to 409 "Multiple possible networks found, use a Network ID to be more specific". I'm sifting through NeutronFixture changes | |
| 22:41:54 | efried | sean-k-mooney: with those extra lines, now stack fails when libvirt-bin won't start. | |
| 22:41:55 | sean-k-mooney | efried: there is still a profile in /etc/apparmor.d/libvirt/ in my case | |
| 22:42:32 | efried | File /etc/apparmor.d/usr.sbin.libvirtd not found, skipping... | |
| 22:42:32 | efried | +/opt/stack/devstack-plugin-libvirt-qemu/devstack/libs/libvirt:install_libvirt_qemu_src:210 sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd | |
| 22:42:32 | efried | that line is skipping | |
| 22:43:05 | efried | Nov 05 16:40:14 nucle systemd[1]: libvirtd.service: Failed with result 'exit-code'. | |
| 22:43:05 | efried | Nov 05 16:40:14 nucle systemd[1]: libvirtd.service: Start request repeated too quickly. | |
| 22:43:05 | efried | Nov 05 16:40:14 nucle systemd[1]: Stopped Virtualization daemon. | |
| 22:43:05 | efried | Nov 05 16:40:14 nucle systemd[1]: libvirtd.service: Scheduled restart job, restart counter is at 5. | |
| 22:43:05 | efried | Nov 05 16:40:14 nucle systemd[1]: libvirtd.service: Service hold-off time over, scheduling restart. | |
| 22:43:05 | efried | the libvirt-bin service is failing thusly: | |
| 22:43:06 | efried | Nov 05 16:40:14 nucle systemd[1]: Failed to start Virtualization daemon. | |
| 22:43:08 | sean-k-mooney | ya it has obviously moved | |
| 22:43:45 | efried | /opt/stack/libvirt/src/security/apparmor/usr.sbin.libvirtd | |
| 22:43:45 | efried | /etc/apparmor.d/cache/usr.sbin.libvirtd | |
| 22:43:45 | efried | I see these | |
| 22:43:46 | sean-k-mooney | do you have anything in the journal | |
| 22:44:18 | efried | Assuming I would find it with journalctl -u libvirt-bin, no | |
| 22:44:35 | sean-k-mooney | it would be libvirtd-bin | |
| 22:44:43 | sean-k-mooney | but i dont think that is where its logging | |
| 22:44:48 | efried | either way | |
| 22:44:49 | efried | no | |
| 22:45:03 | sean-k-mooney | ok give me a sec to push this up and test it | |
| 22:45:14 | efried | crap, I f'ed up the libvirtd.conf hack, sec. | |
| 22:47:50 | melwitt | mriedem: ok, looks like I just needed more of that change and now things are working. gonna try backporting the whole thing | |
| 22:48:57 | mriedem | melwitt: which target branch? is this upstream? | |
| 22:49:06 | mriedem | because i don't think we want to backport that whole thing... | |
| 22:50:09 | melwitt | mriedem: this is downstream queens. really? ok | |
| 22:50:19 | mriedem | well, it's big | |
| 22:50:26 | mriedem | note you already have an abandoned backport of that upstream on stable/queens | |
| 22:50:38 | mriedem | https://review.opendev.org/#/c/599764/ | |
| 22:51:05 | melwitt | cause it looks like I need the bits removing the validate_networks stub, add the shared network stuff, and add stuff from this change too https://review.opendev.org/#/c/585385/15/nova/tests/fixtures.py | |
| 22:51:18 | melwitt | lol welp | |
| 22:51:25 | mriedem | you can't just do fake_network.set_stub_network_methods again? | |
| 22:51:42 | melwitt | maybe I can. I have zero memory of doing that before | |
| 22:51:45 | melwitt | I'll try it | |
| 22:52:05 | melwitt | that would be way easier | |
| 22:54:09 | sean-k-mooney | efried: im testing https://review.opendev.org/#/c/693094/1 now | |
| 22:55:02 | sean-k-mooney | efried: assuming that correct the issue ill see about setting up a tempest gate jobs that uses it and then you can just reuse/extend it for your own stuff | |
| 23:05:31 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: doc: Fix supported version descriptions https://review.opendev.org/693097 | |
| 23:25:58 | sean-k-mooney | efried: looks like i should be adding "--with-apparmor_profiles" when compiling to have it install them correctly. | |
| 23:26:54 | sean-k-mooney | of cource the only mention of that flag in the libvirt repo is in the configure script ... | |
| 23:27:30 | sean-k-mooney | sometimes i forget how "fun" C is | |
| 23:34:26 | efried | sean-k-mooney: how do I check my version of qemu? | |
| 23:35:00 | efried | but the plugin should have set up 4.1.0, no? | |
| 23:35:00 | efried | Nova requires QEMU version 2.8.0 or greater. | |
| 23:35:00 | efried | n-cpu failed to start with | |
| 23:35:35 | sean-k-mooney | correct | |
| 23:35:41 | sean-k-mooney | http://paste.openstack.org/show/785829/ | |
| 23:35:50 | sean-k-mooney | i have 4.1 | |
| 23:39:31 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Remove redundant call to get/create default security group https://review.opendev.org/692907 | |
| 23:39:31 | openstack | bug 1824435 in OpenStack Compute (nova) stein "fill_virtual_interface_list migration fails on second attempt" [Medium,In progress] https://launchpad.net/bugs/1824435 - Assigned to melanie witt (melwitt) | |
| 23:39:31 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Add regression test for bug 1824435 https://review.opendev.org/692906 | |
| 23:45:43 | sean-k-mooney | efried: ill try running this all again on a clean ububut cloud image in the morning and ill provide you with the local.conf and anychagnes that are needed on the system if any in an update to the readme. | |
| 23:46:17 | sean-k-mooney | one way or another its in my best interest to get this documented and updated in the repo because i will need this again at somepoint in the futrue | |
| 23:53:16 | efried | sean-k-mooney: I gotta run and break some arms, but when I run the qemu commands I see 4.1, but obv nova thinks I've got something lesser | |
| 23:53:27 | efried | anyway, thanks for the help so far, I'll bug you again tomorrow. | |
| 23:54:32 | sean-k-mooney | no worries sorry it didnt "just work" | |
| 23:55:02 | efried | :) | |
| #openstack-nova - 2019-11-06 | |||
| 02:37:40 | openstackgerrit | Archit Modi proposed openstack/nova master: Revert "openstack server create" to "nova boot" in nova docs https://review.opendev.org/693065 | |
| 02:40:19 | gibi_ptg | the nova ptg table: | |
| 02:40:21 | gibi_ptg | http://vogon.no-ip.org/gibizer/ptg/signal-attachment-2019-11-06-103414_001.jpeg | |
| 02:40:26 | gibi_ptg | http://vogon.no-ip.org/gibizer/ptg/signal-attachment-2019-11-06-103414_002.jpeg | |
| 02:40:42 | gibi_ptg | so far we can fit | |
| 04:28:31 | eandersson | We just deployed the latest nova patches for Rocky | |
| 04:28:34 | eandersson | and are getting | |
| 04:28:36 | eandersson | > Could not retrieve compute node resource provider XXX and therefore unable to error out any instances stuck in BUILDING state. | |
| 04:28:41 | eandersson | On hundreds of nodes on startup | |
| 04:36:07 | eandersson | pretty sure what happens is that in Rocky with latest stable it will error if you restart a compute node with no instances | |
| 04:36:26 | eandersson | https://github.com/openstack/nova/commit/13bb7ed701121955ba015103c2e44429927e78d4#diff-77f9348ab09642ba46409b6828af4af0R1327 | |
| 04:44:25 | eandersson | actually it happens on non-empty computes as well | |
| 05:07:34 | eandersson | Interesting | |
| 05:07:46 | eandersson | An error'd compute due to quota still takes up an allocation | |
| 05:08:24 | eandersson | Preventing heal_allocation from progressing | |
| 05:15:47 | eandersson | Turns out placement isn't in a great place in that region. Probably why we are hitting errors. | |
| 05:39:50 | eandersson | I wish heal_allocations would have an option to skip failed computes. | |
| 05:40:25 | eandersson | When you are in a bad state you basically need to do some mysql magic to figure out "bad" hosts. Otherwise you are fixing them sequentially. | |
| 08:57:03 | kashyap | gibi_ptg: Good luck :-) | |
| 10:36:12 | openstackgerrit | Boris Bobrov proposed openstack/nova master: Also enable iommu for virtio controllers and video in libvirt https://review.opendev.org/684825 | |
| 10:36:13 | openstackgerrit | Boris Bobrov proposed openstack/nova master: Create a controller for qga when SEV is used https://review.opendev.org/693072 | |
| 12:23:44 | sean-k-mooney | ... so the random number generator has been broken in nova for a while... | |
| 12:24:09 | sean-k-mooney | and by a while i mean since https://github.com/openstack/nova/commit/be548a3396ee5f3a78b2b8b59321eb4c4530f7fb merged in queens | |
| 12:25:31 | sean-k-mooney | the comments is in the wrong place | |
| 12:25:33 | sean-k-mooney | https://github.com/openstack/nova/commit/be548a3396ee5f3a78b2b8b59321eb4c4530f7fb#diff-f4019782d93a196a0d026479e6aa61b1L4037-L4071 | |
| 12:25:57 | sean-k-mooney | after teh refactor the rng is only enabled if you also enable the qemu guest agent | |
| 12:26:39 | sean-k-mooney | before it was eneabled if you enabled it in the image and the host was qemu or kvm | |
| 12:27:52 | sean-k-mooney | kashyap: ^ we proably need to fix that but we talked before about enableing the rng by default. it should be two seperate patches but do you think we shoudl still enable it by default | |
| 12:38:03 | openstack | Launchpad bug 1851497 in OpenStack Compute (nova) "rng is only enabled if qemu guest agent is enabled" [Medium,Triaged] - Assigned to sean mooney (sean-k-mooney) | |
| 12:38:03 | sean-k-mooney | i have filed a bug for it here https://bugs.launchpad.net/nova/+bug/1851497 | |
| 12:54:46 | sean-k-mooney | efried: hi so http://paste.openstack.org/show/785846/ seams to now work. im going to kick it off on a clean vm to test but that stacked and i could boot vms | |
| 13:16:13 | efried | sean-k-mooney: cool beans | |
| 13:21:39 | efried | sean-k-mooney: I just need to update that enable_plugin line for my purposes, right? The rest ought to be unchanged from yesterday? | |
| 13:22:12 | sean-k-mooney | yep | |
| 13:22:41 | sean-k-mooney | ill proably merge the patch today or tomorow but i want to do a clean install first to make sure i have not missed something | |
| 13:23:18 | sean-k-mooney | i need to actully set up a gate job to actully test it rather then manually testing it every time i update it | |