| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-01 | |||
| 18:27:36 | mriedem | yee be warned | |
| 18:31:38 | larsks | So noted. | |
| 18:32:43 | openstackgerrit | Jay Pipes proposed openstack/nova master: Add Project object https://review.openstack.org/565401 | |
| 18:32:44 | openstackgerrit | Jay Pipes proposed openstack/nova master: Add User object to placement modeling https://review.openstack.org/565402 | |
| 18:32:45 | openstackgerrit | Jay Pipes proposed openstack/nova master: add Consumer object https://review.openstack.org/565403 | |
| 18:32:46 | openstackgerrit | Jay Pipes proposed openstack/nova master: Add create() methods to Project, User and Consumer https://review.openstack.org/565404 | |
| 18:32:47 | openstackgerrit | Jay Pipes proposed openstack/nova master: move consumer ensure to API layer https://review.openstack.org/565405 | |
| 18:32:48 | openstackgerrit | Jay Pipes proposed openstack/nova master: remove Allocation.project_id & Allocation.user_id https://review.openstack.org/565406 | |
| 18:32:49 | openstackgerrit | Jay Pipes proposed openstack/nova master: rework allocation handler _allocations_dict() https://review.openstack.org/565407 | |
| 18:32:50 | openstackgerrit | Jay Pipes proposed openstack/nova master: increment consumer generation when allocs change https://review.openstack.org/565513 | |
| 18:36:02 | melwitt | looks like libvirt-qemu is the right user and group, actually logged in the devstack log http://logs.openstack.org/49/479949/71/check/nova-next/7751dc4/logs/devstacklog.txt.gz#_2018-05-01_15_04_42_683 | |
| 18:36:26 | mriedem | huh | |
| 18:36:29 | mriedem | nuts | |
| 18:36:54 | melwitt | yeah. back to the drawing board | |
| 18:42:01 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate the nova-consoleauth service https://review.openstack.org/565367 | |
| 18:43:37 | mriedem | alright https://review.openstack.org/#/q/topic:bp/convert-consoles-to-objects+status:open is ready to go | |
| 18:43:44 | mriedem | and it's the last day in the runway | |
| 18:45:45 | melwitt | so close ... | |
| 18:46:04 | mriedem | well, efried and dansmith can make that magic happen | |
| 18:46:24 | mriedem | *or | |
| 18:46:27 | melwitt | note that once the "convert websocketproxy" change is approved, we need to ask frickler to lift the -2 on the first devstack sandwich change so it can go through | |
| 18:46:33 | mriedem | yup | |
| 18:46:36 | efried | mriedem: subtle. | |
| 18:54:33 | melwitt | dansmith: do you remember why we have to "enable before stop" for the conductor fleet? I had the same thing for the console proxies https://review.openstack.org/#/c/484973/12/lib/nova@1078 | |
| 18:56:50 | dansmith | if the thing isn't enabled, then stop_service won't do anything, | |
| 18:57:08 | dansmith | but I thnk that conductor case was just for when we were moving to super conductor | |
| 18:57:19 | dansmith | although maybe it's still required since we generate the service id for services in the cell? | |
| 19:00:28 | melwitt | I wasn't sure why we can't assume they're enabled from the earlier call to start_nova_console_proxies | |
| 19:01:11 | dansmith | idk, I'm kinda into something else at the moment so I can't look deeply, | |
| 19:01:21 | dansmith | but what I said is what I remember off the top of my head for that stuff | |
| 19:02:45 | melwitt | yeah, np, thanks for that. not looking for a deep answer, wish I had written a comment on here from back when someone explained it | |
| 19:05:18 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: Granular GET /allocation_candidates https://review.openstack.org/517757 | |
| 19:05:27 | melwitt | I think it might be because start_* and stop_* aren't going to happen in the same run (stack.sh vs unstack.sh) and there's not going to be any saving of state about the cell services between those runs | |
| 19:06:20 | melwitt | so during the unstack, you can't stop the per cell processes unless you enable them first | |
| 19:07:41 | dansmith | that's what I'm saying, | |
| 19:07:49 | dansmith | because we generate their service names from the cell we're in | |
| 19:07:52 | dansmith | n-cond-cell1, etc | |
| 19:07:55 | melwitt | right | |
| 19:08:37 | melwitt | what I was missing is that there's no saving of state after a stack.sh so there's no way the teardown would "save" the fact that the services were created and enabled by the stack.sh run | |
| 19:21:02 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: update add-consumer-generation to focus on API https://review.openstack.org/565565 | |
| 20:00:49 | openstackgerrit | Oliver Walsh proposed openstack/nova master: libvirt: fix setting tx_queue_size when rx_queue_size is not set https://review.openstack.org/565573 | |
| 20:07:39 | efried | owalsh: Do we need a bug for ^ ? | |
| 20:13:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup placement policy generator docs https://review.openstack.org/565225 | |
| 20:13:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add granular policy rules for /resource_classes* https://review.openstack.org/565578 | |
| 20:15:07 | mriedem | efried: i thought about that, but it's not even tagged in rc1 | |
| 20:15:21 | mriedem | so not really a good chance of someone hitting that besides red hat QE | |
| 20:15:54 | efried | mriedem: Sure, it's only been broken for a week, but still, paperwork. | |
| 20:16:03 | efried | mriedem: If you of all people don't need a bug, I'm sure not gonna push for it :) | |
| 20:16:38 | owalsh | awww but I love paperwork :-( | |
| 20:18:03 | owalsh | efried, mriedem: yea, I'm lazy and it's not released yet but can raise one if you think it's necessary | |
| 20:19:20 | openstackgerrit | Merged openstack/nova master: Add user_id to RequestSpec https://review.openstack.org/565340 | |
| 20:20:56 | openstackgerrit | Brianna Poulos proposed openstack/nova master: Plumb trusted_certs through libvirt driver image paths https://review.openstack.org/561262 | |
| 20:20:57 | openstackgerrit | Brianna Poulos proposed openstack/nova master: Add trusted_image_certificates to REST API https://review.openstack.org/486204 | |
| 20:20:58 | openstackgerrit | Brianna Poulos proposed openstack/nova master: Add notification support for trusted_certs https://review.openstack.org/563269 | |
| 20:34:01 | melwitt | hm, looks like all of the third party CIs suddenly are failing | |
| 20:35:00 | mriedem | all or just powervm? | |
| 20:35:04 | melwitt | hyperv CI can't resolve a hostname, some others like powervm and virtuozzo, the log files aren't found | |
| 20:35:14 | mriedem | i blame zuul | |
| 20:35:16 | melwitt | and it seems like this has happened starting today | |
| 20:43:52 | randomhack_ | was this bug ever solved? https://bugs.launchpad.net/nova/+bug/1633033. ran into this today on a Newton in | |
| 20:43:52 | openstack | Launchpad bug 1633033 in OpenStack Compute (nova) "live migration with encrypted volume fails" [Undecided,In progress] - Assigned to Lee Yarwood (lyarwood) | |
| 20:44:54 | randomhack_ | love migration of an instance with encrypted volume resulted in volume being mounted without decryption and corrupted the file system | |
| 20:45:42 | randomhack_ | s/love/live | |
| 20:46:49 | randomhack_ | known limitation of encrypted iscsi volumes? | |
| 20:48:15 | mriedem | randomhack_: lyarwood would know but it's really late for him (UK) | |
| 20:49:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Implement granular policy rules for placement https://review.openstack.org/524425 | |
| 20:49:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add granular policy rules for /resource_classes* https://review.openstack.org/565578 | |
| 20:49:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup placement policy generator docs https://review.openstack.org/565225 | |
| 20:49:24 | randomhack_ | mriedem: thanks i’ll check tomorrow earlier | |
| 20:49:32 | melwitt | yeah. IIRC, it will work as of pike when using qemu native luks decryption https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/libvirt-qemu-native-luks.html | |
| 20:49:59 | randomhack_ | melwitt: thanks! | |
| 20:50:04 | melwitt | because I thought lyarwood had a depends-on patch that ran live migration on top of the series | |
| 20:50:46 | melwitt | randomhack_: double check with lyarwood to make sure | |
| 20:52:25 | melwitt | this was one of the live migration test patches https://review.openstack.org/545074 | |
| 20:53:06 | mriedem | melwitt: i think that was queens | |
| 20:53:28 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/libvirt-qemu-native-luks.html :) | |
| 20:53:28 | mriedem | yeah | |
| 20:53:30 | melwitt | ah, you're right, it was queens | |
| 20:59:04 | melwitt | ceph job failing 100% with SnapshotIsBusy during a volume delete now. le sigh http://logs.openstack.org/49/479949/71/check/legacy-tempest-dsvm-full-devstack-plugin-ceph/94fbc69/logs/screen-c-vol.txt.gz#_May_01_15_45_34_845542 | |
| 21:00:30 | melwitt | there's even a note from you in the tempest test about this (I think?) and yet it's still happening https://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L239 | |
| 21:00:59 | melwitt | oh, wait, it's a TODO | |
| 21:01:33 | melwitt | I'll try to do the todo | |
| 21:01:38 | mriedem | it's a glorious todo | |
| 21:02:06 | melwitt | apparently. I don't know how this used to work before if there's an inherent issue with the volume dependencies in the test | |
| 21:02:32 | melwitt | I don't know volumes about volumes | |
| 21:03:11 | melwitt | thanks | |
| 21:03:41 | mriedem | so in this test, we boot from volume and then create a volume-backed server snapshot, so we have an image with bdmv2 metadata pointing at the volume snapshot, | |
| 21:03:56 | mriedem | we then create a 2nd server from that image-defined bdm (the volume snapshot) | |
| 21:04:11 | mriedem | because delete_on_termination=True, | |
| 21:04:29 | mriedem | when we go to delete the 2nd server, the compute service will attempt to delete the 2nd volume which is the root disk for the 2nd server, created from the volume snapshot, | |
| 21:04:35 | mriedem | but cinder won't let you delete a volume that has snapshots, | |
| 21:04:47 | mriedem | so nova-compute will get an error from cinder and ignore it | |
| 21:05:14 | melwitt | hmm okay | |
| 21:05:35 | mriedem | in here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2422 | |
| 21:05:53 | mriedem | you should see "Failed to delete volume" in the n-cpu logs for that test | |
| 21:06:12 | melwitt | it's weird because what's happening is it's the test trying to cleanup the volumes it had created, | |
| 21:06:31 | melwitt | and it can't delete the snapshot because of dependent things | |
| 21:06:31 | mriedem | http://logs.openstack.org/49/479949/71/check/legacy-tempest-dsvm-full-devstack-plugin-ceph/94fbc69/logs/screen-n-cpu.txt.gz#_May_01_15_45_20_004599 | |
| 21:06:45 | melwitt | it's not failing because of anything about n-cpu as far as I can tell | |
| 21:07:31 | mriedem | i believe jbernard had a really old patch for dealing with a failure to delete snapshots because they were busy with the cinder rbd driver | |