| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-10 | |||
| 17:09:03 | openstackgerrit | Dan Smith proposed openstack/nova master: Refactor resource tracker to account for migration allocations https://review.openstack.org/506419 | |
| 17:09:03 | openstackgerrit | Dan Smith proposed openstack/nova master: Revert allocations by migration uuid https://review.openstack.org/498949 | |
| 17:09:04 | openstackgerrit | Dan Smith proposed openstack/nova master: Make live migration hold resources with a migration allocation https://review.openstack.org/507638 | |
| 17:09:04 | openstackgerrit | Dan Smith proposed openstack/nova master: Make migration uuid hold allocations for migrating instances https://review.openstack.org/506420 | |
| 17:09:17 | openstackgerrit | Jay Pipes proposed openstack/nova master: rp: streamline InventoryList.get_all_by_rp_uuid() https://review.openstack.org/509030 | |
| 17:09:18 | openstackgerrit | Jay Pipes proposed openstack/nova master: rp: remove ability to delete 1 allocation record https://review.openstack.org/509032 | |
| 17:09:18 | openstackgerrit | Jay Pipes proposed openstack/nova master: rp: remove dead code in Allocation._create_in_db() https://review.openstack.org/509031 | |
| 17:09:19 | openstackgerrit | Jay Pipes proposed openstack/nova master: rp: rework AllocList.get_all_by_consumer_id() https://review.openstack.org/509035 | |
| 17:09:19 | openstackgerrit | Jay Pipes proposed openstack/nova master: rp: fix up AllocList.get_by_resource_provider_uuid https://review.openstack.org/509033 | |
| 17:09:20 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: integrate ProviderTree to report client https://review.openstack.org/415921 | |
| 17:09:20 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: set/check if inventory change in tree https://review.openstack.org/470575 | |
| 17:09:21 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215 | |
| 17:09:21 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: add nested resource providers https://review.openstack.org/377138 | |
| 17:09:22 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693 | |
| 17:09:22 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807 | |
| 17:09:58 | jaypipes | dansmith, mriedem, efried: k, n-r-p series updated... ^ | |
| 17:10:11 | efried | woot | |
| 17:13:51 | gibi | mriedem: I just noticed that indeed there is one notification transformation patch that already has a +2 https://review.openstack.org/#/c/453077/ | |
| 17:14:15 | gibi | mriedem: you were even +2 before on that | |
| 17:14:29 | dansmith | mriedem: I'm getting pretty fatigued on this one, so please apply extra critical thinking when reviewing: https://review.openstack.org/#/c/507638/ | |
| 17:14:36 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] manage cache headers for inventories https://review.openstack.org/510954 | |
| 17:14:36 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] manage cache headers for /resource_providers https://review.openstack.org/495380 | |
| 17:14:49 | dansmith | I pushed it up to see if it really passes live migration tests and then I might try to think on some more cleanup of it | |
| 17:15:13 | mriedem | easy regression fix needed for pike https://review.openstack.org/#/c/510603/ | |
| 17:15:47 | mriedem | gibi: ok | |
| 17:15:48 | mriedem | dansmith: ok | |
| 17:15:51 | mriedem | everyone: ok | |
| 17:21:11 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Fix sending legacy instance.update notification https://review.openstack.org/510957 | |
| 17:26:47 | mriedem | weeee we stacktrace during snapshot a whole bunch https://bugs.launchpad.net/nova/+bug/1722571 | |
| 17:26:48 | openstack | Launchpad bug 1722571 in OpenStack Compute (nova) "NotImplementedError(_('direct_snapshot() is not implemented')) stacktraces in n-cpu logs" [Medium,Confirmed] - Assigned to hongbin (hongbin034) | |
| 17:28:06 | hongbin | mriedem: i am looking into it (try to fix some bugs to pick up) | |
| 18:02:48 | mriedem | hongbin: thanks, should be pretty easy | |
| 18:06:17 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for inventories https://review.openstack.org/457533 | |
| 18:06:18 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for usages https://review.openstack.org/457535 | |
| 18:06:18 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for allocations https://review.openstack.org/457534 | |
| 18:06:19 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for aggregates https://review.openstack.org/505643 | |
| 18:18:33 | mriedem | sdague: i just realized that when showing server details, we provide a link to view image details, and the link for the image details is to the compute api images proxy api, which is deprecated starting in 2.36 :) | |
| 18:19:30 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L254 | |
| 18:20:04 | mriedem | seems like if the microversion is >= 3.26 we should generate the alternate link to the images api | |
| 18:20:20 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/images.py#L133 | |
| 18:22:05 | mriedem | ooo, and with efried's new service type stuff, if you don't set CONF.glance.api_servers, we'd actually generate maybe a valid images api url for you | |
| 18:31:20 | sdague | mriedem: honestly, I'm not sure we should be generating an image link | |
| 18:31:28 | sdague | I thought we had gotten rid of that | |
| 18:31:48 | mriedem | are you thinking of the location header in the response when creating a snapshot or backup image? | |
| 18:32:10 | mriedem | https://docs.openstack.org/nova/pike/reference/api-microversion-history.html#id40 | |
| 18:32:46 | mriedem | yeah we probably should have used that microversion to also stop returning image api url links | |
| 18:33:07 | sdague | I actually misremembered that we did | |
| 18:33:16 | sdague | maybe we stopped accepting image url somewhere | |
| 18:33:29 | mriedem | i remember gmann fixing something but can't remember what | |
| 18:36:15 | mriedem | unrelated, for some weird reason, i'm tracing a reboot request through the API, and i see the request ID in the n-api logs but it's not showing up in the n-cpu logs | |
| 18:39:44 | cfriesen | does anyone know why in nova.virt.libvirt.host.Host_init_events_pipe() we have a queue as well as a pipe? Why can't we just block on the queue rather than the pipe? | |
| 18:40:31 | mriedem | talk to mdbooth | |
| 18:41:40 | cfriesen | looks like it was dpb that added it originally | |
| 18:59:21 | openstackgerrit | Elod Illes proposed openstack/nova master: WIP: Transform scheduler.select_destinations notification https://review.openstack.org/508506 | |
| 19:14:57 | openstackgerrit | Elod Illes proposed openstack/nova master: WIP: Transform scheduler.select_destinations notification https://review.openstack.org/508506 | |
| 19:30:38 | mtreinish | mriedem: changing error codes on us | |
| 19:31:18 | mriedem | making bad requests | |
| 19:31:20 | mriedem | tsk tsk | |
| 19:32:00 | mriedem | jgriffith: ildikov: debugging a failed test on the new style attach patch, i can't figure out why but we're doing an attachment_update during a reboot | |
| 19:32:21 | mriedem | http://logs.openstack.org/85/330285/144/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/89229f6/logs/subnode-2/screen-n-cpu.txt.gz#_Oct_10_13_02_05_685303 | |
| 19:32:39 | mriedem | i can't tell what's eventually calling that | |
| 19:33:56 | ildikov | mriedem: does that share code path with any other operation? | |
| 19:34:39 | ildikov | I don't recall a reboot scenario right now | |
| 19:34:54 | ildikov | maybe jgriffith_ does | |
| 19:35:26 | ildikov | mriedem: on that attachment_complete, I added the note on the higher level to clarify why that call is there as I kept re-investigating it | |
| 19:35:50 | mriedem | ildikov: yeah i saw the note, but the logic is basically: | |
| 19:36:01 | mriedem | 1. if do_driver_attach - call attachment_complete down in the bowels, else | |
| 19:36:03 | mriedem | 2. do it here | |
| 19:36:15 | mriedem | as noted, we should just always do it in the bowels as we did before | |
| 19:36:20 | ildikov | mriedem: I'm fine with just covering it in bdm.attach if we don't want to deal with the different flows for this | |
| 19:36:44 | mriedem | i definitely don't want to have to know in the compute manager about what's going on in the DriverVolumeBlockDevice.attach based on flags passe din | |
| 19:37:01 | mriedem | i.e. the logic above | |
| 19:37:21 | ildikov | mriedem: I agree, I will change that | |
| 19:38:07 | ildikov | however I don't think that has much to do with the current failures, except if something got introduced that would need that call | |
| 19:38:11 | mriedem | as for what's doing the attachment_update during reboot, it's not clear to me yet | |
| 19:38:32 | mriedem | refreshing the volume connection info would do it, but i don't see that called during reboot | |
| 19:39:13 | mriedem | this is a soft reboot too, so we're not rebuilding the guest at all | |
| 19:39:50 | ildikov | update is called only at three places | |
| 19:40:32 | mriedem | swap volume, volume attach, and refresh_connection_info | |
| 19:40:42 | ildikov | _init_volume_connection, _volume_attach, and _get_connection_updates which is called in refresh_connection_info | |
| 19:41:06 | mriedem | yeah i'm failing to see where reboot calls any of those, directly or otherwise | |
| 19:41:17 | mriedem | but the request id for the reboot operation is showing it happening | |
| 19:42:07 | efried | mriedem sdague FYI glance will only let you skip *all* conf after https://review.openstack.org/#/c/490057/ is in place. Which is to be proven by https://review.openstack.org/#/c/490031/ passing jenkins. | |
| 19:42:16 | ildikov | I couldn't get there to check that scenario test, I didn't think there's a reboot in it... :/ | |
| 19:42:44 | mriedem | ildikov: yeah TestMinimumBasicScenario.test_minimum_basic_scenario attaches a volume and then soft reboots the guest | |
| 19:42:52 | mriedem | then ssh's into the guest to make sure the block device is still attached | |
| 19:43:20 | mriedem | at the end when the test is cleaning up it goes to detach the volume which fails because the volume status is 'attaching' rather than 'in-use' | |
| 19:43:30 | mriedem | and i think that's happening because of a wild attachment_update call during the reboot | |
| 19:43:36 | mriedem | which screws up the volume status | |
| 19:44:48 | ildikov | jgriffith_ said that it's calling the old flow | |
| 19:44:54 | ildikov | which seemed odd | |
| 19:45:32 | ildikov | and yeah, if we don;t have attachment_complete following attachment_update then the volume status remains "attaching" | |
| 19:46:06 | mriedem | it can't be the old flow because otherwise we wouldn't call attachment_update | |
| 19:46:22 | mriedem | which i see happen right here during reboot http://logs.openstack.org/85/330285/144/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/89229f6/logs/subnode-2/screen-n-cpu.txt.gz#_Oct_10_13_02_05_685303 | |
| 19:46:35 | mriedem | PUT https://10.210.129.28/volume/v3/2732673e8c4a46249b654cee4b963920/attachments/0797bbdd-ecd6-488b-bc85-b23b5de49643 | |
| 19:46:42 | mriedem | -d '{"attachment": {"connector": {"platform": "x86_64", "host": "ubuntu-xenial-2-node-rax-ord-11308327-949541", "do_local_attach": false, "ip": "10.210.195.60", "os_type": "linux2", "multipath": false, "initiator": "iqn.1993-08.org.debian:01:cc478c1ad9"}}}' | |
| 19:46:48 | mriedem | it's updating the attachment with the host connector | |
| 19:47:24 | sdague | efried: nice | |
| 19:48:43 | ildikov | yeah, I saw that too on the Cinder side | |
| 19:48:51 | ildikov | I mean the new flow call | |