| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-12-11 | |||
| 17:29:20 | dtantsur | good plan, go for it | |
| 17:29:58 | openstackgerrit | Chris Dent proposed openstack/nova master: placement: skip authentication on root URI https://review.openstack.org/522002 | |
| 17:33:03 | stephenfin | mriedem: I'd to rework that patch. Turns out the glance API docs are wrong. Would appreciate a second look at it https://review.openstack.org/526329 | |
| 17:33:41 | mriedem | cdent: https://review.openstack.org/#/c/521640/ | |
| 17:34:11 | cdent | ack | |
| 17:38:46 | sean-k-mooney | stephenfin: you have a seperate bug open for the itorator to list conversion on line 179 of glance.py correct? | |
| 17:43:11 | openstackgerrit | Merged openstack/nova stable/pike: placement: avoid returning duplicated alloc_reqs when no sharing rp https://review.openstack.org/508885 | |
| 17:55:58 | cdent | mriedem: is your opinion we should go ahead with the want_version thing inside the serialization loop for the last-modified stuff. I read your first comment as maybe we should consider, and then as you progressed is became more certain. So I’ll take it as “doooo iiiiit” unless you feel like saying something to the contrary | |
| 18:28:44 | mriedem | cdent: i think we should probably do that for at least the requests that can generate a lot of results in a big cloud | |
| 18:28:53 | mriedem | so GET /traits and GET /resource_providers | |
| 18:29:09 | mriedem | GET /traits will always be at least reasonable size because of the library | |
| 18:29:26 | cdent | mriedem: i’ve done it for allocations, traits, resource classes, resource providers, but not inventories | |
| 18:29:32 | dtantsur | mriedem: re that patch: should I keep the functional tests changes, given that the behavior is no longer the default? | |
| 18:30:30 | mriedem | cdent: allocations are per provider or consumer right? so i didn't think that one was as big a deal | |
| 18:30:38 | mriedem | but if you're going to establish a pattern... | |
| 18:30:52 | mriedem | i just tend to think of allocations like inventories - they are mapped to a single provider | |
| 18:31:00 | cdent | mriedem: I was imagining something other than libvirt, where maybe the compute-mode is ginormous | |
| 18:31:05 | mriedem | dtantsur: i probably wouldn't | |
| 18:31:08 | cdent | or a shared disk | |
| 18:31:26 | dtantsur | k | |
| 18:31:36 | mriedem | cdent: or vmware :) | |
| 18:31:42 | mriedem | *vcenter | |
| 18:32:06 | cdent | i was mostly thinking of my.petapbytediskfarm.rp-5 | |
| 18:32:32 | mriedem | but then the inventory is per resource class per provider isn't it? | |
| 18:32:46 | mriedem | so you'd only have a lot of inventory entries for a provider if you have a lot of resource classes on that provider | |
| 18:34:28 | _ix | Hello friends. I'm having some trouble getting a compute node to show up in the hosts list. | |
| 18:34:40 | cdent | mriedem: right, which isn’t predicted, so that one is left off | |
| 18:35:07 | cdent | but when it turns into a problem, is fixable | |
| 18:35:31 | mriedem | stephenfin: question in https://review.openstack.org/#/c/526329/ | |
| 18:35:41 | mriedem | _ix: you have to run the discover_hosts CLI | |
| 18:35:47 | mriedem | nova-manage cell_v2 discover_hosts --verbose | |
| 18:35:52 | _ix | I have. It says there aren't any unmapped hosts. | |
| 18:36:05 | mriedem | what does nova service-list show? | |
| 18:36:11 | mriedem | does the entry show up there? | |
| 18:36:44 | _ix | For nova compute? Yes. | |
| 18:37:08 | _ix | This is just 1/5 of the hosts not showing up. | |
| 18:37:20 | _ix | But, I might be misleading a bit. I see there's an rpc problem on the host. | |
| 18:37:31 | mriedem | what does nova hypervisor-list show? | |
| 18:37:40 | mriedem | nova host-list is deprecated since pike btw | |
| 18:37:53 | smatzek | The Trove gates started failing on Friday or Saturday, right about the time the new attach flows landed. In particular they are failing when resizing or rebuilding instances with attached volumes. Does Nova have gate tests that do this? | |
| 18:38:11 | openstackgerrit | Dmitry Tantsur proposed openstack/nova master: Re-use existing ComputeNode on ironic rebalance https://review.openstack.org/508555 | |
| 18:38:14 | dtantsur | mriedem: I hope I did not mess up anything ^^^ | |
| 18:38:17 | mriedem | smatzek: yes for resize with volume attached - pretty sure anyway | |
| 18:38:24 | mriedem | smatzek: not sure about rebuild with a volume attached (in tempest anyway) | |
| 18:38:51 | _ix | mriedem: I generally use the openstack client. Is `openstack host list` also deprecated? | |
| 18:39:49 | mriedem | smatzek: test_resize_volume_backed_server_confirm | |
| 18:39:51 | mriedem | in tempest | |
| 18:40:07 | smatzek | mriedem: resize is failing like this, which in looking at the code means that connection_info is missing the driver type, and in looking at the previous log entries I can see that connection_info is indeed missing it: http://logs.openstack.org/39/524539/1/check/legacy-trove-functional-dsvm-mysql/3ef681d/logs/screen-n-cpu.txt.gz#_Dec_11_03_03_28_755007 | |
| 18:40:43 | mriedem | _ix: i deprecated nova host-list and didn't update openstack client so it's probably not deprecated | |
| 18:41:25 | _ix | Cool. I see that the host is disabled in the `nova hypervisor-list` Thanks for the help. Odd. | |
| 18:41:44 | mriedem | _ix: starting in pike, | |
| 18:41:46 | smatzek | rebuild, which I haven't dug as deep into today, is failing a bit more nuanced way. The mount point it is getting from either cinder or nova is giving /dev/na, as in "not applicable" which is a default for some fields in Cinder. | |
| 18:41:55 | mriedem | nova-compute will auto-disable the host if it's had 10 failed build attempts | |
| 18:41:58 | mriedem | in a row | |
| 18:42:57 | _ix | Oh, hey. That's a fine tip. Thanks again! I'm sure that's what's going on here. | |
| 18:42:57 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for allocations https://review.openstack.org/457534 | |
| 18:42:58 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for usages https://review.openstack.org/457535 | |
| 18:42:58 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for aggregates (v1.1) https://review.openstack.org/505643 | |
| 18:42:59 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for resource classes (v1.2) https://review.openstack.org/511182 | |
| 18:42:59 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: RP list: member_of and resources parameters (v1.3, v1.4) https://review.openstack.org/511183 | |
| 18:43:00 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: RP delete inventories (v1.5) https://review.openstack.org/514642 | |
| 18:43:00 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for traits (v1.6) https://review.openstack.org/514643 | |
| 18:43:01 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: Resource class set (v1.7) https://review.openstack.org/514644 | |
| 18:43:01 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: Usages per project and user (v1.8, v1.9) https://review.openstack.org/514646 | |
| 18:43:02 | openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI allocation candidates (v1.10) https://review.openstack.org/514647 | |
| 18:43:15 | _ix | Also, a side note. That's a super handy feature from my previous deployments. | |
| 18:43:20 | mriedem | _ix: there is a config option if you want to change that | |
| 18:43:29 | mriedem | but yeah, ^ is why it's on by default | |
| 18:43:30 | _ix | I don't. I think that's great. | |
| 18:43:35 | mriedem | dansmith: ^ high praise | |
| 18:44:11 | smatzek | mriedem: the tempest test test_resize_volume_backed_server_confirm is doing resize with a volume backed / BFV instance. The trove test is different, the instance has ephemeral disk and a volume attached. | |
| 18:44:42 | mriedem | smatzek: hmm http://logs.openstack.org/39/524539/1/check/legacy-trove-functional-dsvm-mysql/3ef681d/logs/screen-n-cpu.txt.gz#_Dec_11_03_03_28_499795 | |
| 18:45:02 | mriedem | {"attachment": {"status": "reserved", "detached_at": "", "connection_info": {}, "attached_at": "", "attach_mode": null, "instance": "85cdb482-63a5-487a-b103-95b9383ffcc7", "volume_id": "b3c82217-38f4-4c1f-b0f0-486a4fda6dda", "id": "848654a4-9647-46b8-9485-b3647fe87cfb"}} | |
| 18:45:03 | _ix | mriedem: Can you offer any conjecture about kernel versions, etc? We're on Centos 7.4, and upgraded to kernel 4.14. It looks like the rpcbind.socket is failing us. | |
| 18:45:13 | mriedem | _ix: nope | |
| 18:45:39 | mriedem | smatzek: that empty connection_info is presumably the problem | |
| 18:45:49 | mriedem | because nova pulls the driver_volume_type out of the connection_info on the attachment | |
| 18:46:00 | mriedem | smatzek: although, | |
| 18:46:11 | mriedem | i think that's normal for a 'reserved' attachment maybe? we'd have to ask jgriffith | |
| 18:46:26 | mriedem | until nova updates the attachment with the host connector, the attachment doesn't have a connection_info | |
| 18:46:39 | _ix | Will the openstack client one day be the only client? | |
| 18:46:48 | smatzek | yep, I came to the same conclusion, that the empty connection_info coming back from Cinder is part of the issue, but was drawing a blank on if it was suppose to be empty there or not. | |
| 18:47:06 | ildikov | mriedem: if it's only reserved connection_info is empty | |
| 18:47:18 | _ix | (That is, the openstack cli client) | |
| 18:47:26 | mriedem | _ix: at this rate i doubt it | |
| 18:47:36 | _ix | That's a sad story. | |
| 18:47:38 | mriedem | _ix: been talking about that for years now | |
| 18:47:43 | smatzek | To my knowledge Trove hasn't had changes here and it just started about when the Nova code landed so I thought I'd check. Also, it's not "mainline". Regular attach works and only resize and rebuild are having issues afaik. | |
| 18:48:04 | mriedem | _ix: would need to get feature parity in osc | |
| 18:48:08 | mriedem | _ix: and no one is working on that | |
| 18:48:15 | dansmith | mriedem: nice | |
| 18:48:18 | _ix | Forgive me, what is osc? | |
| 18:48:25 | mriedem | _ix: we could make that a priority, but that means de-prioritizing other work | |
| 18:48:28 | mriedem | openstack client | |
| 18:48:34 | mriedem | OpenStackClient | |
| 18:48:36 | _ix | Oh, I see. | |
| 18:49:48 | ildikov | smatzek: I need to jump on a call, but will look into the logs once I got off the phone | |
| 18:50:34 | ildikov | smatzek: the new flow shouldn't blow those calls up, but tempest might lack some coverage, I don't think we did a thorough manual testing :( | |
| 18:51:01 | mriedem | hmm | |
| 18:51:02 | mriedem | 'block_device_mapping': [{'guest_format': None, 'boot_index': -1, 'mount_device': u'/dev/vdb', 'connection_info': {'serial': u'b3c82217-38f4-4c1f-b0f0-486a4fda6dda'}, 'disk_bus': u'virtio', 'device_type': u'disk', 'attachment_id': '848654a4-9647-46b8-9485-b3647fe87cfb', 'delete_on_termination': True}] | |