| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-23 | |||
| 18:21:45 | rybridges | Ok so if the CLI defaults to the latest version that the server understands, then my server does not understand 2.41. If it did, then I would see the uuid in the response body of this call: http://paste.openstack.org/show/710386/ | |
| 18:23:02 | rybridges | either that or its not defaulting to the highest version available but rather the lowest base | |
| 18:23:30 | jgwentworth | you're running ocata, yes? you should have it https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id37 | |
| 18:23:31 | mriedem | {"version": {"status": "SUPPORTED", "updated": "2011-01-21T11:33:21Z", "links": [{"href": "http://XXX:8774/v2/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}], "min_version": "", "version": "", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2"}], "id": "v2.0"}} | |
| 18:23:39 | mriedem | that's your server's compute endpoint url version doc | |
| 18:23:46 | mriedem | note there is no min or version | |
| 18:23:50 | mriedem | version == max | |
| 18:23:54 | mriedem | so yeah, your cloud is f'ed | |
| 18:24:17 | jgwentworth | yeah ... you have ocata right rybridges? | |
| 18:24:24 | rybridges | yes i am running stable/ocata | |
| 18:24:36 | jgwentworth | hm, wtf | |
| 18:24:42 | superdan | this seems like #openstack fodder, no? | |
| 18:24:54 | mriedem | after 2 days of debugging this issue, yes | |
| 18:24:58 | mriedem | this is BASIC stuff | |
| 18:25:05 | superdan | yeah | |
| 18:25:05 | superdan | especially if this is just installation basically | |
| 18:25:27 | mriedem | where are sean-k-mooney[m] and cfriesen around when i need to flood this channel with placement and NFV talk? | |
| 18:26:15 | figleaf | leakypipes: ack. Will re-review in a few | |
| 18:30:11 | cdent | mriedem, superdan : if you like less code, the requirements dependency for removing microversion stuff in placement to a library has been improved: https://review.openstack.org/#/c/550265/ | |
| 18:30:42 | cdent | sigh: s/improved/approved/ | |
| 18:30:56 | mriedem | i like more code on fridays | |
| 18:31:03 | openstackgerrit | Merged openstack/nova master: trivial: omit condition evaluations https://review.openstack.org/545248 | |
| 18:31:13 | openstackgerrit | Merged openstack/nova master: Stop using mox in virt/xenapi/image/test_vdi_through_dev.py https://review.openstack.org/342682 | |
| 18:31:15 | cdent | mriedem: you never were any fun | |
| 18:36:50 | cdent | leakypipes you might also like https://review.openstack.org/#/c/550265/ <- placement using microversion-parse 0.2.1 | |
| 18:47:00 | openstackgerrit | Eric Fried proposed openstack/nova stable/queens: Change compute mgr placement check to region_name https://review.openstack.org/555901 | |
| 18:47:05 | fried_rolls | mriedem: Done ^ | |
| 18:47:20 | fried_rolls | leakypipes: ack | |
| 18:48:10 | mriedem | thanks | |
| 18:48:26 | jgwentworth | indeed looks like it's an installation problem. from the novaclient --debug output, it's talking to the /v2 nova endpoint which doesn't do microversions. it's supposed to be /v2.1 (in the service catalog) | |
| 18:48:49 | cdent | fried_rolls: wasn't half the bug there that people should have to express either region_name or os_region_name if they are using the default? not just that they were using the wrong one? | |
| 18:49:49 | fried_rolls | cdent: The bug was that, if you specified os_region_name, you got a deprecation warning, but it worked. But if you specified region_name, it just didn't work at all. | |
| 18:50:11 | fried_rolls | but yeah, pretty sure you have to specify one or the other. | |
| 18:50:15 | fried_rolls | which... is kind of a pity, really. | |
| 18:50:19 | cdent | that seems wrong | |
| 18:53:07 | figleaf | leakypipes: can you explain what's going on in https://review.openstack.org/#/c/554529/2/nova/api/openstack/placement/objects/resource_provider.py@3055 ? | |
| 18:56:32 | figleaf | leakypipes: oh, never mind. I think I got it | |
| 19:02:20 | sean-k-mooney[m] | mriedem: today on vacation | |
| 19:02:34 | JunoMan | Sorry for wasting your time with the microversion confusion y'all. We missed that in the kilo release notes. | |
| 19:03:01 | mriedem | sean-k-mooney[m]: was just joking, probably missed the context | |
| 19:03:02 | mriedem | ignore | |
| 19:03:09 | sean-k-mooney[m] | mriedem: also after yesterday i think i need to not think about numa/nfv for at least a day or two | |
| 19:05:00 | mriedem | superdan: the reader/writer fixes are ready to go imo https://review.openstack.org/#/q/topic:bug/1746509+(status:open+OR+status:merged) | |
| 19:06:23 | sean-k-mooney[m] | mriedem: hehe ya i know. i may have installed riot.im on my phone and now get irc routed too me now. it has its pros and cons | |
| 19:06:43 | jroll | JunoMan: excellent friday nick | |
| 19:09:50 | superdan | mriedem: okay will look in a few | |
| 19:11:19 | openstackgerrit | Julia Kreger proposed openstack/nova master: WIP: Add microversion to ironic client wrapper call https://review.openstack.org/554762 | |
| 19:11:50 | jgwentworth | will superdan strike down my code comment sentence? time will tell | |
| 19:12:22 | superdan | sources from beyond the grave say.....PLAUSIBLE | |
| 19:12:31 | jgwentworth | lol | |
| 19:18:13 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver https://review.openstack.org/526094 | |
| 19:28:45 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Always deallocate networking before reschedule if using Neutron https://review.openstack.org/555907 | |
| 19:45:33 | openstackgerrit | Merged openstack/nova master: Updated from global requirements https://review.openstack.org/555576 | |
| 20:02:38 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Add functional regression test for bug 1746509 https://review.openstack.org/555914 | |
| 20:02:39 | openstackgerrit | melanie witt proposed openstack/nova stable/queens: Move _make_instance_list call outside of DB transaction context https://review.openstack.org/555915 | |
| 20:02:39 | openstack | bug 1746509 in OpenStack Compute (nova) "TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction" [Medium,In progress] https://launchpad.net/bugs/1746509 - Assigned to melanie witt (melwitt) | |
| 20:06:22 | openstackgerrit | melanie witt proposed openstack/nova stable/pike: Add functional regression test for bug 1746509 https://review.openstack.org/555058 | |
| 20:06:23 | openstackgerrit | melanie witt proposed openstack/nova stable/pike: Move _make_instance_list call outside of DB transaction context https://review.openstack.org/555923 | |
| 20:06:23 | openstack | bug 1746509 in OpenStack Compute (nova) "TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction" [Medium,In progress] https://launchpad.net/bugs/1746509 - Assigned to melanie witt (melwitt) | |
| 20:06:49 | openstackgerrit | Konstantinos Samaras-Tsakiris proposed openstack/nova master: Add `hide_hypervisor_id` flavor extra_spec https://review.openstack.org/555861 | |
| 20:14:21 | mlavalle | rybridges: yeah, you are not getting the uuid attribute from the API. If you look at the response in http://paste.openstack.org/show/710244/, I get a header with the API version: X-OpenStack-Nova-API-Version: 2.60. Yours doesn't even have that header | |
| 20:15:28 | jgwentworth | mlavalle: yeah, I think they have a wrong (old) url for nova in their service catalog, /v2 (does not support microversions) where it should be /v2.1 | |
| 20:17:55 | jroll | yep | |
| 20:26:03 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Add disabled option to create_cell command https://review.openstack.org/555417 | |
| 20:26:23 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Modify nova-manage cell_v2 list_cells to display "disabled" column https://review.openstack.org/555415 | |
| 20:27:53 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Add --enable and --disable options to nova-manage update_cell https://review.openstack.org/555416 | |
| 20:30:16 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527 | |
| 20:32:41 | jgwentworth | fried_rice, superdan: I was looking at this review and it says that the powervm driver capabilities will be dynamic but I notice none of the other virt drivers have dynamic capabilities. would that not be problematic at all if the driver can change its capabilities on-the-fly? as far as the end user's experience/expectation? https://review.openstack.org/#/c/547169/3/nova/virt/powervm/driver.py@55 | |
| 20:34:22 | jgwentworth | because I thought usually we advertise those things through the hypervisor matrix doc | |
| 20:37:13 | superdan | jgwentworth: I don't think it would be a problem necessarily, | |
| 20:37:20 | superdan | if we're updating traits in placement | |
| 20:37:23 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527 | |
| 20:37:40 | superdan | you restart a compute, it has a new libvirt version, exposes some new feature, tells placement, and suddenly things can boot there needing that feature | |
| 20:37:56 | jgwentworth | okay, I see | |
| 20:38:28 | superdan | most of our capabilities have been rather static thus far, but that's just because they're like "have they implemented resize yet" sort of things I think | |
| 20:40:58 | jgwentworth | thanks | |
| 20:47:30 | openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: Resize https://review.openstack.org/553583 | |
| 21:10:49 | fried_rice | jgwentworth: Yuh, what superdan said. In this case, it's "dynamic" only per start of the compute service, based on which disk driver you picked. | |
| 21:11:52 | openstackgerrit | Julia Kreger proposed openstack/nova master: Add microversion to ironic client wrapper call https://review.openstack.org/554762 | |
| 21:12:31 | jgwentworth | fried_rice: ack | |
| 21:15:11 | openstackgerrit | Merged openstack/nova master: Add functional regression test for bug 1746509 https://review.openstack.org/555092 | |
| 21:15:12 | openstack | bug 1746509 in OpenStack Compute (nova) pike "TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction" [High,In progress] https://launchpad.net/bugs/1746509 - Assigned to melanie witt (melwitt) | |
| 21:20:12 | rybridges | mlavalle: jgwentworth: Ya it was the URL. The problem is there are no errors anywhere. The logs do not warn us about the fact that it cannot instantiate a client with a microversion b/c the url is not pointing to 2.1 (like it would on the CLI). Also there is no error or stack trace when we try to reference aggregate.uuid which does not exist. Everything just silently fails which makes it extremely | |
| 21:20:14 | rybridges | difficult to debug. We thought it was a bug at first because this was really strange behavior | |
| 21:23:06 | jgwentworth | rybridges: aggregate.uuid coming back from the novaclient python bindings? yeah, I suspect that no one has run into this + proposed novaclient patches to enhance the warnings/errors | |
| 21:27:40 | mriedem | presumably there are errors in the neutron logs | |
| 21:27:59 | mriedem | which is the client in this case | |
| 21:28:34 | mriedem | calling aggregate.uuid on the Aggregate object from novaclient would result in an AttributeError | |
| 21:28:39 | mriedem | and all of that should be barfed into the neutron logs | |
| 21:29:46 | openstackgerrit | Merged openstack/nova master: Move _make_instance_list call outside of DB transaction context https://review.openstack.org/555093 | |
| 21:29:59 | mlavalle | rybridges: agree with jgwentworth, it is a bug. My assumption when writing the code was that the following was going to take care of the API version: https://github.com/openstack/neutron/blob/stable/ocata/neutron/services/segments/plugin.py#L159 | |
| 21:32:26 | rybridges | mriedem: Attribute errors were not printed at all in our logs. No warnings either. Nothing. I was tailing nova-api, placement, neutron server and all of the agent logs on our hypervisors and at no point was there ever any error or anything. That is why I came here for help. If there had been anything in the logs I would have been able to solve it pretty easily i think. | |
| 21:32:36 | jgwentworth | holy crap, I think that's the first time my patches have ever merged without CI issues | |
| 21:33:46 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527 | |
| 21:42:45 | mriedem | rybridges: ok, i don't know how that's possible, because http://paste.openstack.org/show/710652/ | |
| 21:43:19 | mriedem | anywho | |
| 21:43:55 | openstackgerrit | melanie witt proposed openstack/nova master: Migrate tempest-dsvm-multinode-live-migration job in-tree https://review.openstack.org/555945 | |
| 21:47:14 | openstackgerrit | Merged openstack/nova master: trivial: Resolve 'oslo.context' deprecation warnings https://review.openstack.org/532822 | |
| 21:55:24 | openstackgerrit | melanie witt proposed openstack/nova master: Migrate tempest-dsvm-multinode-live-migration job in-tree https://review.openstack.org/555945 | |
| 22:04:19 | jgwentworth | mriedem: queens backport is ready2rock https://review.openstack.org/#/c/555914 | |