| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-03 | |||
| 19:17:09 | mriedem | talking about https://developer.openstack.org/api-ref/compute/?expanded=create-image-createimage-action-detail#create-image-createimage-action right? | |
| 19:17:23 | mordred | I was testing against vexxhost, which is running newton I believe ... | |
| 19:17:26 | mordred | (and yes) | |
| 19:17:36 | mordred | one sec - getting the actual value from my logs so I can paste the right things | |
| 19:18:26 | mriedem | might have been related to some glance v1 stuff that we intentionally broke in newton | |
| 19:18:28 | mriedem | for glance v2 | |
| 19:18:39 | mriedem | i seem to remember some issues there with returning locations to glance api servers | |
| 19:19:30 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L1112 | |
| 19:20:08 | mriedem | https://github.com/openstack/nova/blob/master/nova/image/glance.py#L58 | |
| 19:20:13 | mordred | RESP: [202] Content-Type: text/html; charset=UTF-8 Content-Length: 0 Location: http://172.21.1.10:9292/images/f5d5b63b-e710-4d59-aa12-a9bd42f6652a Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-4b6fb5ed-3055-4d84-a16f-4057685ce737 Date: Sun, 02 Apr 2017 13:26:20 GMT | |
| 19:20:18 | mordred | that's the response I got | |
| 19:20:52 | mordred | but the Location _should_ have been https://image-ca-ymq-1.vexxhost.net/v2/images/f5d5b63b-e710-4d59-aa12-a9bd42f6652a | |
| 19:21:06 | mriedem | yeah i was going to say, it's wrong b/c it's missing v1 or v2 right? | |
| 19:21:14 | mordred | well - and also the host is wrong | |
| 19:21:20 | mriedem | the host comes from the config in nova | |
| 19:21:24 | mriedem | for glance api servers | |
| 19:21:26 | mordred | 172.21.1.10:9292 is unreachable by me as a user | |
| 19:21:41 | mordred | so it seems it's returning the thing that is the internal url | |
| 19:21:41 | sdague | so, the host is going to because they have private glance servers | |
| 19:21:44 | mordred | yah | |
| 19:21:52 | mriedem | yeah comes from config https://github.com/openstack/nova/blob/master/nova/image/glance.py#L107 | |
| 19:22:03 | mriedem | we'd have to get the host from the service catalog right? | |
| 19:22:05 | sdague | yeh, for historical reasons that's just a randomly picked one of their api_services | |
| 19:22:06 | mriedem | for publicUrl | |
| 19:22:19 | sdague | mriedem: yeh, it's more complicated than that, but yes | |
| 19:22:24 | mordred | kk. so this is still just part of the whole "use the catalog for glance stuff" effort | |
| 19:22:30 | sdague | because the glance stuff doesn't use the catalog anywhere | |
| 19:22:33 | sdague | mordred: yes | |
| 19:22:33 | mordred | yah | |
| 19:22:43 | mordred | cool - ok - so no new issue to file | |
| 19:22:44 | mriedem | is that an effort? | |
| 19:22:51 | mriedem | no one is working on this | |
| 19:23:34 | mordred | maybe effort is the wrong word "aspirational future state that people would love to be in but nobody is currently driving" | |
| 19:23:36 | mriedem | mordred: you should report a bug regardless | |
| 19:23:39 | mordred | kk | |
| 19:23:41 | mordred | will do | |
| 19:23:51 | mordred | also - I'm just stealing novaclient's workaround for now | |
| 19:24:02 | mriedem | i'm surprised we actually returned a location header | |
| 19:24:10 | mordred | which is to just pop the uuid off the end of the url and return _that_ | |
| 19:24:13 | sdague | mriedem: that's been there for a long time | |
| 19:24:30 | mriedem | sdague: there's lots of things that have been around for lots of years that surprise me :) | |
| 19:24:35 | sdague | :) | |
| 19:25:01 | sdague | returning location is actually a pretty good pattern, it's just that it depends on too much config being correct | |
| 19:26:12 | sdague | I'm trying to figure out if vexxhost's config is going to hurt something else in terms of migration | |
| 19:26:13 | mriedem | sdague: so is the basic idea, long-term we want to drop CONF.glance.api_servers and just have nova lookup the image service url in the catalog? | |
| 19:26:21 | sdague | mriedem: yes | |
| 19:26:29 | mriedem | same for neutron and cinder | |
| 19:26:32 | mriedem | since those are config driven | |
| 19:26:34 | mordred | yah | |
| 19:26:57 | mriedem | oh right and raj_singh had a patch to use ksa with glance in nova... | |
| 19:27:05 | sdague | right, so we probably are going to have to have a slightly more complicated case for glance where we consider an internal and a public path | |
| 19:27:14 | mriedem | https://review.openstack.org/#/c/412634/ | |
| 19:27:32 | sdague | because glance is one of those services where your would tend to build a really high bw internal path to the api service | |
| 19:27:43 | mordred | mriedem: I've also got this one: https://review.openstack.org/#/c/438512/ | |
| 19:27:48 | mriedem | we make that configurable for placement | |
| 19:27:53 | mriedem | which url to use i mean | |
| 19:27:58 | mordred | mriedem: which needs a little more cleanip before I bother you fine people with it much more | |
| 19:28:15 | sdague | mriedem: well... except we reflect urls straight to the user | |
| 19:28:22 | sdague | so we're going to need 2 config options | |
| 19:28:27 | mriedem | sdague: right i know | |
| 19:28:46 | mriedem | internal defaults to internalUrl or adminUrl or whatever, and public defaults to publicUrl | |
| 19:28:56 | mriedem | i was thinking about this https://github.com/openstack/nova/blob/master/nova/conf/placement.py#L31 | |
| 19:29:02 | sdague | mriedem: yeh | |
| 19:29:13 | mordred | yah - I'd argue to not have that | |
| 19:30:09 | mordred | internal and public are known interfaces. nova should reflect urls based on public to API consumers and should use internal to talk to other services - we shouldn't make operators tell nova to do that- if they don't want an internal url, they should just register the same url twice with keystone | |
| 19:30:10 | sdague | mordred: well, we tried to get away with be prescriptive with placement | |
| 19:30:12 | mordred | but I'm unreasonable | |
| 19:30:19 | sdague | and people got mad | |
| 19:30:23 | mordred | of course they did | |
| 19:31:02 | sdague | mordred: so, lots of deployments only use public | |
| 19:31:09 | sdague | or at least some do | |
| 19:31:17 | sdague | and just route correctly | |
| 19:31:24 | mordred | yah. that would be more smarter | |
| 19:31:40 | sdague | is there anyone else with a newton cloud? | |
| 19:32:20 | mordred | dunno - I only know vexxhost is because I was chatting with mnaser about a config-drive issue over the weekend | |
| 19:32:48 | mordred | but given the way novaclient works here, it's highly unlikely many real-world folks are broken by this issue, fwiw | |
| 19:34:06 | mordred | I mean - honestly rather than fixing it - just returning a json dict in the body with the image_id in it would be a win - those urls aren't really used much in the rest of the overall api flow | |
| 19:34:24 | mordred | and the first thing you're going to do with this as a user is make a call to glance to get the image dict by id | |
| 19:34:43 | mordred | (since you have to poll that for status to know when it's done) | |
| 19:35:00 | mriedem | mordred: yeah that's why i said i was surprised, | |
| 19:35:10 | mriedem | because we just return ids for things like this in response bodies | |
| 19:35:15 | mordred | so it likely _could_ be fixed with a microversion easier than actually fixing the internal/public glance url thing | |
| 19:35:18 | mriedem | but, changing the response will be a microversion | |
| 19:35:22 | mordred | mriedem: yah. I was very surprised myself | |
| 19:35:46 | mordred | mriedem: as you might can tell from my comment: https://review.openstack.org/#/c/452566/2/shade/openstackcloud.py | |
| 19:36:25 | mriedem | ha | |
| 19:36:32 | mriedem | so dramatic | |
| 19:36:54 | mordred | well - it was a sunday and I managed to break a server by making the api call against nova | |
| 19:37:03 | mordred | because it _ALSO_ apparently turns the server off to do the snapshot | |
| 19:37:15 | mordred | so I might have been in a bad mood at the time | |
| 19:37:29 | mriedem | live snapshot is a thing if libvirt is new enough and you've configured nova-compute to attempt it | |
| 19:37:40 | mriedem | we disabled live snapshot way back when b/c it resulted in 25% failure rate in the gate | |
| 19:37:59 | mriedem | you as the api user have no idea if it's live or cold though | |
| 19:38:04 | mriedem | since it's config drive on the compute | |
| 19:38:08 | mriedem | *driven | |
| 19:38:40 | mordred | \o/ | |
| 19:38:52 | mriedem | were you surprised that it was a cold snapshot? i see we don't mention that in the api ref | |
| 19:39:30 | mordred | well - I shouldn't have been - I know we don't use snapshots for any purpose in infra because it used to make the servers unresponsive for potentially hours - but I think I had just assumed that was some Xen/Rackspace thing | |
| 19:39:44 | mordred | but yes - I was totally unprepared for it to be a COMPLETELY unsafe thing to run | |
| 19:40:08 | mordred | that left me with a VM in a down state for an unknowable period of time | |