Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2019-06-11
11:27:40 gtema Is it possible to see a real query going there?
11:27:47 mordred I'm working on running some ... yeah - one sec
11:28:06 mordred http://paste.openstack.org/raw/752708/
11:28:30 mordred I'm working on doing the same upload again with 0.27 to get a comparable chunk of logging
11:28:38 gtema hmm, looks good. Exactly
11:28:42 mordred because I don't see anything wrong there - yeah
11:32:15 mordred the awesome part is that because rax uses the image-task method, I have to upload a full actually valid image :)
11:32:27 gtema :D
11:38:34 gtema what I see could be different - presence of "Accept: "
11:43:58 mordred yes. although I did try a run hacking it to not send the accept header - and that sent the same 400
11:44:12 gtema really? hm
11:44:25 gtema that's the only thing I might think of
11:45:58 mordred yeah. same here
11:51:41 gtema btw, to which version you downgraded?
11:54:24 mordred gtema: 0.27.0
11:54:38 mordred gtema: which is what nodepool had been running before this recent restart that picked up latest sdk
11:54:44 gtema ok
11:54:46 mordred ok - so - I have confirmed the upload works under 0.27.0
11:55:10 gtema and the request looks like ...?
11:55:32 mordred the main difference I can see in the REST calls is that in 0.27.0 that request is not made after the task import is complete - I THINK
11:55:47 mordred (I really should have logged this to a file - scrollback is a bit hard to scane because of the list calls)
11:56:29 mordred you know what? I'm going to do it again and log to a file - then I can upload the whole file somewhere
11:56:38 gtema cool
11:57:30 mordred have I mentioned how happy I am that image uploading is so complicated? :)
11:57:47 gtema nope, but I totally agree with you
11:58:09 gtema server normalization (with respect to network addresses) is not much better
11:58:14 mordred yeah
11:59:01 gtema but btw have found why nodepool was not happy with flavors - dogpile.cache is not happy with Resource, since it is returned from a function from a generator from a ....
11:59:13 mordred I mean - ultimately a large percentage of the crazy logic in the shade layer is related to rackspace being weird (the one example in the other direction is how hard it is to boot a server with an ip address)
11:59:16 mordred AH - joy
11:59:29 mordred well - I think we should rework how the caching works anyway ...
11:59:42 gtema it's basically pickle.dumps(list(self.compute.flavors())) does not work
11:59:54 gtema yeah, would need to
12:00:55 mordred you know what - maybe I should write a quick spec with the thoughts that were in my head WRT caching and we can talk it through and come up with an actual plan
12:01:29 gtema btw, racks support creating server with block_device_mapping2. Currently we use os-volumes_boot if bdm_v2 is present in params
12:01:39 gtema for caching - cool
12:01:42 gtema let's do this
12:01:50 mordred \o/
12:02:24 mordred so - I'm also thinking - the support you added for having a cloud/vendor-specific driver that overrides some of the proxies ...
12:02:39 gtema :D
12:02:46 mordred maybe it would be cleaner to extract the rackspace-specific logic we have into a rackspace driver
12:03:46 gtema the ones "originated" all OpenStack are now a separate incompatible driver ???
12:04:00 mordred yeah. it's "funny"
12:04:14 mordred but we carry a lot of logic that's just for them
12:04:24 gtema but yes, we can start really packing all of that into "drivers"
12:04:32 mordred like - the task import stuff - it's technically in upstream glance, but it's untestable in devstack
12:06:18 gtema I'm back in 10-20 min
12:18:11 gtema am back
12:19:20 mordred just added the call from 0.27.0
12:20:11 gtema so it's really just "Accept:"
12:21:29 gtema well and then the order of "path" and "value" for each item
12:22:59 mordred I'm thinking maybe it's something to do with call sequencing
12:23:09 mordred I'm going to re-run latest logging to a file
12:23:17 gtema if you quickly remove line 284 in image/v2/image.py (setting Accept) to be also sure
12:23:30 mordred yeah - I did that yesterday
12:23:34 gtema ok
12:23:37 mordred but I can do it again and do the log if we want
12:24:27 gtema would be nice, but if you say it didn't help - not prio
12:25:09 gtema let's then really compare order or calls. However unittest tests this also. And here we had no change
12:25:36 mordred yeah. I was pretty confident in the changes because we have this covered in unittests
12:29:16 gtema latest trace is broken - incomplete
12:29:45 mordred oh yeah? poo - maybe there was a length limit here
12:29:47 mordred hit
12:30:02 mordred http://paste.openstack.org/show/752725/ <-- there's a trimmed version with just the request sequence
12:31:49 gtema the order is same as in tests
12:32:04 mordred yeah
12:32:44 mordred running 0.31.0 now - we'll see how it goes
12:32:49 mordred takes about 20 minutes
12:32:53 gtema ok
12:48:56 mordred http://paste.openstack.org/show/752728/
12:49:05 mordred there's 0.31.0 sequence - looks the same :(
12:49:51 gtema yeah. Is it now with "Accept: " or without?
12:53:14 mordred with
12:54:08 gtema what I also see in the first trace before GET/PATCH image was first searched in the LIST. Could it "theoretically" be, that the ID changes?
12:59:33 mordred well - but in the failing one we do a GET on the image by id
13:01:09 gtema yes , exactly
13:01:49 mordred {"op": "add", "value": "False", "path": "/xenapi_use_agent"} 0.27.0
13:02:02 mordred {"op": "add", "path": "/xenapi_use_agent", "value": false} 0.31.0
13:02:14 mordred we're passing false as a string in 0.27 and as a bool in 0.31
13:02:23 gtema yeah, I see
13:05:54 mordred in the original _make_v2_image_params - we made sure to str() wrap each item if it wasn't in a specific list
13:06:35 mordred we have the same thing in the image proxy
13:06:42 gtema hmm
13:07:48 mordred we call it in master in _update_image_properties - but then we call self.update_image with the resulting dict - maybe that's flowing it back through the resource mapping and causing the False to get boolean'd
13:08:52 mordred that doesn't make any sense though - xenapi_use_agent isn't a known property - resource shouldn't be transforming its data type
13:09:46 gtema you know everything what's running in the background of all the methods ;-)?
13:12:02 gtema should we then force all the props before the line 277 (v2/_proxy) to update_image to be strings?
13:12:37 gtema or probably better in v2/image/_prepare_request (if patch ...)
13:13:31 gtema who is setting xenapi_ise_agent?
13:13:36 gtema nodepool?
13:15:37 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Filter properties through make_v2_image_params https://review.opendev.org/664582
13:15:58 mordred gtema: I *think* that ^^ should do the trick (looking at shade vs. current code, we're missing that call in that location)
13:16:17 mordred gtema: but - the things you were saying are likely a *better* approach
13:16:34 mordred gtema: xenapi_use_agent is actually set in the vendor profile of rackspace
13:16:35 gtema yes, and I remember explicitely skipping it and relying to the general "patch" in the resource
13:17:08 mordred oh. you know - it's set as boolean in the vendor file
13:17:18 mordred maybe let's try just updating that to "False"
13:17:27 gtema that's what I also mean. I think it will help either
13:17:53 mordred I'm going to try patching that (the vendor profile) and trying another upload
13:18:23 gtema and probably it is a better place to fix. Normal patch is just comparing two things and generates a diff out of it.

Earlier   Later