| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-sdks - 2019-08-06 | |||
| 18:15:44 | Shrews | mordred: would that come from a clouds.yaml setting? | |
| 18:16:08 | Shrews | rackspace profile maybe? | |
| 18:17:30 | Shrews | hrm, all i see is "image_api_use_tasks": true | |
| 18:29:05 | mordred | yeah - it's not a setting we set in clouds.yaml | |
| 18:29:26 | mordred | Shrews: so - a few things ... | |
| 18:30:07 | mordred | Shrews: I think it's entirely possible that we will leak swift objects any time sometihng _else_ goes wrong in an image upload that doesn't trigger that finally | |
| 18:30:27 | mordred | also - maybe we leaked a bunch of things a while back - I think there was a time when the default of use_slo was the opposite | |
| 18:31:08 | mordred | Shrews: maybe on use_tasks clouds we should have delete_image go and look for the object that would have been used and try to delete it? | |
| 18:31:41 | Shrews | maybe | |
| 18:34:19 | Shrews | mordred: is the slo manifest an object within itself? where would i find it? | |
| 18:35:39 | Shrews | wondering if i can view it from the openstack client. all i see are the segments in 'images' | |
| 18:40:27 | mordred | Shrews: it is an object | |
| 18:40:55 | mordred | Shrews: iirc, there should be a bunch of segment objects like foo/123456 - and then there should be a manifest object foo | |
| 18:41:36 | mordred | which should contain a json document pointing to all of the segment objects | |
| 18:47:41 | Shrews | mordred: ah, maybe something... | |
| 18:47:46 | Shrews | openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://storage101.ord1.clouddrive.com/v1/MossoCloudFS_28a933f9-900d-40b8-9de3-deb24d213aa2/images/opensuse-15-1564920980?multipart-manifest=put, Bad Request | |
| 18:48:09 | Shrews | mordred: if we fail to save the manifest, i bet we don't (can't?) delete the segments | |
| 18:49:25 | Shrews | that one happened 2 days ago, and is the only image segments i did not delete in ORD because it was too recent | |
| 18:50:21 | Shrews | mordred: so feeling like your first suspicion might be viable | |
| 18:51:29 | mordred | Shrews: ah - yeah | |
| 18:52:11 | mordred | Shrews: sounds like we should maybe retry the manifest put - and if it exceeds retries maybe unwind deleting teh segments since we do have the list of segments in the manifest at that point? | |
| 18:52:57 | Shrews | mordred: yup | |
| 18:53:30 | mordred | Shrews: also - a retry at that point is WAY cheaper than a hard-fail followed by re-uploading all of the segments later | |
| 18:56:28 | Shrews | totally | |
| 18:56:47 | Shrews | will see what kind of code i can barf out after the meeting | |
| 18:57:49 | Shrews | mordred: also, i'm guessing the 'images_segments' container is an old artifact? doesn't seem to be used | |
| 18:58:46 | Shrews | it exists in rax, but empty across the board | |
| 19:08:41 | mordred | yeah - I thnk old | |
| 19:12:06 | mordred | efried: dammit. your sensible request for tests has caused me to remember that something else is broken and find a whole new thing is broken | |
| 19:17:20 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Add support for global_request_id https://review.opendev.org/674807 | |
| 19:17:45 | mordred | efried, dtantsur|afk: ^^ that adds a test and fixes the other issues - and also 2 things are borked that I would like to get fixed | |
| 19:49:22 | efried | mordred: ack, and #sorrynotsorry | |
| 20:00:53 | mordred | Shrews: we're naming the swift object after the image name, right? | |
| 20:01:20 | Shrews | mordred: i believe so | |
| 20:01:33 | mordred | Shrews: so we could also do a scan of objects matching the pattern ${imagename}/${index} on image delete perhaps? | |
| 20:02:26 | Shrews | mordred: well, yeah, but why? as an extra precaution? | |
| 20:03:49 | efried | mordred: reviewed | |
| 20:05:29 | Shrews | mordred: unless in rax, the images automatically reference the swift object? i had assumed that there was some step after uploading to make it a proper "image", but maybe that's not correct | |
| 20:07:40 | mordred | Shrews: well - there's no swift object anywhere but rax | |
| 20:07:52 | mordred | Shrews: but yeah - extra precaution - probably not worth the energy | |
| #openstack-sdks - 2019-08-07 | |||
| 07:52:48 | openstackgerrit | Adrian Turjak proposed openstack/keystoneauth master: add support for auth_receipts and multi-method auth https://review.opendev.org/675049 | |
| 07:59:10 | openstackgerrit | Adrian Turjak proposed openstack/keystoneauth master: add support for auth_receipts and multi-method auth https://review.opendev.org/675049 | |
| 10:10:06 | dtantsur | mordred: we don't have anything to build swift temporary URLs in SDK, do we? | |
| 10:13:05 | dtantsur | we could probably just copy https://opendev.org/openstack/python-swiftclient/src/branch/master/swiftclient/utils.py#L71 as it is :) | |
| 13:04:47 | mordred | dtantsur: we've got support for form_post which uses the temp-url-key infrastructure | |
| 13:05:04 | mordred | dtantsur: so yeah, we could probably lift in that code for generating temp_urls themselves pretty easily | |
| 13:05:09 | openstackgerrit | Loan Harrouin proposed openstack/openstacksdk master: Add get_actions for servers https://review.opendev.org/675113 | |
| 13:05:55 | dtantsur | makes sense. it seems that the code in swiftclient doesn't depend on anything specific to swiftclient | |
| 13:08:40 | openstackgerrit | Loan Harrouin proposed openstack/openstacksdk master: Add get_actions for server resource https://review.opendev.org/675113 | |
| 13:22:30 | edleafe | Who knew that our API work extended to snail mail? https://twitter.com/shawnacscott/status/1158813468704698368 | |
| 13:24:26 | dtantsur | oh lol | |
| 13:39:20 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Initalize pool_executor so close works https://review.opendev.org/675120 | |
| 13:39:20 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Add support for global_request_id https://review.opendev.org/674807 | |
| 13:39:21 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Fix discovery cache sharing https://review.opendev.org/675121 | |
| 13:39:38 | mordred | dtantsur, efried: ^^ there we go - with discovery cache fixed and also with context manager fixed | |
| 13:46:21 | Shrews | mordred: do we have any existing tests in sdk that exercise the large object code? I'm not finding any | |
| 13:46:37 | mordred | Shrews: yes - becuase they sucked to write | |
| 13:46:45 | Shrews | :) | |
| 13:46:56 | Shrews | i must be looking in the wrong places then | |
| 13:48:09 | mordred | Shrews: test_create_dynamic_large_object and test_create_static_large_object in openstack/tests/unit/cloud/test_object.py | |
| 13:48:26 | Shrews | ah ha | |
| 13:48:33 | mordred | also test_object_segment_retry_failure | |
| 13:48:53 | Shrews | i forgot we still have mixed tests | |
| 13:52:47 | mordred | yeah. one of these days it's all going to be perfect and shiny | |
| 14:03:12 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Add support for global_request_id https://review.opendev.org/674807 | |
| 14:03:12 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Fix discovery cache sharing https://review.opendev.org/675121 | |
| 14:04:02 | mordred | Shrews: if you have a sec, the first two in that stack are bugfixes I noticed while working on the third for efried | |
| 14:04:52 | openstackgerrit | Loan Harrouin proposed openstack/openstacksdk master: Add get_actions for server resource https://review.opendev.org/675113 | |
| 14:05:19 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Add support for global_request_id https://review.opendev.org/674807 | |
| 14:05:30 | mordred | efried: you are quick at re-reviewing things | |
| 14:06:39 | mordred | dtantsur: if we do lift in the temp-url stuff, we should probably look to see what from the form_post method we can reuse. (we're using that form-post method in zuul log uploading to swift, so I'm pretty sure it's solid) | |
| 14:07:06 | dtantsur | makes sense, yeah | |
| 14:41:21 | efried | mordred: I just happened to be looking at my email at the time :) | |
| 14:47:06 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Rationalize endpoint_for and get_endpoint_from_catalog https://review.opendev.org/675130 | |
| 14:47:38 | mordred | Shrews, dtantsur|bbl: ^^ that also adds some tests and docs for a method we added last week - and makes it make sense with the existing method that was already there | |
| 14:52:09 | efried | mordred: Attempting to use sdk auto passthrough proxy for cyborg gets Sundar this error: http://paste.openstack.org/show/755594/ | |
| 14:52:16 | efried | is that expected? | |
| 14:55:31 | mordred | efried: does cyborg not publish a version discovery document? | |
| 14:56:19 | efried | I was certain they did | |
| 14:56:37 | efried | mordred: esp because they have a v1 that's dead, and are working up v2 now. | |
| 14:58:41 | mordred | hrm. do you have a link to code where that's happening? I'd like to see the value for accelerator_api_version - or more generally the clouds.yaml / config settings and also the discovery documents | |
| 14:59:35 | mordred | that error basically happens when a version has been requested and discovery can't find a major api version on the service | |
| 15:00:15 | mordred | in any cas e- let's dig in and figure out what's going on there | |
| 15:01:02 | mordred | efried: oh - I'm sorry - I misspoke - that's what happens when there is no configured api version | |
| 15:01:15 | efried | mordred: Okay. Sundar is Pacific so might be a bit before he's on. The conf is here http://paste.openstack.org/show/755593/ | |
| 15:01:46 | mordred | efried: oh - blerg. this is dumb | |
| 15:01:47 | efried | mordred: Oh, so you mean he could put a version in the conf and/or constructor and be good to go? | |
| 15:01:59 | mordred | efried: I have NO IDEA why I thought the logic for this case would be reasonable | |
| 15:02:16 | efried | heh | |
| 15:05:21 | mordred | efried: fix coming. also - I think we can make a more substantial improvement - which is to make a base class and a subclass of the ServiceDescription object - one that we use for services we know about, and one for thigns that we're just creating becaues they're in service-types-authority | |
| 15:05:53 | mordred | because there's warnings we throw in the current case for "you're only going to get passthrough REST here" - but that's a warning if we have a supported Proxy version because that means something went wrong | |
| 15:06:09 | mordred | if we DON'T have one of those, then you're expecting a REST passthrough, so the warning is stupid | |
| 15:06:22 | mordred | but - there's just a dumb logic error for the cyborg case | |
| 15:06:32 | efried | mordred: So then we wouldn't actually need the placement passthrough you made earlier this week? | |
| 15:07:03 | efried | this feels like it's kinda in the same neighborhood | |
| 15:08:23 | openstackgerrit | David Shrewsbury proposed openstack/openstacksdk master: WIP: Retry SLO manifest upload https://review.opendev.org/675134 | |
| 15:09:12 | Shrews | mordred: Want to check that I'm attacking this in the right way before I get too far? ^^^ | |
| 15:10:01 | mordred | efried: it is in the same neighborhood - and perhaps | |
| 15:10:23 | mordred | Shrews: yeah - I thnik so | |
| 15:13:37 | mordred | efried: yah - I can make one small patch to make this whole thing much easier I think | |