Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2018-10-19
13:00:44 cmurphy :D
13:00:53 dtantsur heh
13:00:58 dtantsur Python 4.2
13:01:06 cmurphy lol
13:12:52 samueldmq morning sdk
13:15:50 mordred morning samueldmq !
13:25:17 openstackgerrit Samuel de Medeiros Queiroz proposed openstack/openstacksdk master: DNM: Logging image code path https://review.openstack.org/611868
13:29:33 openstackgerrit Samuel de Medeiros Queiroz proposed openstack/openstacksdk master: DNM: Logging public server code path https://review.openstack.org/611869
14:15:41 openstackgerrit Monty Taylor proposed openstack/keystoneauth master: Add support for client-side rate limiting https://review.openstack.org/605043
14:18:19 mordred dtantsur: ^^ I reworked it a bit based on your review and put the rate limit wait into the semaphore itself
14:18:33 mordred dtantsur: I tried to add some comments - but it's still a pretty dense class
14:18:41 dtantsur cool, will check
14:19:34 mordred kmalloc, cmurphy: ^^ also y'all. of course, it can't land without tests - so I still have to figure that out - but if you feel like attacking it logically, awesome
14:25:43 dtantsur mordred: I have a feeling there is a deadlock lurking there
14:25:46 dtantsur see inline
14:28:11 mordred dtantsur: ah - yeah. it should be fine as long as there are more than one thread executing at the same time - but not for the 1st thread or in a single-threaded mode ... good catch
14:30:43 dtantsur mordred: I'm not sure how multiple thread would help. You're calling wait() while holding a lock. For wait() to return something needs to acquire the same lock and fire the event.
14:30:55 dtantsur It cannot acquire the lock held during wait(). Am I missing something?
14:31:57 mordred dtantsur: no - you're totally right
14:32:04 mordred dtantsur: I need another coffee :)
14:32:33 dtantsur mordred: the resource starvation problem you mention in the commit message.. is it really so bad to justify writing your own semaphore class? :)
14:33:16 mordred dtantsur: yup :)
14:34:35 openstackgerrit Stephen Finucane proposed openstack/cliff master: Stop wildcard importing argparse https://review.openstack.org/611889
14:34:35 openstackgerrit Stephen Finucane proposed openstack/cliff master: Remove dead files https://review.openstack.org/611888
14:34:36 openstackgerrit Stephen Finucane proposed openstack/cliff master: WIP: Add support for environment variables https://review.openstack.org/611890
14:34:37 mordred dtantsur: amazingly enough - all of the varied use cases this solves for are represented by actual real-world issues
14:35:11 mordred dtantsur: you're think that the server side would just be able to handle all of this without the client needing to care ... but oh golly
14:35:15 dtantsur that damn reality; I like dealing with round things in vacuum much more
14:35:18 mordred right?
14:35:23 dtantsur ofc
14:35:32 mordred dtantsur: remind me to tell you the story of the time we crashed the HP Public Cloud
14:35:45 mordred just by running nodepool against its public api
14:35:58 dtantsur OH I'd LOVE to hear that
14:37:02 mordred they had a hardware load balancer / api gateway in front of the api services - which would time out client connections for requests that took too long
14:37:23 mordred so, on receiving a 500 from the load balancer - we'd auto retry again
14:37:55 mordred problem was - the requests were taking so long because there was a missing index on a sql table, so each request was causing a long running query to get started in mysql
14:39:00 dtantsur LOOOL
14:39:47 mordred bcause it was doing a full table scan on the server table for every operation - oh, and also they hadn't been cleaning out old deleted entries from the table
14:40:19 dtantsur hashtag successstory
14:40:25 mordred so we'd to, while True: try: create_server ; break except: delete_server
14:40:42 mordred every create would fail, at which point we'd attempt to clean up after ourselves
14:40:53 mordred in a tight loop with no pausing
14:41:11 mordred and then everything melted to the ground :)
14:41:35 dtantsur well, you gave them a free DoS attack testing :D some people pay for that
14:41:41 mordred RIGHT?
14:41:53 mordred this is how we try to convince more people to give us resources for nodepool
14:42:03 mordred "we promise we can find the errors in your cloud deployment"
14:43:11 dtantsur okay, I should stop laughing so loudly, my wife will become suspicious :D
14:50:11 mordred hehehe
14:50:41 openstackgerrit Monty Taylor proposed openstack/keystoneauth master: Add support for client-side rate limiting https://review.openstack.org/605043
14:51:07 mordred dtantsur: figuring out how to write unittests for this thing is going to be ... fun
14:51:18 kmalloc Mock
14:51:22 kmalloc Lots of mock
14:51:54 mordred kmalloc: I dunno ... seems like a good risk of testing that we wrote some mocks
14:51:58 kmalloc My recommendation is mock plus freezegun, check to make sure x calls were made
14:52:05 dtantsur mock the heck out of it \o/
14:52:11 kmalloc With the right args
14:53:37 kmalloc You can introspect the magicmock object and see what was called
14:54:27 kmalloc The underlying request stuff is tested elsewhere.
14:54:41 mordred oh - totally - definitely not worried about testing that
14:54:57 kmalloc So we just want to check the ratelimiting and locking.
14:55:12 dtantsur oh, how I "love" ironicclient+keystoneauth spitting "Service catalog is empty" on wrong magical combination of arguments..
14:55:14 mordred mostly just want to make sure we're hitting things like dead locks and max concurrency and stuff
14:55:19 kmalloc All doable with simple wsgiref webtest app.
14:56:00 kmalloc You're looking to do a simple reflector thing :)
14:56:09 mordred you're a simple reflector thing
14:56:42 dtantsur lol
14:59:08 mordred kmalloc: this is more moist than I prefer
14:59:45 kmalloc I suggest a lemur dryer then ;)
15:45:23 mnaser mordred: i was just hit by this .. https://github.com/ansible/ansible/pull/42913 -- do you mind giving it another shipit if it makes sense to you?
15:45:36 mnaser cc Shrews rcarrillocruz ^
15:55:14 mordred mnaser: seems reasonable - but also that kinda feels like a default behavior sdk should have on the backend
15:55:34 mordred oh - wait- I misread soemthing
16:09:31 mnaser mordred: in this case a cloud with multiple users having “admin” would fail (in different domains)
16:09:54 mnaser mordred: thank you!
17:07:42 kmalloc mordred: the limits code for SDK looks correct
18:17:58 mordred kmalloc: yay!
#openstack-sdks - 2018-10-20
11:03:51 openstackgerrit Yuval Shalev proposed openstack/openstacksdk master: Added assign function to identity proxy https://review.openstack.org/612006
11:08:05 openstackgerrit Yuval Shalev proposed openstack/openstacksdk master: Added assign function to identity v3 proxy https://review.openstack.org/612006
14:14:46 openstackgerrit Yuval Shalev proposed openstack/openstacksdk master: Added assign function to identity v3 proxy https://review.openstack.org/612006
#openstack-sdks - 2018-10-21
02:17:45 openstackgerrit Daniel Speichert proposed openstack/openstacksdk master: Fix upload of Swift object smaller than segment limit (create_object) https://review.openstack.org/612113
12:43:27 openstackgerrit Maxim Babushkin proposed openstack/openstacksdk master: Add vnic_type to create_port valid kwargs https://review.openstack.org/612162
13:33:12 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Add support for per-service rate limits https://review.openstack.org/604521
15:26:09 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Replace TaskManager with a keystoneauth concurrency https://review.openstack.org/604926
15:26:09 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Add support for per-service rate limits https://review.openstack.org/604521
15:30:14 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Don't start task managers passed in to Connection https://review.openstack.org/612186
15:43:34 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Don't start task managers passed in to Connection https://review.openstack.org/612186
15:54:49 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Replace TaskManager with a keystoneauth concurrency https://review.openstack.org/604926
15:54:49 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Add support for per-service rate limits https://review.openstack.org/604521
15:59:19 openstackgerrit Monty Taylor proposed openstack/openstacksdk master: Replace TaskManager with a keystoneauth concurrency https://review.openstack.org/604926
18:03:43 samueldmq a few weeks ago we had a discussion around some clouds that use radosgw implementing swift APIs in OpenStack deployments
18:04:14 samueldmq we concluded that we don't need to support radosgw in SDK, and that's the very right thing aligned with OpenStack principles
18:04:44 samueldmq OpenStack Primarily Produces Software: '[...] We expect people who say they run “OpenStack” to run the software produced by and in the community, rather than alternative implementations of the API.'
18:04:52 samueldmq #link https://governance.openstack.org/tc/reference/principles.html
18:04:56 samueldmq just fyi
18:15:48 mordred samueldmq: well - we actually do support people running radosgw - and are using sdk to talk to both swift and radosgw in the upload-to-swift ansible role for swift
18:16:41 mordred samueldmq: I agree with the principle in general (I think I helped write it) - but in this case it's not clear to a user of an openstack cloud whether the cloud in question is running swift or radosgw - for the most part the differences have been reasonably minor
18:18:11 samueldmq mordred: interesting. for the user I agree it's not fair to say "I can't talk to that cloud because t does not meet the OpenStack principles, please talk to the cloud provider"
18:19:10 mordred yeah. similarly we have several piecesof logic in sdk to support weird behaviors in rackspace public cloud that also aren't *really* openstack
18:19:23 samueldmq but we also need to be careful how far that goes, for now it's just one in the future might be multiple code saying they support our apis

Earlier   Later