Earlier  
Later
New Search
Posted Nick Remark
#openstack-sdks - 2018-07-24
15:48:58 mordred samueldmq: woot.
15:49:22 samueldmq mordred: where is shade code in the new -sdk?
15:49:37 mordred samueldmq: although - for your case, it's also possible that doing the formal validation by hand would be fine - since that's different from making sure to run validation on every patch
15:49:44 mordred samueldmq: openstack/cloud
15:50:03 samueldmq mordred: ++ exactly... just want to validate the tool works
15:50:11 samueldmq also, I don't need/want to be exhaustive or too complex
15:50:22 mordred samueldmq: and it's all attached to the Connection object - so as a user if you do "conn - openstack.connect(cloud='vexxhost')" - you can use that conn object same as the OpenStackCloud object from shade
15:50:26 mordred samueldmq: ++
15:50:41 samueldmq validating a core set of functionalities should be fine (whatever be a core set: vm managemnt?)
15:50:56 mordred samueldmq: https://review.openstack.org/#/c/582459/ is a patch you might want to look at
15:51:27 mordred samueldmq: it's a patch to allow running our ansible tests against a public cloud that is not devstack
15:51:39 mordred Shrews: ^^ speaking of - you might want to check out that patch too
15:51:50 samueldmq mordred: awesome
15:51:56 samueldmq do all the tests normally pass?
15:52:17 samueldmq I wouldn't be surprised otherwise... there may be some services that do not exist in some cases, for ex
15:52:33 mordred samueldmq: well - our ansible tests have a pretty small surface area
15:53:06 mordred I'd love to be able to run our functional tests against a cloud - and hopefully we'll also get that working at some point
15:53:20 samueldmq mordred: how do you see that?
15:53:21 mordred samueldmq: and I think it's ok for the test suite to skip some tests if they don't have a service
15:53:27 samueldmq reimplement with ansible
15:53:36 samueldmq or wrap the existing tests somehow to make it owrk
15:53:39 mordred nah - it's mostly just about reworking base fixtures
15:54:06 mordred in some clouds we may not have admin, for instance, so making a new project and testing keystone admin functions should all be skipped
15:54:08 samueldmq to allow oyu input the cloud to talk against
15:54:11 mordred but we could still test that booting a vm works
15:54:15 samueldmq when running tests right
15:55:22 mordred the tests already will skip if the service doesn't exist - then we have a devstack-specific test that responds to env vars and fails if an envvar asserts a service should exist but the shade tests don't find it
15:55:31 mordred that way we can make sure we don't accidentally fail open
15:55:36 mordred i the gate
15:55:48 mordred so we could probably expand on that somehow
15:56:10 Shrews mordred: lgtm
15:56:12 mordred to allow manual running against public clouds to be more flexible without sacrificing gate stuff
15:56:14 mordred Shrews: woot
15:56:18 samueldmq mordred: how are those vars set up then?
15:56:29 samueldmq I assume that cn be automatic by looking up the service catalog
15:56:34 samueldmq but would require some work
15:56:48 mordred well - "has_service('network')" uses the service catalog
15:56:54 mordred but the env var is defined in the test job
15:57:18 mordred so the test job (which sets up the services in the first place) also sets vars that say "I believe I set up magnum"
15:57:32 mordred then the shade test says "the test config expects magnum, do I see it in the catalog"
15:57:41 samueldmq ah makes sense
15:57:48 samueldmq but none of that apply to manual run
15:57:51 mordred exactly
15:58:01 samueldmq also, runnig against real public clouds wouldn't require any setup
15:58:08 mordred that's the theory :)
15:58:09 samueldmq openstack/ocnfig that is
15:58:25 mordred in practice, I think there are still a good amount of code in fixtures that assumes we're running against a devstack
15:58:34 mordred and could be make more smarter, we just haven't had the time yet
15:58:38 Shrews yeah
15:58:39 samueldmq "whatever that can happen will happen" someone said
15:58:45 mordred ++
15:58:56 samueldmq and then shade exists :-)
15:59:33 samueldmq kk I have some info to mull over
16:00:00 samueldmq I'm happy I got an old laptop running linux again, was bored using windows all the time for company stuff
16:00:01 Shrews somewhere in shade i had a script that allowed us to run those tests against another (non-devstack) cloud
16:00:13 samueldmq which means I am expecting to start having fun around here
16:00:28 mordred samueldmq: yay!
16:00:29 samueldmq Shrews: that'd be a good start
16:00:32 mordred Shrews: yeah- I remember that ...
16:00:34 samueldmq \o/
16:00:48 Shrews oh, even tox support for that
16:00:56 Shrews tox -e ansible -- -c cloudX [TAG ...]
16:01:05 Shrews how clever i was
16:01:37 samueldmq and that looks up on the regular /opt/openstack/clouds.yaml?
16:01:41 Shrews yeah
16:01:47 samueldmq or is there a special thing that was setup for devstack
16:01:48 samueldmq kk
16:02:04 Shrews and the tags were to selectively choose which things you wanted to test (in case the cloud didn't support something)
16:02:49 samueldmq gotcha
16:03:10 Shrews looks like we carried that over to sdk, but i haven't tested it there
16:03:23 samueldmq I'll take a look to get familiar
16:03:30 samueldmq and see what works and what doesn't
16:04:14 Shrews \o/
16:04:29 samueldmq :D
17:38:52 openstackgerrit Merged openstack/openstacksdk master: Run ansible tests against specific public cloud https://review.openstack.org/582459
18:22:44 openstackgerrit Merged openstack/keystoneauth master: Add ability to filter version data by service-type https://review.openstack.org/585029
18:37:40 mordred Shrews, dtroyer, dhellmann: feel like reviewing some os-service-types patches? (https://review.openstack.org/#/q/project:openstack/os-service-types+status:open)
18:51:02 Shrews i don't even know what that repo is (though i can infer from the name) lol
18:54:48 mordred Shrews: :)
19:32:05 corvus mordred: i'm having trouble using the new data= parameter to create_object:
19:32:07 corvus TypeError: object of type 'DeflateFilter' has no len()
19:32:07 corvus file_size = len(data)
19:32:07 corvus File "/home/corvus/git/openstack-infra/zuul-jobs/.tox/py35/lib/python3.5/site-packages/openstack/cloud/openstackcloud.py", line 7525, in create_object
19:32:45 corvus mordred: the session.put code i was using before handled data being an iterator
19:33:40 mordred corvus: ah! hrm. SO ...
19:34:32 corvus er, cloud.object_store.put which i guess is the proxy put. which, i think, is basically going to call session.put. you get the idea.
19:34:33 mordred corvus: that len() is there in service of figuring out whether the data to be uploaded needs to be uploaded as multiple segments
19:34:54 corvus mordred: i believe httplib handles that automatically
19:35:31 mordred well, not for swift it doesn't
19:35:36 corvus (briefly, if it is an iterator with no len, it gets uploaded in multiple segments)
19:35:45 corvus sorry, chunked encoding
19:35:55 corvus are you talking about something else?
19:36:07 mordred yah - this is about uploading to multiple swift objects if the object size is > max_objet_size
19:36:23 mordred and then creating either a DLO or SLO object out of that
19:36:30 mordred (dynamic large object or static large object)
19:36:45 corvus mordred: that's something you have to decide on before you upload?
19:37:21 mordred yup- it impacts how you upload things - and also causes sdk to spawn up a set of worker threads to handle it if it's needed
19:38:54 corvus that sounds fundamentally incompatible with streaming compression
19:39:32 mordred well ...
19:40:12 mordred I believe each segment upload itself could be uploaded with streaming compression

Earlier   Later