| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-sdks - 2018-10-25 | |||
| 18:27:16 | kmalloc | ++ | |
| 18:27:40 | peschk_l | edleafe: we'd like to refactor the whole API... But we won't be able to accomplish that in a single release. So the idea would be to implement new endpoints within v2 only, and to slowly migrate "legacy" endpoints from v1 to v2 | |
| 18:27:49 | peschk_l | (thanks for the link btw) | |
| 18:28:45 | kmalloc | i also highly recommend, whatever you do, not version your endpoints in the catalog | |
| 18:28:59 | kmalloc | support v1/v2 (as you transition) on the same server/wsgi application | |
| 18:29:07 | kmalloc | versioned endpoints are a bad idea. | |
| 18:29:08 | peschk_l | Honestly, v1 has some major issues, for example (and I'm ashamed), there is no way to paginate results, except with begin/end timestamps | |
| 18:29:22 | edleafe | peschk_l: A general approach for creating the new API is along these lines: | |
| 18:29:39 | edleafe | 1) create a new endpoint, labeled 'EXPERIMENTAL' | |
| 18:29:53 | edleafe | 2) start adding more and more features to that endpoint | |
| 18:29:54 | peschk_l | kmalloc: we weren't planning to manage the endpoints. versions would only have been "markers" for which endpoints are supported, and which are not | |
| 18:30:41 | edleafe | 3) When you finally have a good, working version, change that version to 'CURRENT' and change the old version to 'SUPPORTED' | |
| 18:31:06 | edleafe | This way users can opt into the new API if they wish at their own pace, knowing that it is subject to change | |
| 18:31:10 | kmalloc | ++ | |
| 18:31:28 | kmalloc | ^ that is what i was in the middle of typing before edleafe beat me to it | |
| 18:31:40 | edleafe | You don't flip the switch until the new API is solid, and your can't see any more work needed on it | |
| 18:31:54 | kmalloc | and at that point you can support microversions on v2 | |
| 18:32:05 | kmalloc | and v1 is legacy, maintained for compat with old clients/users | |
| 18:32:35 | kmalloc | you give all the shiny new things to v2/v2+microversions so folks are encouraged to use v2 | |
| 18:32:36 | peschk_l | I see. Would you still add endpoint for new features on v2 only, and mark them as experimental ? | |
| 18:32:53 | peschk_l | kmalloc: ok, got my answer :) | |
| 18:32:55 | kmalloc | i would support in your service /v1 <current> | |
| 18:32:59 | kmalloc | and /v2 <new> | |
| 18:33:05 | kmalloc | and have / be a discovery doc | |
| 18:33:17 | kmalloc | so folks can determine what is there (standard for openstack projects these days) | |
| 18:33:27 | kmalloc | keystoneauth has a standard way of doing the discovery bits | |
| 18:33:36 | kmalloc | (ignoring ironic's legacy stuff) | |
| 18:34:01 | peschk_l | OK, I'll look into that | |
| 18:34:19 | kmalloc | that way your endpoints aren't serviceV2 it is just "service" | |
| 18:34:29 | kmalloc | and the user can see "oh service supports v2, i want to use v2" | |
| 18:34:42 | kmalloc | vs needing to go to a totally different endpoint (service/port/host) for v2 | |
| 18:34:51 | edleafe | And existing tools/SDKs can still work against v1 forever | |
| 18:35:36 | kmalloc | just make sure v1 informs smart/sane defaults for the values that the user can't pass to the new backends | |
| 18:35:47 | kmalloc | or the backends have sane defaults built in | |
| 18:35:57 | kmalloc | that way new storage doesn't break if someone uses v1 | |
| 18:36:42 | peschk_l | I see. But then the only way do distinguish between v1+v2 with 1 endpoint and v1+v2 with 4 endpoints and fixes would be the openstack release ? | |
| 18:36:58 | kmalloc | in the discovery doc | |
| 18:37:35 | kmalloc | so you'd look at Service X / and see it supports v1, or v1+v2 or v1+v2(and microversions on v2) | |
| 18:37:45 | peschk_l | current v1 API is compatible with the v2 storage. However, v2 API features won't be compatible with the v1 storage | |
| 18:37:49 | edleafe | peschk_l: if you ever have insomnia and need something to help you sleep, this is excellent reading: http://specs.openstack.org/openstack/api-sig/guidelines/consuming-catalog.html#consuming-catalog | |
| 18:38:33 | kmalloc | if you only ever implement new storage once v2 is there, the v1 stuff populates defaults to the storage, v2 allows greater depth of configuration | |
| 18:38:46 | kmalloc | if someone is hitting an endpoint with only v1, theyn get current behavior | |
| 18:38:51 | kmalloc | typically the way this is done is | |
| 18:38:56 | kmalloc | 1) implement new storage, make v1 compat | |
| 18:39:05 | kmalloc | 2) implement expirmental v2 api, v1 still works | |
| 18:39:13 | kmalloc | 3) stable v2 api, v1 still works | |
| 18:39:20 | peschk_l | edleafe: thx! (funny, it seems like you are providing a lot of links which I desperately looked for but just couldn't find) | |
| 18:39:34 | kmalloc | and you have no fundamental incompatibilities | |
| 18:39:44 | kmalloc | you can change the requirements for storage in future versions of the service | |
| 18:40:03 | kmalloc | the API should abstract it out so the user can't tell the implementation details on the backend (a good API designed does that) | |
| 18:40:13 | kmalloc | s/good API designed/well designed API | |
| 18:40:25 | edleafe | peschk_l: Here's a good link to bookmark: http://specs.openstack.org/openstack/api-sig/#guidelines | |
| 18:42:15 | peschk_l | kmalloc: does this means that we'd need to backport every new v2 api feature to v1 until support for v1 storage is dropped ? | |
| 18:42:31 | edleafe | peschk_l: no, not at all | |
| 18:42:45 | kmalloc | iterate on storage and get it to where you want. | |
| 18:42:51 | kmalloc | and just make v1 always use the new storage | |
| 18:43:02 | kmalloc | with sane defaults | |
| 18:43:13 | kmalloc | don't backport features, make it so v1 continues to work as expected | |
| 18:43:27 | kmalloc | ideally the API and the storage backend should have little bearing on eachother | |
| 18:43:49 | kmalloc | Users interact with the API, the service translates those interactions to something the storage system can understand | |
| 18:43:53 | peschk_l | OK, that's what I understood earlier. This is how it has been implemented until now | |
| 18:44:11 | kmalloc | so you'd have new shiny-storage and everyone would migrate to it | |
| 18:44:15 | kmalloc | v1 just uses that | |
| 18:44:38 | kmalloc | and then v2 features expose the new-awesome(tm) [YES TRADEMARKED!] things to the end users that the storage system can do | |
| 18:45:08 | peschk_l | kmalloc: ok, thanks for the clarification :) | |
| 18:45:10 | kmalloc | :) | |
| 18:45:13 | kmalloc | hope that helps | |
| 18:45:24 | kmalloc | now... wsme/pecan vs flask vs webob | |
| 18:45:32 | kmalloc | all of that is independent of your API | |
| 18:45:41 | kmalloc | those are tools/frameworks to build your service | |
| 18:45:49 | kmalloc | you can convert or not | |
| 18:45:58 | edleafe | peschk_l: Also, the 'Note' at the top of this page is useful for distinguishing the different API statuses: https://developer.openstack.org/api-guide/quick-start/ | |
| 18:46:01 | kmalloc | there are also ways to layer compatibility in | |
| 18:46:19 | kmalloc | if you want flask i can show you some ways to layer v1 in on top of v2 keeping the old stuff | |
| 18:46:31 | kmalloc | but don't feel like you need to change out the framework just to do v2 | |
| 18:46:42 | kmalloc | it is a LOT of work to swap frameworks/change that stuff out | |
| 18:47:16 | kmalloc | fwiw, (and keystone has a huge api surface area) it was almost 100 commits and probably over 30,000 lines of code changed to get there | |
| 18:47:29 | kmalloc | changing frameworks is VERY disruptive, but can be worth it in some cases | |
| 18:48:08 | kmalloc | just keep in mind the technical cost of doing so | |
| 18:48:29 | kmalloc | and plan for that separately from the implementation of the new API(s) | |
| 18:49:39 | peschk_l | kmalloc: we were thinking about an "easy" way: serving two wsgi apps on /v1 and /v2. That way, we'd only need to modify the root controller | |
| 18:50:00 | kmalloc | so i can show you a SUPER easy way | |
| 18:50:35 | peschk_l | (don't worry, cloudkitty's whole codebase is probably smaller thant keystone's API) | |
| 18:50:40 | peschk_l | *than | |
| 18:50:51 | peschk_l | kmalloc: would be glad to hear about it | |
| 18:51:06 | kmalloc | http://werkzeug.pocoo.org/docs/0.14/middlewares/#werkzeug.wsgi.DispatcherMiddleware | |
| 18:51:39 | kmalloc | that middleware right there lets you say "all paths for prefix /XXX goes to app X and all prefixes that don't match fall through to the application" | |
| 18:51:55 | kmalloc | you can have as many rules for matching and passing to apps as you want | |
| 18:52:06 | kmalloc | werkzeug is super cool with the middlewares it provides | |
| 18:52:20 | kmalloc | (werkzeug is the base for flask, but you can use it's middleware without flask) | |
| 18:52:33 | peschk_l | actually, we were planning to use exactly this, but we were not sure if it was a prod-ready | |
| 18:53:01 | kmalloc | keystone ran with it in rocky | |
| 18:53:10 | kmalloc | we subclassed because we needed a LOT of extra stuff | |
| 18:53:21 | kmalloc | but i have no concerns with stable werkzeug code in prod | |
| 18:53:35 | kmalloc | in fact, keystone still uses it | |
| 18:53:42 | kmalloc | and will for the foreseeable future | |
| 18:53:58 | peschk_l | glad to hear that :) is keystone v2 still based on pecan ? | |
| 18:54:09 | kmalloc | https://github.com/openstack/keystone/blob/cd8f7a503673de1cd603b2f5a66e4bbfe3085583/keystone/server/flask/application.py#L163-L171 | |
| 18:54:20 | kmalloc | keystone v2 was based on raw webob | |
| 18:54:27 | kmalloc | keystone v2 also has been deleted and no longer exists | |