| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-sdks - 2018-10-25 | |||
| 18:32:55 | kmalloc | i would support in your service /v1 <current> | |
| 18:32:59 | kmalloc | and /v2 <new> | |
| 18:33:05 | kmalloc | and have / be a discovery doc | |
| 18:33:17 | kmalloc | so folks can determine what is there (standard for openstack projects these days) | |
| 18:33:27 | kmalloc | keystoneauth has a standard way of doing the discovery bits | |
| 18:33:36 | kmalloc | (ignoring ironic's legacy stuff) | |
| 18:34:01 | peschk_l | OK, I'll look into that | |
| 18:34:19 | kmalloc | that way your endpoints aren't serviceV2 it is just "service" | |
| 18:34:29 | kmalloc | and the user can see "oh service supports v2, i want to use v2" | |
| 18:34:42 | kmalloc | vs needing to go to a totally different endpoint (service/port/host) for v2 | |
| 18:34:51 | edleafe | And existing tools/SDKs can still work against v1 forever | |
| 18:35:36 | kmalloc | just make sure v1 informs smart/sane defaults for the values that the user can't pass to the new backends | |
| 18:35:47 | kmalloc | or the backends have sane defaults built in | |
| 18:35:57 | kmalloc | that way new storage doesn't break if someone uses v1 | |
| 18:36:42 | peschk_l | I see. But then the only way do distinguish between v1+v2 with 1 endpoint and v1+v2 with 4 endpoints and fixes would be the openstack release ? | |
| 18:36:58 | kmalloc | in the discovery doc | |
| 18:37:35 | kmalloc | so you'd look at Service X / and see it supports v1, or v1+v2 or v1+v2(and microversions on v2) | |
| 18:37:45 | peschk_l | current v1 API is compatible with the v2 storage. However, v2 API features won't be compatible with the v1 storage | |
| 18:37:49 | edleafe | peschk_l: if you ever have insomnia and need something to help you sleep, this is excellent reading: http://specs.openstack.org/openstack/api-sig/guidelines/consuming-catalog.html#consuming-catalog | |
| 18:38:33 | kmalloc | if you only ever implement new storage once v2 is there, the v1 stuff populates defaults to the storage, v2 allows greater depth of configuration | |
| 18:38:46 | kmalloc | if someone is hitting an endpoint with only v1, theyn get current behavior | |
| 18:38:51 | kmalloc | typically the way this is done is | |
| 18:38:56 | kmalloc | 1) implement new storage, make v1 compat | |
| 18:39:05 | kmalloc | 2) implement expirmental v2 api, v1 still works | |
| 18:39:13 | kmalloc | 3) stable v2 api, v1 still works | |
| 18:39:20 | peschk_l | edleafe: thx! (funny, it seems like you are providing a lot of links which I desperately looked for but just couldn't find) | |
| 18:39:34 | kmalloc | and you have no fundamental incompatibilities | |
| 18:39:44 | kmalloc | you can change the requirements for storage in future versions of the service | |
| 18:40:03 | kmalloc | the API should abstract it out so the user can't tell the implementation details on the backend (a good API designed does that) | |
| 18:40:13 | kmalloc | s/good API designed/well designed API | |
| 18:40:25 | edleafe | peschk_l: Here's a good link to bookmark: http://specs.openstack.org/openstack/api-sig/#guidelines | |
| 18:42:15 | peschk_l | kmalloc: does this means that we'd need to backport every new v2 api feature to v1 until support for v1 storage is dropped ? | |
| 18:42:31 | edleafe | peschk_l: no, not at all | |
| 18:42:45 | kmalloc | iterate on storage and get it to where you want. | |
| 18:42:51 | kmalloc | and just make v1 always use the new storage | |
| 18:43:02 | kmalloc | with sane defaults | |
| 18:43:13 | kmalloc | don't backport features, make it so v1 continues to work as expected | |
| 18:43:27 | kmalloc | ideally the API and the storage backend should have little bearing on eachother | |
| 18:43:49 | kmalloc | Users interact with the API, the service translates those interactions to something the storage system can understand | |
| 18:43:53 | peschk_l | OK, that's what I understood earlier. This is how it has been implemented until now | |
| 18:44:11 | kmalloc | so you'd have new shiny-storage and everyone would migrate to it | |
| 18:44:15 | kmalloc | v1 just uses that | |
| 18:44:38 | kmalloc | and then v2 features expose the new-awesome(tm) [YES TRADEMARKED!] things to the end users that the storage system can do | |
| 18:45:08 | peschk_l | kmalloc: ok, thanks for the clarification :) | |
| 18:45:10 | kmalloc | :) | |
| 18:45:13 | kmalloc | hope that helps | |
| 18:45:24 | kmalloc | now... wsme/pecan vs flask vs webob | |
| 18:45:32 | kmalloc | all of that is independent of your API | |
| 18:45:41 | kmalloc | those are tools/frameworks to build your service | |
| 18:45:49 | kmalloc | you can convert or not | |
| 18:45:58 | edleafe | peschk_l: Also, the 'Note' at the top of this page is useful for distinguishing the different API statuses: https://developer.openstack.org/api-guide/quick-start/ | |
| 18:46:01 | kmalloc | there are also ways to layer compatibility in | |
| 18:46:19 | kmalloc | if you want flask i can show you some ways to layer v1 in on top of v2 keeping the old stuff | |
| 18:46:31 | kmalloc | but don't feel like you need to change out the framework just to do v2 | |
| 18:46:42 | kmalloc | it is a LOT of work to swap frameworks/change that stuff out | |
| 18:47:16 | kmalloc | fwiw, (and keystone has a huge api surface area) it was almost 100 commits and probably over 30,000 lines of code changed to get there | |
| 18:47:29 | kmalloc | changing frameworks is VERY disruptive, but can be worth it in some cases | |
| 18:48:08 | kmalloc | just keep in mind the technical cost of doing so | |
| 18:48:29 | kmalloc | and plan for that separately from the implementation of the new API(s) | |
| 18:49:39 | peschk_l | kmalloc: we were thinking about an "easy" way: serving two wsgi apps on /v1 and /v2. That way, we'd only need to modify the root controller | |
| 18:50:00 | kmalloc | so i can show you a SUPER easy way | |
| 18:50:35 | peschk_l | (don't worry, cloudkitty's whole codebase is probably smaller thant keystone's API) | |
| 18:50:40 | peschk_l | *than | |
| 18:50:51 | peschk_l | kmalloc: would be glad to hear about it | |
| 18:51:06 | kmalloc | http://werkzeug.pocoo.org/docs/0.14/middlewares/#werkzeug.wsgi.DispatcherMiddleware | |
| 18:51:39 | kmalloc | that middleware right there lets you say "all paths for prefix /XXX goes to app X and all prefixes that don't match fall through to the application" | |
| 18:51:55 | kmalloc | you can have as many rules for matching and passing to apps as you want | |
| 18:52:06 | kmalloc | werkzeug is super cool with the middlewares it provides | |
| 18:52:20 | kmalloc | (werkzeug is the base for flask, but you can use it's middleware without flask) | |
| 18:52:33 | peschk_l | actually, we were planning to use exactly this, but we were not sure if it was a prod-ready | |
| 18:53:01 | kmalloc | keystone ran with it in rocky | |
| 18:53:10 | kmalloc | we subclassed because we needed a LOT of extra stuff | |
| 18:53:21 | kmalloc | but i have no concerns with stable werkzeug code in prod | |
| 18:53:35 | kmalloc | in fact, keystone still uses it | |
| 18:53:42 | kmalloc | and will for the foreseeable future | |
| 18:53:58 | peschk_l | glad to hear that :) is keystone v2 still based on pecan ? | |
| 18:54:09 | kmalloc | https://github.com/openstack/keystone/blob/cd8f7a503673de1cd603b2f5a66e4bbfe3085583/keystone/server/flask/application.py#L163-L171 | |
| 18:54:20 | kmalloc | keystone v2 was based on raw webob | |
| 18:54:27 | kmalloc | keystone v2 also has been deleted and no longer exists | |
| 18:54:41 | kmalloc | i used that middleware to dispatch each of keystone's APIs for the transition to flask | |
| 18:54:56 | kmalloc | so i moved /auth to flask and still dispatched code to the old webob v3 api for /projects | |
| 18:55:01 | kmalloc | until /projects was converted | |
| 18:55:02 | kmalloc | for example | |
| 18:55:47 | kmalloc | peschk_l: you can see what we did here https://github.com/openstack/keystone/blob/stable/rocky/keystone/server/flask/application.py#L81 | |
| 18:56:05 | kmalloc | we moved parts of our api to flask in smaller chunks | |
| 18:56:24 | kmalloc | (master looks far different as you can see, since we run 100% on flask now) | |
| 18:57:21 | kmalloc | doctor appointment | |
| 18:57:25 | kmalloc | be back in a few hours | |
| 18:58:19 | peschk_l | kmalloc: I'll probably be sleeping when you return, but thank you very much for all the information, it's been a HUGE help :) | |
| 18:58:41 | kmalloc | happy to help! | |
| 18:58:45 | kmalloc | edleafe: i don't envy you | |
| 18:58:48 | peschk_l | edleafe are you still there ? | |
| 18:59:00 | edleafe | yeah | |
| 18:59:32 | edleafe | but about to disappear into several meetings | |
| 19:00:16 | peschk_l | oh, didn't see you need to go too. Just for the record: I saw that paste is still maintained. Is this only for backward compatibility and should we plan to drop it, or can we stick with it ? | |
| 19:01:55 | peschk_l | and in the case we need to move on: is there a recommendation from your side ? | |
| 20:04:05 | mriedem | who besides dean is an osc core that i can bug for reviews on this old bug fix? https://review.openstack.org/545946 | |
| 20:08:13 | mriedem | amotoki: ^? | |
| 20:21:48 | mordred | mriedem: looking | |
| 20:22:25 | mriedem | i'm sure it'll be the best thing you've looked at all day | |