| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-29 | |||
| 15:12:38 | stephenfin | oslo.policy? | |
| 15:12:44 | sean-k-mooney | dansmith: no that was brogh back for a different reason | |
| 15:12:48 | dansmith | i.e. we provide a base yaml, they edit it to add their stuff and tweak their prefs and then when they go to upgrade they're sadface? | |
| 15:13:12 | stephenfin | that was another concern of mine, but I figured we'd just load multiple files | |
| 15:13:17 | sean-k-mooney | e.g. if the new microversion enabled validateion and we extend the flavor update again in the future you cant use the new feautre with validation idsabled | |
| 15:13:30 | stephenfin | I'm almost certain someone will go tweak the YAML file we provide though | |
| 15:13:39 | bauzas | yuuuuup | |
| 15:13:46 | dansmith | stephenfin: I would argue that we don't/shouldn't encode our baseline validations into yaml and distribute them that way, as it's just asking for that problem | |
| 15:13:49 | dansmith | right | |
| 15:13:52 | sean-k-mooney | dansmith: so i was suggesting add a &validate query arge to turn off validation in the new microversion | |
| 15:13:53 | efried | If we do in-tree stuff with yaml, we put the nova-owned stuff within the nova package so nobody fs with it. | |
| 15:14:06 | dansmith | efried: riiiight | |
| 15:14:07 | stephenfin | sean-k-mooney: hold up, let's figure the YAML thing out first | |
| 15:14:38 | efried | riiiight they'll still f with it? Let 'em. That's breaking the seal, voiding the warranty, not different than if they go patch the code. | |
| 15:14:46 | sean-k-mooney | efried: distos will modify it every time we backport stuff | |
| 15:14:51 | gibi | we somebody want to tweak what is written in the python DSL for in-tree validation he can do it, it is almos like a yaml file | |
| 15:14:52 | dansmith | surely the yaml to validate something complex like numa whatever is going to be nasty or naive right? | |
| 15:15:12 | sean-k-mooney | it will be an interesting regex | |
| 15:15:23 | gibi | but nothing more than a regex | |
| 15:15:26 | stephenfin | gibi: I don't think it's a DSL, really. It's just an object. I could use dicts if that would be preferable | |
| 15:15:28 | dansmith | an "interesting" regex or some relatively simple python right? | |
| 15:15:42 | bauzas | :) | |
| 15:15:43 | sean-k-mooney | actully its still a regex in stephns code | |
| 15:15:49 | sean-k-mooney | well poc | |
| 15:15:56 | stephenfin | not entirely | |
| 15:15:57 | sean-k-mooney | but it could be simpler | |
| 15:16:11 | stephenfin | I let you specify enums etc. | |
| 15:16:19 | sean-k-mooney | yes | |
| 15:16:37 | stephenfin | granted, all that gets converted into a regex but there's no reason I have to do that for all cases. It's a PoC after all | |
| 15:16:43 | sean-k-mooney | although you convert the enum int an bool filed back to regex form | |
| 15:17:25 | dansmith | so I think my feeling on this is: (a) I'm not sure we need an interface for people to be able to specify their own validators, but (b) if we do, that seems like an obvious use case for entrypoints and a dead simple plugin interface with a super-controlled input and output behavior set | |
| 15:17:51 | efried | gibi: it won't be a lot of duplication, really. The parser will construct the same kind of python objects we have in tree and then feed all of them into the validation engine. | |
| 15:18:04 | stephenfin | gibi: yeah, what efried said | |
| 15:18:19 | gibi | OK, I rest my case then | |
| 15:18:21 | stephenfin | it's just a slightly different representation of the exact same data | |
| 15:18:30 | stephenfin | dansmith: So kill the YAML idea entirely? | |
| 15:19:39 | stephenfin | If so, the main issue I'd see with that is that they'd have to write and install a Python package for the custom stuff | |
| 15:19:49 | dansmith | stephenfin: I don't think our validations should be in yaml, and I don't really see why custom ones should be either.. they'd be a lot harder to test than a single python function you can run with a string you would put in your flavor | |
| 15:20:07 | bauzas | stephenfin: I don't see a problem here, we asked for that since a while | |
| 15:20:25 | efried | If we do that, we can't have strict validation. Writing python code and figuring out stevedore for snowflake extra specs is too high a barrier to entry for that to be the only option. | |
| 15:20:31 | dansmith | stephenfin: I think that would be a pretty simple example thing, they have to do that for scheduler filters, and I expect anyone that feels they need this (honestly I don't see it being that important) will be fine with that standard | |
| 15:20:36 | bauzas | (like, Blazar was having a specific nova-ish package for nova plugins he needed) | |
| 15:21:06 | bauzas | yeah, it's very similar to the scheduler filters case | |
| 15:21:06 | stephenfin | I was about to ask what the story with extra scheduler filters was. Those are a package too so | |
| 15:21:13 | bauzas | yup | |
| 15:21:22 | bauzas | or delivered by other means | |
| 15:21:27 | dansmith | efried: stevedore is on the loader side, they don't have to do anything like that.. entry points are a stock python thing | |
| 15:21:27 | stephenfin | efried: Are snowflake extra specs a real thing people use? | |
| 15:21:50 | stephenfin | Far as I knew, extra specs were for scheduler filters and virt drivers | |
| 15:21:53 | sean-k-mooney | they use custom one for filter | |
| 15:21:57 | efried | f man, even if you managed to miss one of the stock ones | |
| 15:22:00 | dansmith | stephenfin: you can in conjunction with the json filter or yeah a custom scheduler filter | |
| 15:22:19 | dansmith | stephenfin: if you're writing a custom scheduler flter, you're doing a package, so throwing a validator in there makes plenty of sense | |
| 15:22:29 | stephenfin | If it's a custom filter, we're saying they're using a package already so no big deal | |
| 15:22:31 | stephenfin | dansmith: yeah | |
| 15:22:37 | gibi | my employer's downstream product has at least two snowflake extra spec | |
| 15:22:48 | gibi | but sure I can implement a python filter for that | |
| 15:22:49 | sean-k-mooney | same with a thrid party virt dirver | |
| 15:22:50 | dansmith | stephenfin: imagine the fame and fortune if you wrote up a nice blog post on how to do this super cleanly :D | |
| 15:23:00 | dansmith | sean-k-mooney: shut yo' mouth! :D | |
| 15:23:11 | sean-k-mooney | :) | |
| 15:23:31 | stephenfin | If I can do one thing well, it's write lots of docs and bug people continuously to review them | |
| 15:23:39 | bauzas | scheduler filters are the exact reasons why we never constrained extra specs | |
| 15:23:42 | stephenfin | no issues there | |
| 15:24:06 | bauzas | so I guess 3rd-parties would *love* to consider filters and validation code to be considered equal | |
| 15:24:10 | sean-k-mooney | so that bring up stict vs permissive vs off | |
| 15:24:17 | bauzas | so they could package this at once | |
| 15:24:23 | stephenfin | efried: I don't think that's as big a deal as you think. Realistically, how often do people create new flavors | |
| 15:24:30 | sean-k-mooney | we could skip all non namespaced extra space or ones in a namespace we dont know about | |
| 15:24:37 | sean-k-mooney | in permisive mode at least | |
| 15:24:52 | stephenfin | They try create one, we fail because we don't recognize the spec, they report and bug and we add/backport the missing validator | |
| 15:24:59 | stephenfin | It's just a bug | |
| 15:25:21 | stephenfin | *report a bug | |
| 15:25:47 | dansmith | are you going to make admins able to create flavors that fail validation | |
| 15:25:48 | dansmith | ? | |
| 15:25:58 | stephenfin | dansmith: that's the question | |
| 15:26:11 | stephenfin | I want a microversion that starts failing extra specs that don't pass validation | |
| 15:26:19 | dansmith | if there is a &force=yes then that gives them a way past the validation if there's a bug | |
| 15:26:57 | sean-k-mooney | dansmith: that is what is was kind of suggestin with &validate=strict|permissive|off | |
| 15:27:00 | stephenfin | and I'm arguing that ^ is unnecessary since we should be able to audit the extra specs we have in-tree | |
| 15:27:14 | dansmith | sean-k-mooney: oh per-create, I thought you meant global config | |
| 15:27:26 | sean-k-mooney | no per request | |
| 15:27:40 | stephenfin | efried suggested global config. I said no cos config-driven API behavior is bad | |
| 15:27:42 | dansmith | stephenfin: yeah, I dunno.. I think some of them are complex enough that you might not be able to do that as well as you think | |
| 15:27:48 | dansmith | stephenfin: agree | |
| 15:28:31 | dansmith | stephenfin: but I think it's legit to have a validation=no per-request, but that can return 403 either because of policy or config if it's documented when you add it, no? so later we could turn that off or default it off | |
| 15:29:13 | stephenfin | hmm, that wouldn't be so bad | |
| 15:29:27 | dansmith | either way, I just think you might not want to assume you can enumerate every crazy way people have abused those extra_specs and having a bunch of angry people calling your home :D | |
| 15:29:45 | sean-k-mooney | the reason we would wont that is if we add this in 2.68 and in 2.69 we add another field to the flavor but we still want to disable validtion it would be nice to be able to turn it off | |
| 15:30:18 | bauzas | honestly, leave them time to change their flavors | |
| 15:30:21 | stephenfin | sean-k-mooney: yeah, I've been suggesting using the older microversion purely as a stop gap to allow us iron out the kinks | |
| 15:30:31 | dansmith | IMHO, and AFAICT, validation is *just* to help make the admin's life easier so they don't have to keep booting instances to test that they completely broke an extra_spec right? | |
| 15:30:42 | bauzas | also, are we proposing to fix existing embedded flavors ? | |
| 15:30:45 | dansmith | so being able to turn it off in an emergency doesn't seem like a bad thing to me | |
| 15:30:45 | sean-k-mooney | but that would not work if the thing i wanted to set was added in 2.69 | |
| 15:30:54 | stephenfin | This also won't break existing flavors. It'll only prevent them creating new broken flavors | |
| 15:30:55 | dansmith | you're not going to periodically fsck your flavors so, what does it matter? | |
| 15:30:58 | bauzas | stephenfin: I mean, instance's embedded flavors | |
| 15:31:30 | sean-k-mooney | stephenfin: creating or updating | |
| 15:31:40 | stephenfin | bauzas: Nope. Out of scope for this. This focuses purely on setting new flavor extra specs | |