| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-29 | |||
| 15:02:31 | efried | conceived, if I'm not mistaken, before yaml was sexy. | |
| 15:02:57 | efried | It would be possible to do these validators ini-style and use oslo.conf | |
| 15:03:02 | sean-k-mooney | gibi: so i really hate the idea of only doing this with yaml files but if we do then i think there is no excust to not use the json based metadef format that glance has and share the files as a lib | |
| 15:03:26 | stephenfin | eeew, I'm not writing that much JSON | |
| 15:03:41 | efried | Yeah, definitely don't want the files to be json. yaml exists because json is ugly. | |
| 15:03:51 | stephenfin | also, I think I did look at that and it didn't offer everything we'd need | |
| 15:04:00 | sean-k-mooney | but we are needless reinventing the wheel here | |
| 15:04:08 | gibi | I don't want to start a format war, I just wanted to avoid implementing two pieces of code when one piece would be enough technically | |
| 15:04:20 | stephenfin | But glance's stuff wasn't invented here, Sean. | |
| 15:04:27 | stephenfin | :P | |
| 15:04:29 | sean-k-mooney | the have a format for declaring this. they expose it via an api for heat and horizon to consume to generate use | |
| 15:04:31 | bauzas | honestly, what's the problem with stevedore ? | |
| 15:04:38 | efried | Oh, yeah, IMO ini isn't powerful enough for long-term use, even for nova.conf (as demonstrated by the travesty that is passthrough_whitelist). | |
| 15:04:40 | bauzas | nova was built around it | |
| 15:04:48 | bauzas | plugins FTW | |
| 15:04:55 | sean-k-mooney | we could pull the galnce sutff into a lib and share it | |
| 15:05:01 | efried | bauzas: to me, just unnecessary complexity and an additional thing we don't need. | |
| 15:05:10 | sean-k-mooney | if we go file based only i think that is what we shoudl do | |
| 15:05:25 | bauzas | efried: entrypoints are complex to manage ? | |
| 15:05:44 | bauzas | honestly, we haven't heard about this since nova exists | |
| 15:05:59 | efried | entrypoints plus python code plus packaging, versus putting a yaml file in a directory? Hell yes. | |
| 15:06:01 | sean-k-mooney | bauzas: there is one complexity with them form a packaging point of view | |
| 15:06:14 | stephenfin | sean-k-mooney: so we'd be using a package to distribute a YAML file? | |
| 15:06:35 | efried | ugh, no, please. | |
| 15:06:38 | bauzas | it's config | |
| 15:06:39 | efried | Can we not boil the ocean? | |
| 15:06:40 | sean-k-mooney | json file and the code to do all the validation which they have laready written as far as i knwo | |
| 15:07:15 | stephenfin | I'd personally rather strip the flavor stuff out of glance | |
| 15:07:24 | stephenfin | Why do they need to care about flavors? They're an image service | |
| 15:07:53 | efried | how did glance get involved here? | |
| 15:07:55 | sean-k-mooney | they care about it becasue we had this problem years ago and decied that they would be used as the catalog of support meatdata itmes | |
| 15:08:05 | bauzas | I thought the spec was simple enough to just re-accept it | |
| 15:08:09 | efried | (rhetorical question; can we stay on topic?) | |
| 15:08:12 | bauzas | given it was accepted beofre | |
| 15:08:29 | bauzas | but now, looks like we rathole around any possible better way to provide config | |
| 15:09:09 | stephenfin | so I really don't want another os-whatever library to maintain. This should be in-tree, with some kind of mechanism to export to glance if we really care | |
| 15:09:28 | stephenfin | Which I'd be surprised if we do, since sean-k-mooney is the only person that's ever heard of this feature in glance | |
| 15:09:30 | bauzas | that's exactly why we chose plugins before | |
| 15:09:38 | bauzas | but... whatever | |
| 15:09:54 | bauzas | looks like we have consensus around YAML | |
| 15:10:14 | bauzas | the glance issue shouldn't be taken care as of now, until we know whether glance needs it as well or not | |
| 15:10:19 | stephenfin | so the question comes down to Python objects or YAML, and whether we need to make the strict behavior opt-in or not | |
| 15:10:19 | sean-k-mooney | well im going to be out voted so i guess yes | |
| 15:10:35 | dansmith | yaml for what? rules of extra spec validation? | |
| 15:10:43 | stephenfin | yup | |
| 15:10:50 | dansmith | and... why? | |
| 15:10:53 | gibi | I think having a simple api microversion to opt in is enough | |
| 15:11:01 | dansmith | because we can share that file between us and glance? | |
| 15:11:05 | efried | no | |
| 15:11:22 | efried | so we can accommodate extra specs nova doesn't own or know about | |
| 15:11:31 | gibi | dansmith: because we need the yaml for deployer defined extra specs anyhow | |
| 15:11:38 | stephenfin | so we don't have two different ways to represent a rule | |
| 15:11:38 | stephenfin | https://review.opendev.org/#/c/704643/2/nova/api/validation/extra_specs/hw.py | |
| 15:11:41 | bauzas | honestly, I was preferring the python approach thru stevedore plugins | |
| 15:11:45 | stephenfin | we want to provide a way for operators to specify their own custom extra specs for out of tree filters, or whatever | |
| 15:11:49 | sean-k-mooney | dansmith: gibi does not like the python class way of doing it and suggeste that if we are going to support yaml anyway just use it for everything. fair summary? | |
| 15:12:00 | stephenfin | sean-k-mooney: yup, that | |
| 15:12:11 | gibi | I don't want to start a format war, I just wanted to avoid implementing two pieces of code when one piece would be enough technically | |
| 15:12:17 | dansmith | so, this brings the policy problem screaming back from hell? | |
| 15:12:38 | stephenfin | oslo.policy? | |
| 15:12:44 | sean-k-mooney | dansmith: no that was brogh back for a different reason | |
| 15:12:48 | dansmith | i.e. we provide a base yaml, they edit it to add their stuff and tweak their prefs and then when they go to upgrade they're sadface? | |
| 15:13:12 | stephenfin | that was another concern of mine, but I figured we'd just load multiple files | |
| 15:13:17 | sean-k-mooney | e.g. if the new microversion enabled validateion and we extend the flavor update again in the future you cant use the new feautre with validation idsabled | |
| 15:13:30 | stephenfin | I'm almost certain someone will go tweak the YAML file we provide though | |
| 15:13:39 | bauzas | yuuuuup | |
| 15:13:46 | dansmith | stephenfin: I would argue that we don't/shouldn't encode our baseline validations into yaml and distribute them that way, as it's just asking for that problem | |
| 15:13:49 | dansmith | right | |
| 15:13:52 | sean-k-mooney | dansmith: so i was suggesting add a &validate query arge to turn off validation in the new microversion | |
| 15:13:53 | efried | If we do in-tree stuff with yaml, we put the nova-owned stuff within the nova package so nobody fs with it. | |
| 15:14:06 | dansmith | efried: riiiight | |
| 15:14:07 | stephenfin | sean-k-mooney: hold up, let's figure the YAML thing out first | |
| 15:14:38 | efried | riiiight they'll still f with it? Let 'em. That's breaking the seal, voiding the warranty, not different than if they go patch the code. | |
| 15:14:46 | sean-k-mooney | efried: distos will modify it every time we backport stuff | |
| 15:14:51 | gibi | we somebody want to tweak what is written in the python DSL for in-tree validation he can do it, it is almos like a yaml file | |
| 15:14:52 | dansmith | surely the yaml to validate something complex like numa whatever is going to be nasty or naive right? | |
| 15:15:12 | sean-k-mooney | it will be an interesting regex | |
| 15:15:23 | gibi | but nothing more than a regex | |
| 15:15:26 | stephenfin | gibi: I don't think it's a DSL, really. It's just an object. I could use dicts if that would be preferable | |
| 15:15:28 | dansmith | an "interesting" regex or some relatively simple python right? | |
| 15:15:42 | bauzas | :) | |
| 15:15:43 | sean-k-mooney | actully its still a regex in stephns code | |
| 15:15:49 | sean-k-mooney | well poc | |
| 15:15:56 | stephenfin | not entirely | |
| 15:15:57 | sean-k-mooney | but it could be simpler | |
| 15:16:11 | stephenfin | I let you specify enums etc. | |
| 15:16:19 | sean-k-mooney | yes | |
| 15:16:37 | stephenfin | granted, all that gets converted into a regex but there's no reason I have to do that for all cases. It's a PoC after all | |
| 15:16:43 | sean-k-mooney | although you convert the enum int an bool filed back to regex form | |
| 15:17:25 | dansmith | so I think my feeling on this is: (a) I'm not sure we need an interface for people to be able to specify their own validators, but (b) if we do, that seems like an obvious use case for entrypoints and a dead simple plugin interface with a super-controlled input and output behavior set | |
| 15:17:51 | efried | gibi: it won't be a lot of duplication, really. The parser will construct the same kind of python objects we have in tree and then feed all of them into the validation engine. | |
| 15:18:04 | stephenfin | gibi: yeah, what efried said | |
| 15:18:19 | gibi | OK, I rest my case then | |
| 15:18:21 | stephenfin | it's just a slightly different representation of the exact same data | |
| 15:18:30 | stephenfin | dansmith: So kill the YAML idea entirely? | |
| 15:19:39 | stephenfin | If so, the main issue I'd see with that is that they'd have to write and install a Python package for the custom stuff | |
| 15:19:49 | dansmith | stephenfin: I don't think our validations should be in yaml, and I don't really see why custom ones should be either.. they'd be a lot harder to test than a single python function you can run with a string you would put in your flavor | |
| 15:20:07 | bauzas | stephenfin: I don't see a problem here, we asked for that since a while | |
| 15:20:25 | efried | If we do that, we can't have strict validation. Writing python code and figuring out stevedore for snowflake extra specs is too high a barrier to entry for that to be the only option. | |
| 15:20:31 | dansmith | stephenfin: I think that would be a pretty simple example thing, they have to do that for scheduler filters, and I expect anyone that feels they need this (honestly I don't see it being that important) will be fine with that standard | |