| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-04-28 | |||
| 16:10:19 | zzzeek | Mysqls' might as well | |
| 16:10:20 | artom | Or can help in any way? | |
| 16:10:22 | zzzeek | mariadb, not too sure | |
| 16:10:24 | dansmith | zzzeek: sure, if the column type is json then maybe | |
| 16:10:40 | stephenfin | artom: I considered that, but I don't know when someone upgraded | |
| 16:11:49 | stephenfin | so if ~in theory~ someone was on Juno for ages - say, until 2016, and eventually decided to upgrade, and are now upgrading that same cloud to Victoria | |
| 16:11:59 | artom | stephenfin, actually - what about a two step approach? 1. the update on read thing we talked about earlier 2. let it sit for a cycle or two, then do a query with udpated_at earlier than time spent sitting | |
| 16:12:07 | zzzeek | dansmith: oh here's an intersting approahc, uisng a generated column to index parts of the json document when they are inserted | |
| 16:12:11 | zzzeek | https://www.compose.com/articles/mysql-for-json-generated-columns-and-indexing/ | |
| 16:12:23 | dansmith | zzzeek: that doesn't help this case | |
| 16:12:25 | stephenfin | and are moving the instance for the first time since before that upgrade off of Juno :) | |
| 16:12:33 | zzzeek | dansmith: brcause....table cannot be changed, right? | |
| 16:12:44 | dansmith | zzzeek: we're specifically looking at old data yeahg | |
| 16:12:58 | zzzeek | dansmith: old data can be copied into...new tables and columns? | |
| 16:13:11 | dansmith | zzzeek: no, we might as well convert it if we're going to read it | |
| 16:13:37 | zzzeek | dansmith: yup. if you have a big old JSON blob and you want to pull things out of it and performance is an issue then you would need to put this data into some indexable format somewhere else | |
| 16:14:01 | zzzeek | stephenfin: ^^^ | |
| 16:14:58 | dansmith | artom: updated_at is on the instance_extra row, right? so it doesn't really mean we've updated any one or all columns necessarily | |
| 16:15:05 | stephenfin | zzzeek: okay, thanks for the input :) | |
| 16:16:01 | artom | dansmith, hrmm, yeah - I guess I was assuming "update on read" would trigger for any read of any column of that row | |
| 16:16:19 | artom | Not just instance_extra.numa_topology reads | |
| 16:16:39 | dansmith | instance_extra was really supposed to be individually queried and updated, although I dunno how much that really happens.. it was mostly for lazy-loadable blobby things originally | |
| 16:17:26 | artom | Is this is the first time we're hitting this problem? | |
| 16:17:59 | artom | Apparently so - at least using .contains() | |
| 16:29:50 | zigo | If you guys think we don't have enough time before the final release (which I can agree on), why not just delay this for 21.0.1 or something? | |
| 16:30:25 | zigo | My biggest concern is not being able to actually "see" what's set by what we have in defaults. | |
| 16:30:34 | zigo | Yes, it's documented, but it's not the same. | |
| 16:30:43 | zigo | I'll try using policy.yaml though ... :P | |
| 16:30:56 | dansmith | zigo: delay "this" meaning the policy stuff in general? | |
| 16:31:03 | zigo | Yeah. | |
| 16:31:37 | dansmith | zigo: it was a large number of patches.. reverting it now would be .. huge. | |
| 16:32:31 | zigo | dansmith: I thought it'd be just a simple patch to the oslo-policy-sample-generator ... :P | |
| 16:32:36 | zigo | Maybe I'm being naive. | |
| 16:35:00 | dansmith | zigo: the change in question was on the nova side, not oslo.. we'd need new code on the oslo side to allow generating the policy file with the now-deprecated original defaults | |
| 16:37:01 | zigo | Now I see what you guys were talking about for the yaml thing: it's by default generated with everything commented out ... | |
| 16:37:11 | zigo | (I just tried...) | |
| 16:37:38 | zigo | That's probably nicer indeed, and probably good enough for operators, and also maybe more easy to have a policy.d | |
| 16:37:50 | zigo | Though the last time I tried, it wouldn't load ... :/ | |
| 16:41:16 | zigo | Looks like it works as one would expect... :) | |
| 16:41:45 | jsuchome | dansmith: ok, so it seems it's up to me, so if you could reopen and reassign it ... thanks | |
| 16:42:04 | dansmith | jsuchome: all you need to do is propose the changes yourself | |
| 16:43:00 | jsuchome | so should I just cherry pick to my branch? | |
| 16:44:21 | zigo | All this would be great if the Debian infrastructure wasn't completely down today ... :/ | |
| 16:44:45 | dansmith | jsuchome: just grab the latest version of that patch into your tree, make changes, git review. | |
| 17:33:28 | artom | dansmith, answers provided: https://review.opendev.org/#/c/672595/73 | |
| 17:33:34 | artom | (Hopefully) | |
| 17:33:43 | gmann | zigo: nice. +1 thanks for checking. | |
| 17:43:57 | artom | \o/ | |
| 17:44:14 | artom | I have a 1:1 in 15 minutes, otherwise it'd be beer time | |
| 17:53:28 | gmann | zigo: and you generated with same tool right ? | |
| 17:54:21 | zigo | gmann: Yeah, just --format yaml. Now, I'll have to set policy.yaml as default in nova.conf | |
| 17:54:51 | zigo | Though as I wrote earlier, Debian @UBC is down, so can't do anything right now ... :( | |
| 17:55:12 | zigo | No Git to play with. | |
| 17:55:31 | gmann | +1 | |
| 17:57:15 | zigo | Also, this clashes with puppet-openstack which uses .json. | |
| 17:57:26 | zigo | It's going to be fun time to fix too ... | |
| 18:08:11 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Clarify the policy new defaults upgrade notes https://review.opendev.org/723645 | |
| 18:11:12 | openstackgerrit | Merged openstack/nova master: Remove Babel requirement https://review.opendev.org/720725 | |
| 18:11:20 | openstackgerrit | Merged openstack/nova master: Remove translation sections from setup.cfg https://review.opendev.org/723206 | |
| 18:57:05 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: DNM: Partial cherry-pick of report client changes https://review.opendev.org/723750 | |
| 18:57:06 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: DNM: Add a placement audit command https://review.opendev.org/723751 | |
| 20:31:32 | melwitt | gmann: I thought you wanted to wait for these first? https://review.opendev.org/#/q/topic:qa-ussuri-release+status:open | |
| 20:32:21 | gmann | melwitt: devstack and grenade setup mainly which are merged. devstack-gate changes is for legacy jobs which this patch replacing | |
| 20:32:33 | melwitt | oh ok | |
| 20:39:07 | sean-k-mooney | am https://bugs.launchpad.net/nova/+bug/1875418 have we ever support a policy.yaml? | |
| 20:39:07 | openstack | Launchpad bug 1875418 in OpenStack Compute (nova) "Generated policy.json in Ussuri is broken by default" [High,In progress] - Assigned to Ghanshyam Mann (ghanshyammann) | |
| 20:39:40 | sean-k-mooney | i tought we only supported policy files in json format | |
| 20:41:18 | sean-k-mooney | i ask because teh " [puppet][packaging] Switching to policy.yaml (over policy.json)" thread seams to imply that is a thing | |
| 20:41:36 | sean-k-mooney | but i did not think we supported a policy.yaml file instead of policy.json | |
| 20:42:47 | sean-k-mooney | huh i guess its a thing https://docs.openstack.org/oslo.policy/latest/admin/policy-yaml-file.html | |
| 20:43:52 | bnemec | We've been recommending yaml since policy in code happened. | |
| 20:44:02 | bnemec | Apparently we need to work on our PR though. :-) | |
| 20:45:07 | gmann | yeah, and not much attention on that and this issue came up now | |
| 20:45:08 | sean-k-mooney | ya i mean yaml is probably nice to work with in terms of generating and reading the file and form a python point of view it makes little difference as we will jsut load it into a python dict in etierh case then process it | |
| 20:45:30 | sean-k-mooney | but i never heard that we added supprot for parseing yaml files instead | |
| 20:46:16 | sean-k-mooney | bnemec: it might be something that woudl be worth make a comuntiy goal | |
| 20:46:32 | sean-k-mooney | to get everyone to move to policy.yaml espeically the deployment tools | |
| 20:46:38 | openstackgerrit | Merged openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595 | |
| 20:46:39 | gmann | it is supported in oslo side right | |
| 20:46:47 | bnemec | sean-k-mooney: It was: https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.html | |
| 20:46:48 | sean-k-mooney | although i guest they should not be setting one by default | |
| 20:46:59 | bnemec | Oh, deployment tools. | |
| 20:47:01 | sean-k-mooney | bnemec: well plocy in code was | |
| 20:47:44 | sean-k-mooney | bnemec: i was more thinking of makeing sure our docs advise to use policy.yaml files | |
| 20:48:12 | sean-k-mooney | and any deployment tools that currently supprot customising policy.json use policy.yaml | |
| 20:48:13 | bnemec | They do: https://docs.openstack.org/oslo.policy/latest/admin/policy-json-file.html ;-) | |
| 20:48:23 | bnemec | But yeah, clearly we need to push it harder. | |
| 20:48:32 | sean-k-mooney | bnemec: do the nova docs? | |
| 20:49:13 | gmann | bnemec: and that happen when things get broken otherwise its less chance people read doc or be in sync on new things | |
| 20:49:25 | bnemec | That I don't know. The only in-project policy doc I'm aware of is the policy sphinx plugin. | |
| 20:49:33 | sean-k-mooney | https://docs.openstack.org/nova/rocky/configuration/policy.html i gues it just list the polices | |
| 20:49:55 | sean-k-mooney | and the sample kind of looks like yaml https://docs.openstack.org/nova/rocky/configuration/sample-policy.html | |
| 20:50:15 | gmann | sean-k-mooney: the file form mentioned is yaml one -https://docs.openstack.org/nova/latest/configuration/sample-policy.html | |
| 20:50:19 | sean-k-mooney | ok it is a yaml file | |
| 20:50:34 | bnemec | Yep, that's the yaml sample policy output. | |
| 20:50:34 | sean-k-mooney | ya you can download it https://docs.openstack.org/nova/rocky/_downloads/nova.policy.yaml.sample | |
| 20:50:59 | sean-k-mooney | ok im just used to people saying policy.json the whole time | |
| 20:52:03 | sean-k-mooney | we still refer to it as policy.json here https://docs.openstack.org/nova/rocky/configuration/ | |
| 20:54:48 | bnemec | Yeah, and unfortunately policy.json is still the default name in oslo.policy. We've had discussions about making it look for both json and yaml, but it has security implications if we guess wrong. | |
| 20:55:48 | sean-k-mooney | so does it only look for one of them | |