| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-04-28 | |||
| 15:50:11 | bauzas | and gosh saves the rest | |
| 15:50:33 | bauzas | gmann: ^ | |
| 15:51:12 | jsuchome | efried, bauzas, dansmith: Hi, could we please get https://review.opendev.org/#/c/572805/ reopen _again_ ? | |
| 15:51:16 | gmann | how we will differentiate the upgrade with re-generated file vs new deployment/upgrade moving to new system | |
| 15:51:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add online migration for legacy NUMA objects https://review.opendev.org/537414 | |
| 15:51:38 | bauzas | gmann: greenfields don't run the nova-status check | |
| 15:52:12 | dansmith | ah, yeah I guess gmann has a point | |
| 15:52:34 | bauzas | make it a flag (c) | |
| 15:52:35 | efried | gmann: restored | |
| 15:53:02 | dansmith | jsuchome: are you going to work on it yourself? | |
| 15:53:50 | jsuchome | unless tobiash has time to pick it up ... | |
| 15:53:51 | stephenfin | artom: I switched that to startswith which is as optimal as I can get, aside from enumerating every possible legacy NUMA topology configuration :) | |
| 15:54:01 | artom | stephenfin, hehe | |
| 15:54:23 | gmann | I still feel (dansmith might be angry on me saying this again and again ) this usage of re-generated file is wrong thought common and telling them to use it in right way (with consistent guide on oslo side) can correct the things for long term too. | |
| 15:54:41 | gmann | *though | |
| 15:54:53 | dansmith | gmann: the right way is to switch to yaml, fully commented :) | |
| 15:55:57 | bauzas | gmann: the ship has sailed. | |
| 15:56:03 | dansmith | bauzas: agree :) | |
| 15:56:38 | gmann | yeah, yaml and they can un-comment the rule with new values if they want. Honestly satying, that the way i was thinking people using policy file after policy-in-code | |
| 15:56:43 | gmann | humm | |
| 15:56:50 | bauzas | gmann: can you please clarify the problem you have for distinguishing the different upgrade cases ? | |
| 15:57:06 | bauzas | I see three cases | |
| 15:57:15 | bauzas | 1/ Ussuri greenfields | |
| 15:57:29 | jsuchome | dansmith: so last time it was tobiash, let's wait if we wants to continue and if not, I would start myself | |
| 15:57:32 | bauzas | 2/ T->U and no policy.json changes | |
| 15:57:45 | bauzas | 3/ T->U and specific policy.json | |
| 15:57:49 | artom | stephenfin, yeah, I left a follow-up comment. I guess I just don't know enough in this case, and what I've been able to learn isn't enough | |
| 15:57:52 | bauzas | gmann: amirite ? | |
| 15:57:58 | dansmith | jsuchome: okay | |
| 15:57:58 | gmann | bauzas: policy file with new defaults only (no deprecated old default) cover two case 1. this broken case 2. people move to new defaults intentionally. | |
| 15:58:23 | gmann | bauzas: 3rd one has ^^ above cases | |
| 15:58:27 | bauzas | gmann: sure, but that's not the case you wanna warn, right? | |
| 15:58:40 | tobiash | jsuchome: feel free to take it, atm I unfortunately I don't have time to work on it | |
| 15:58:41 | stephenfin | zzzeek: I know you're busy, but any chance you'd be able to shine a light on https://review.opendev.org/#/c/537414/ at some point? | |
| 15:58:46 | stephenfin | zzzeek: please and thank you :) | |
| 15:59:32 | gmann | bauzas: yeah, so we just say in nova-status what we are saying in reno ? | |
| 15:59:45 | gmann | i mean new reno - https://review.opendev.org/#/c/723645/ | |
| 16:00:33 | bauzas | gmann: but basically, yeah, code in python what you write in restructuredtext | |
| 16:00:59 | bauzas | because parsers aren't good with english syntax | |
| 16:01:06 | zzzeek | stephenfin: whats the quesiton, is contains() or startswith() faster? | |
| 16:01:27 | stephenfin | zzzeek: Yes, if there isn't an index on the column | |
| 16:01:48 | zzzeek | stephenfin: both contains() / startswith() are based on LIKE which is going to need a table scan | |
| 16:02:00 | artom | zzzeek, and more generally, if we're stuck doing text filtering on a column without an index, what would be the lest sucky way? | |
| 16:02:06 | artom | *least | |
| 16:02:18 | bauzas | gmann: I briefly looked at the note | |
| 16:02:25 | zzzeek | I don't see the db/model files here that we're talking about I just see objects files, is there a SQL query in those ? | |
| 16:02:42 | zzzeek | and is the issue that you don't wnat to add a new column with the indexed information you need ? | |
| 16:02:43 | bauzas | gmann: and unless I'm wrong, I don't see a technical problem for providing a nova-status check command verifying this | |
| 16:02:53 | stephenfin | zzzeek: sec, lemme drag up the model | |
| 16:03:04 | zzzeek | oh i see it | |
| 16:03:12 | zzzeek | .filter(~models.ComputeNode.numa_topology.startswith( | |
| 16:03:12 | zzzeek | compute_nodes = (context.session.query(models.ComputeNode) | |
| 16:03:12 | zzzeek | '{"nova_object.name"')) | |
| 16:03:12 | zzzeek | .filter(models.ComputeNode.numa_topology != null()) | |
| 16:03:16 | zzzeek | that ? yeah that's not great :) | |
| 16:03:22 | zzzeek | depends on number of rows | |
| 16:03:25 | zzzeek | < 1000, no problem | |
| 16:03:27 | bauzas | gmann: ie. look at the file, and if you find both old-style and new-style, yell it | |
| 16:03:30 | artom | zzzeek, well, all instances | |
| 16:03:44 | artom | So multiple thousands, maybe into the millions for large deployments | |
| 16:03:46 | zzzeek | including old ones that are offline ? | |
| 16:03:53 | zzzeek | artom: yeah that's not going to be fast | |
| 16:03:54 | stephenfin | artom: not deleted instances | |
| 16:04:06 | stephenfin | tbc | |
| 16:04:10 | zzzeek | you want to get the # of rows as low as possible before applying a like on it | |
| 16:04:14 | artom | stephenfin, does that filtering happen before or after the string filter? | |
| 16:04:31 | gmann | bauzas: ok, let me add in nova-status also | |
| 16:04:36 | bauzas | gmann: if you wanna some examples, go over there https://github.com/openstack/nova/blob/master/nova/cmd/status.py#L364 | |
| 16:04:38 | zzzeek | there are other ways to do this, mysql/mariadb might ahve a MATCH operator but that requires changes | |
| 16:04:51 | stephenfin | I actually don't know /o\ Does filter ordering matter? | |
| 16:05:16 | artom | zzzeek, yeah, IIUC MATCH needs full text index, no? | |
| 16:05:42 | zzzeek | but ideally you'd have the information you need captured in some dedicated column, or if for example the "numa_topology" were a forieng key to some collection table, you could limit the rows on that side to a list of posible foreign key ids, that kind of thing | |
| 16:05:56 | zzzeek | artom: yes | |
| 16:06:24 | gmann | bauzas: sure, thanks | |
| 16:06:24 | bauzas | anyway, /me calls it a day | |
| 16:07:08 | stephenfin | zzzeek: Unfortunately this is a JSON blob rather than an actual table. No foreign keys here | |
| 16:07:19 | zzzeek | so as far as LIKE '%foo%' vs. LIKE 'foo%', I don't know the details of what mysql/mariadb would do, we would imagine the second is "faster" but i dont know that it can use indexes for that | |
| 16:07:22 | stephenfin | and we're trying to move from one format of JSON blob to another | |
| 16:07:27 | artom | zzzeek, we can't play around with the schema for this, we need to work with the columns/indeces that we have | |
| 16:07:43 | artom | Or don't have :( | |
| 16:08:03 | zzzeek | stephenfin: Mysql /mariadb have JSON operators now, do those apply here? | |
| 16:08:37 | stephenfin | do those apply if the column type is just TEXT? | |
| 16:08:37 | zzzeek | e.g. you can extract values from a json structure | |
| 16:08:57 | zzzeek | stephenfin: yes there is no JSON type on both backends, there's ...some kind of keyword on one of htem | |
| 16:09:22 | dansmith | zzzeek: is that really faster than a string comparison? I would expect that would require reading the whole text, more cpu to parse, and then run the comparison | |
| 16:09:30 | zzzeek | mysql has a native JSON mariasdb does not but the json operators should work on either | |
| 16:09:41 | zzzeek | dansmith: I dunno, would need to read some docs | |
| 16:09:49 | dansmith | I would be absolutely blown away if so :) | |
| 16:10:04 | zzzeek | https://mariadb.com/kb/en/json_value/ | |
| 16:10:14 | artom | stephenfin, I don't support looking at updated_date is enough? | |
| 16:10:14 | zzzeek | dansmith: Postgresql JSON type probably builds some index | |
| 16:10:18 | artom | *suppose | |
| 16:10:19 | zzzeek | Mysqls' might as well | |
| 16:10:20 | artom | Or can help in any way? | |
| 16:10:22 | zzzeek | mariadb, not too sure | |
| 16:10:24 | dansmith | zzzeek: sure, if the column type is json then maybe | |
| 16:10:40 | stephenfin | artom: I considered that, but I don't know when someone upgraded | |
| 16:11:49 | stephenfin | so if ~in theory~ someone was on Juno for ages - say, until 2016, and eventually decided to upgrade, and are now upgrading that same cloud to Victoria | |
| 16:11:59 | artom | stephenfin, actually - what about a two step approach? 1. the update on read thing we talked about earlier 2. let it sit for a cycle or two, then do a query with udpated_at earlier than time spent sitting | |
| 16:12:07 | zzzeek | dansmith: oh here's an intersting approahc, uisng a generated column to index parts of the json document when they are inserted | |