| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-12-14 | |||
| 16:21:39 | dansmith | jaypipes: like whether 1 byte in a MEDIUMTEXT takes up 16mb on disk | |
| 16:22:04 | jaypipes | dansmith: there's almost no difference between a TEXT, MEDIUMTEXT or VARCHAR(8000) w.r.t. on-disk storage requirements or layout. | |
| 16:22:11 | jaypipes | dansmith: for InnoDB that is. | |
| 16:22:12 | bauzas | dansmith: FWIW, I fixed the problem way in the past for hosts https://github.com/openstack/nova/commit/c48c1098cbcd7a9c7980d5fbe4668c38b16767f5 | |
| 16:22:31 | dansmith | jaypipes: okay | |
| 16:22:44 | bauzas | dansmith: but I didn't thought it would be a huge problem for members, given not a lot of operators use a long list of them | |
| 16:23:04 | jaypipes | dansmith: no, for all those field types, InnoDB stores a small amount of data in the data page itself and makes room (when needed) in separate extents by providing a pointer to that extent/page from the main data page. | |
| 16:23:43 | jaypipes | dansmith: the only thing that can be performance-wise a bad thing is if the size of those fields changes often (which isn't the case for these fields in Nova's DB) | |
| 16:23:44 | artom | So for MySQL/InnoDB a migration to mediumtext would actually make sense... | |
| 16:23:55 | jaypipes | artom: for InnoDB, yes. for NDB, no... | |
| 16:24:07 | jaypipes | artom: but that's a different problem entirely :) | |
| 16:24:24 | artom | Do we support NDB? | |
| 16:24:40 | jaypipes | artom: what is the current column type for this field you're talking about? | |
| 16:24:48 | bauzas | mriedem: jaypipes: https://review.openstack.org/#/c/527799/ looks legit to me | |
| 16:24:52 | dansmith | jaypipes: and what is the cost of resizing to medium from text? | |
| 16:25:01 | jaypipes | dansmith: virtually zero. | |
| 16:25:04 | artom | jaypipes, dansmith answered that for me | |
| 16:25:12 | dansmith | jaypipes: awesome, then we should do that too | |
| 16:25:19 | dansmith | artom, mriedem ^ | |
| 16:25:33 | dansmith | still need the real fix, but sounds like a backport is too good to pass up | |
| 16:25:36 | dansmith | of the migration I mean | |
| 16:25:37 | artom | dansmith, that assumes we only support InnoDB | |
| 16:25:38 | jaypipes | dansmith: I don't recommend mediumtext anyway. just set it to TEXT | |
| 16:25:46 | dansmith | jaypipes: text isn't big enough | |
| 16:25:46 | artom | Which... is it the case? | |
| 16:26:02 | artom | Surely some people are running Galera | |
| 16:26:05 | dansmith | artom: I think everyone would be running innodb in prod, AFAIK | |
| 16:26:07 | jaypipes | dansmith: TEXT > MEDIUMTEXT | |
| 16:26:21 | dansmith | jaypipes: um I don't think so | |
| 16:26:33 | jaypipes | sorry, yeah... | |
| 16:26:48 | dansmith | https://stackoverflow.com/questions/13932750/tinytext-text-mediumtext-and-longtext-maximum-storage-sizes | |
| 16:26:53 | jaypipes | was thinking LONGTEXT | |
| 16:27:00 | bauzas | yeah LONGTEXT no ? | |
| 16:27:05 | artom | Oh, Galera uses InnoDB | |
| 16:27:08 | bauzas | TEXT to LONGTEXT I mean | |
| 16:28:07 | jaypipes | dansmith, artom: what is the specific field we are talking about here? | |
| 16:28:10 | mriedem | dansmith: jaypipes: cool - thanks | |
| 16:28:13 | jaypipes | dansmith: is this metadata value? | |
| 16:28:24 | dansmith | jaypipes: request_specs.spec | |
| 16:28:29 | jaypipes | hmm. | |
| 16:28:40 | dansmith | jaypipes: it's a text we serialize the object into, and because we store instance_group.members, which can be huge, | |
| 16:28:41 | artom | Err, instance_group surely? | |
| 16:28:46 | dansmith | we can generate more than 64kb of data | |
| 16:28:59 | dansmith | artom: instance_group is in the spec | |
| 16:29:07 | bauzas | we serialize all at a whole | |
| 16:29:11 | artom | Ah, doh | |
| 16:29:15 | jaypipes | dansmith: k. then I'd recommend changing it to MEDIUMTEXT from TEXT. | |
| 16:29:23 | dansmith | jaypipes: that's what we're doing :) | |
| 16:29:34 | jaypipes | dansmith: ok, sorry for interrupting then :) | |
| 16:29:35 | dansmith | jaypipes: waaaaayyyy ahead of you bro | |
| 16:29:49 | dansmith | jaypipes: and when I say "we" I mean "artom" | |
| 16:29:52 | jaypipes | k | |
| 16:30:22 | artom | I, wha? | |
| 16:30:32 | jaypipes | yet another reason why storing denormalized data in a TEXT-y field is a bad idea... | |
| 16:30:39 | artom | I've barely had breakfast and it's 11:30, I'm not ahead of anything | |
| 16:30:47 | openstackgerrit | Merged openstack/nova master: Fix the bug report link of API Guide https://review.openstack.org/527660 | |
| 16:32:17 | dansmith | jaypipes: well, it has its place, IMHO, and the complexity required to store a reqspec normalized is not worth it since it's a snapshot | |
| 16:32:33 | bauzas | I remember we had a thought on that with alaski | |
| 16:33:18 | dansmith | reqspec is an archive snapshot really, which is fine for this, we just need to not snapshot an infinite amount of stale data into it ;) | |
| 16:33:35 | bauzas | and we said it was better to just store the whole object so we would totally decouple what we store from what we expose | |
| 16:33:48 | bauzas | dansmith: yeah, I still agree on it tbc | |
| 16:34:10 | jaypipes | dansmith: +W'd that patch | |
| 16:34:25 | bauzas | we just expose the instance UUID for querying reasons that's it | |
| 16:34:28 | dansmith | jaypipes: thanks | |
| 16:34:43 | bauzas | dansmith: is there another patch related to that about a DB schema modification ? | |
| 16:34:48 | dansmith | jaypipes: and thanks for being my fleshy mysql manual :P | |
| 16:34:59 | dansmith | bauzas: young artom will be presenting it forthwith | |
| 16:35:03 | bauzas | cool | |
| 16:35:10 | jaypipes | dansmith: sounds kinky. | |
| 16:35:19 | bauzas | oh man, I wish I could "OK jaypipes" on my phone | |
| 16:35:26 | jaypipes | lol | |
| 16:35:27 | artom | jaypipes, you got off easy, I'm his young turdgoat | |
| 16:35:35 | cdent | "fleshy mysql manual" is way too visual | |
| 16:35:40 | jaypipes | artom: yes, I read that. very flattering indeed. ;) | |
| 16:35:41 | dansmith | cdent: you're welcome | |
| 16:35:56 | cdent | clears my head nicely, thank you dansmith | |
| 16:37:42 | artom | dansmith, so I'll handle the backports - who's proposing the DB migration? | |
| 16:38:01 | dansmith | artom: you are | |
| 16:38:04 | artom | Or are we satisfied that the code alone is enough? | |
| 16:38:52 | dansmith | I mean, I can do the migration if you really don't want to, but.. figured you'd want all the glory and praise that will come with it | |
| 16:39:11 | artom | If I want glory and praise I'll become a murderous dictator | |
| 16:39:17 | mriedem | artom: there is a db migratoin for extending the size of build_requests.instance | |
| 16:39:20 | mriedem | artom: just copy ^ | |
| 16:39:21 | mriedem | super easy | |
| 16:39:26 | mriedem | exact same issue | |
| 16:39:34 | dansmith | artom: an important point on the resume of a murderous dictator is "did a schema migration once" | |
| 16:39:38 | artom | For now I just want to revel turdgoatery | |
| 16:39:42 | mriedem | ffs you should probably give me co-author now because i even told you | |
| 16:40:08 | artom | What's the magic word? | |
| 16:40:27 | mriedem | trump | |
| 16:41:52 | artom | Do we support pgsql as well? | |
| 16:42:05 | artom | I remember there was a whole thing about dropping it from the gate a while back | |
| 16:42:17 | bauzas | I'm a 20% time person this week, funny | |
| 16:43:02 | dansmith | artom: we don't anymore | |
| 16:43:16 | dansmith | artom: and if you could figure out some way to break pgsql in this process, I'd sure appreciate it | |
| 16:43:31 | dansmith | if for no other reason than just to cause pain for that smug cfriesen guy | |
| 16:44:01 | bauzas | artom: we are not gating pgsql, hence not supporting it | |
| 16:44:47 | bauzas | I don't know if other projects still gate pg, but we very made it clear that pg is not supported anyway in nova | |
| 16:57:08 | xnox | Hello! | |
| 16:57:24 | xnox | is there nova quota for CPU cores.... per-arch/per-machinetype? | |