| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-20 | |||
| 20:14:45 | dansmith | it's grotesque yet oddly satisfying | |
| 20:14:56 | jaypipes | ewww. | |
| 20:15:17 | jaypipes | dansmith: are you referring to the eagerload thing? | |
| 20:15:26 | dansmith | no | |
| 20:15:35 | dansmith | jaypipes: in the oldentimes, | |
| 20:15:47 | dansmith | we would load an instance, joined with metadata, system_metadata, etc | |
| 20:16:11 | dansmith | which would end up returning X*Y*Z rows for X instances, with Y rows of metadata and Z rows of sysmeta | |
| 20:16:17 | jaypipes | right. | |
| 20:16:29 | dansmith | which was the reason RAX failed to deploy icehouse after we moved flavor data to sysmeta | |
| 20:16:40 | dansmith | now we query metadata separately from the actual instance load | |
| 20:17:05 | dansmith | but I thought a convo with you a long time ago yielded you saying that we could do an inner-outer-blue-unicorn join to avoid that somehow | |
| 20:17:54 | jaypipes | yes, we can do a single query to get all instance metadata (and sysmeta) for all selected instances. | |
| 20:18:21 | dansmith | to be clear, | |
| 20:18:24 | jaypipes | a query that would just yield (instance_uuid, key, value) tuples. | |
| 20:18:33 | dansmith | no, that's not what I'm asking | |
| 20:19:18 | dansmith | the old query was returning the instance data itself, and the metadata key,value and the sysmeta key,value | |
| 20:19:31 | dansmith | so the instance data was repeated for every row in meta, sysmeta | |
| 20:19:48 | dansmith | that was a single query for the instance itself, and the metadata(s) | |
| 20:20:10 | dansmith | I know we can query the instance, and then query for the metadatas efficiently | |
| 20:20:12 | dansmith | but it's two queries | |
| 20:20:27 | dansmith | that's basically what we do now | |
| 20:20:50 | jaypipes | ok. well, that's the most efficient way to solve this particular problem. | |
| 20:20:56 | dansmith | two queries? | |
| 20:20:59 | jaypipes | yup. | |
| 20:21:03 | dansmith | okay | |
| 20:21:19 | dansmith | I'm pretty sure you called me a stupid ignoramus for doing that back in the icehouse days, | |
| 20:21:21 | jaypipes | I thought we were doing >1 query for grabbing instance metadata and system metadata | |
| 20:21:33 | dansmith | well, we are, but only for convenience | |
| 20:21:34 | jaypipes | no, I never called you anything. | |
| 20:21:38 | dansmith | YOU DID | |
| 20:22:10 | dansmith | we're doing three queries now, instance, meta, sysmeta | |
| 20:22:11 | jaypipes | if I ever said anything about performance it would have been because we were doing a query on instances, then *for each instance* issuing a query to get some instance metadata. | |
| 20:22:21 | dansmith | we're not doing that | |
| 20:22:25 | jaypipes | ok, coools. | |
| 20:22:37 | jaypipes | I can reduce the meta + sysmeta to a single query. | |
| 20:22:42 | dansmith | so I guess that means I have to keep plugging at this | |
| 20:22:52 | dansmith | jaypipes: yea, feel free, but that's separate from my other work here | |
| 20:23:08 | jaypipes | k. is there anything I can help you with on your work here? | |
| 20:23:21 | dansmith | jaypipes: you just did | |
| 20:23:24 | dansmith | thanks | |
| 20:23:25 | jaypipes | :) | |
| 20:23:40 | jaypipes | well, at least that gives me something to smile about today. | |
| 20:23:42 | jaypipes | thanks. | |
| 20:25:19 | jaypipes | FTR, on the cyborg thing, I *also* believe that cyborg should be modeling should be inventories, not RPs with tons of traits masquerading as resource classes. | |
| 20:25:38 | jaypipes | dansmith: ^ | |
| 20:25:47 | dansmith | jaypipes: yay. | |
| 20:25:59 | jaypipes | indeed. | |
| 20:26:04 | mriedem | dansmith: on top of the improved join on what you're doing, i think it's a 2-part change in that the metadata api doesn't need to be pre-loading on system_metadata - at least not anymore | |
| 20:26:42 | dansmith | mriedem: yeah, so that will address the acute issue right? | |
| 20:26:45 | mriedem | last i looked the only thing in meta-api that would use sysmeta is a vendor data provider if configured | |
| 20:26:53 | dansmith | maybe I should punt this until we have a better reason to do this work | |
| 20:27:02 | mriedem | i believe so, and i think that's what the workday ops guy said he did in the ML | |
| 20:27:06 | mriedem | heh | |
| 20:27:18 | mriedem | see, i started trying to do what you said and sparks flew immediately | |
| 20:27:19 | mriedem | and i gave up | |
| 20:27:34 | dansmith | oh did you | |
| 20:27:35 | dansmith | ? | |
| 20:27:40 | mriedem | locally | |
| 20:27:45 | dansmith | maybe that should be my impetus to fix it | |
| 20:27:45 | mriedem | never pushed it up b/c tests failed horribly | |
| 20:27:52 | mriedem | to show me up? | |
| 20:27:55 | dansmith | yeah | |
| 20:27:59 | mriedem | by all means | |
| 20:28:06 | dansmith | nah, sounds hard. | |
| 20:28:12 | mriedem | next you can fix the nova/cinder cross az attach mess | |
| 20:28:29 | mriedem | which i have a fix for, but it's fugly as all get out | |
| 20:29:09 | dansmith | I like it already | |
| 20:40:34 | efried | dansmith, jaypipes: Modeling accelerators via specific resource classes, so like CUSTOM_FPGA_GZIP rather than rc=FPGA + traits=[GZIP] ? | |
| 20:41:05 | dansmith | GZIP isn't a trait, IMHO | |
| 20:41:18 | dansmith | like, I don't ask for SOME_SILICON=1024, trait=RAM | |
| 20:41:19 | efried | um. The FPGA is capable of processing gzips? | |
| 20:42:01 | jaypipes | the resource class is a context to a GZIP program flashed to a device. | |
| 20:42:04 | dansmith | right, but there's a difference between asking for an FPGA and asking for GZIP offload to me | |
| 20:42:18 | jaypipes | dansmith++ | |
| 20:42:26 | dansmith | if I want an FPGA that I can program myself, I want an FPGA=1.. if I want a GZIP handler, I want GZIP=1, | |
| 20:42:32 | dansmith | which might be an FPGA in the back end | |
| 20:42:37 | dansmith | or it might be an ASIC | |
| 20:42:39 | dansmith | or whatever | |
| 20:44:01 | efried | But then a resource provider representing a blank (as-yet-unprogrammed) FPGA would have to show inventories of multiple resource classes, and then when one of those is consumed, we would have to nix the other resource classes (or do the reserved=total trick for them). | |
| 20:44:17 | efried | which is racy, as well as being ew. | |
| 20:44:19 | dansmith | same for the trait right? | |
| 20:44:32 | dansmith | you say FPGA=1, trats=GZIP,TLS,BITCOIN | |
| 20:44:51 | efried | no. GZIP_CAPABLE stays. Not sure we have to retrait every time we reprogram. | |
| 20:45:32 | efried | but if we do, the GZIP_IS_ON_THIS_THING_AT_THE_MOMENT trait would be separate, and have separate meaning, than the GZIP_CAPABLE. | |
| 20:45:57 | dansmith | you're just providing no abstraction there | |
| 20:46:02 | efried | the former would be used only as an optimization, if/when we have "preferred traits", to avoid reprogramming if there's one that's already set up. | |
| 20:46:30 | jaypipes | premature optimization... | |
| 20:47:03 | efried | I'd be happy if we skipped that whole bit for the first pass and just used the *_CAPABLE traits. | |
| 20:47:14 | efried | Programming gets done after the claim, if and as necessary. | |
| 20:47:23 | jaypipes | I'd be happy if we just skipped everything other than just using custom resource classes. | |
| 20:47:46 | efried | jaypipes: So preprogram everything? | |
| 20:48:05 | dansmith | pretty sure we said the first step was assuming everything was static, no? | |
| 20:48:22 | dansmith | except for the "user will program it themselves" case of course | |
| 20:48:44 | jaypipes | dansmith++ again. | |
| 20:49:21 | dansmith | when we talked about this in (denver I think?) I think the overwhelming majority of cases where this really applies is the pre-programmed case, | |
| 20:49:39 | dansmith | because it provides for locality in certain FGPAs that have one code region and multiple execution contexts, | |
| 20:50:11 | dansmith | such that if you co-locate a GZIP and a TLS, they both can't use the same FPGA, but if you get two GZIP tenants on the same box, they can | |
| 20:50:41 | dansmith | and I thought we agreed to avoid boiling the ocean with "everything is completely dynamic all the time forever" until we could do, you know, fucking anything :) | |
| 20:50:56 | efried | I can buy it for a first pass. Long-term, that seems like not very cloudy. Though I suppose if the "pre"programming is done by a higher orchestrator, it could fly. | |
| 20:50:58 | dansmith | maybe that's just me missing something, but.. | |