| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-13 | |||
| 22:57:02 | sean-k-mooney | oh that what they ment i though serial number was referint to the 123 e.g. a different name for the teanant id | |
| 22:57:10 | melwitt | heh | |
| 22:57:26 | mriedem | melwitt: yeah i noticed that too | |
| 22:58:32 | melwitt | I think the reason this isn't clear to me is because I know that strict isolation isn't possible, yet "placement_aggregate_required_for_tenants" makes it sound like it is | |
| 23:00:40 | melwitt | ok, looking at the code, I see that placement_aggregate_required_for_tenants means that it will warn and fail if no aggregate has a match for filter_tenant_id | |
| 23:01:31 | melwitt | does that mean strict tenant isolation *is* possible already? sounds like it | |
| 23:01:40 | sean-k-mooney | melwitt: i think you can build stict isolation with that if you manually map each tenant to a different aggretate | |
| 23:02:31 | melwitt | yeah... because if you turn this on, it will not let you land on filter_tenant_id-less hosts | |
| 23:02:34 | sean-k-mooney | then you can set the correct conf options to for the scheduler/plamcent to reject request that are not mapped to an aggregate | |
| 23:02:40 | sean-k-mooney | yep | |
| 23:03:08 | sean-k-mooney | but its just as much a pain in the ass as doing it with host aggrates and the old tenabt isolation filter | |
| 23:03:22 | melwitt | yeah, true | |
| 23:03:23 | sean-k-mooney | i proably performs better | |
| 23:03:49 | openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Load custom providers to Resource Tracker https://review.opendev.org/676029 | |
| 23:04:03 | melwitt | yeah ok, this isn't actually different, just the mechanism is different (better) and then you can get it to warn you if you didn't set the tenant keys on all your aggregates | |
| 23:05:14 | sean-k-mooney | yeah so its the waring/force fail behavior is slightly better but not a lot | |
| 23:06:30 | mriedem | the tenant to aggregate stuff was for cern i think, to model tenant-to-cell with azs / aggregates for a cell | |
| 23:06:35 | mriedem | since cern maps tenants to specific cells | |
| 23:08:07 | mriedem | and with that i'm out | |
| 23:08:55 | melwitt | yeah, I was thinking this was for CERN. the customer case I'm writing this up for is also dividing up the cluster per tenant | |
| 23:11:02 | sean-k-mooney | cern used to use the tenat isoaltion filter to do that tenat to cell mapping with cells v1 when they had two levels of scheduler | |
| 23:11:19 | sean-k-mooney | so ya this was a partity thing for them to be able to move to cells v2 | |
| 23:15:41 | melwitt | ah, thanks | |
| 23:16:20 | melwitt | I knew the basics of that but a lot of these scheduler things confuse me, so here I am :P | |
| 23:20:13 | sean-k-mooney | ya cern is an edge case as they have so many host they needed to use the limit parmater on the placmenet api and and if they did that without the aggrate feature placmenet could return only host that did not map to the correct cell | |
| 23:20:38 | sean-k-mooney | and when it hit the tenat isolation filter it would eliminate all hosts | |
| 23:20:44 | melwitt | ah right | |
| 23:21:39 | sean-k-mooney | so now they use this pluse a limit of like 15 host or something similarly tiny | |
| 23:21:49 | sean-k-mooney | the default limit being 1000 | |
| 23:47:02 | melwitt | sean-k-mooney: with forbidden aggregates, do you know off the top of your head what happens with aggregate-less hosts? will they still come back as eligible to be scheduled upon or no? | |
| 23:48:27 | sean-k-mooney | i belive the intent is the prefilter will append a forbidine aggreate requrest based on info in the the request spec to the placment query | |
| 23:48:56 | sean-k-mooney | so i guess they would be allowed | |
| 23:49:15 | sean-k-mooney | but no i dont know off the top of my head what the actul behavior is for certin | |
| 23:49:57 | melwitt | yeah, ok. that's what I was thinking | |
| 23:51:37 | sean-k-mooney | the spec is a little light on details | |
| 23:52:46 | sean-k-mooney | oh i see we went with sotring the tratit requriemetn in the hostaggreate metadata | |
| 23:52:57 | sean-k-mooney | * nova host agggreate metadata | |
| 23:53:53 | melwitt | ok, so yeah need everything to be in an aggregate in order for it to be excluded. I think. | |
| 23:54:14 | sean-k-mooney | yep exactlyu like my old filter worked... | |
| 23:54:27 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/stein: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676289 | |
| 23:54:37 | sean-k-mooney | except that predated placmenet and match on flavor extra_specs | |
| 23:55:09 | sean-k-mooney | actully this is a little less flexiable but its close | |
| 23:55:19 | melwitt | RIP old filter | |
| 23:56:16 | sean-k-mooney | melwitt: how would you feel about adding cpu pinning or hugepage traits? | |
| 23:56:39 | melwitt | I don't know :( I know very little about those | |
| 23:57:06 | sean-k-mooney | they dont exisats. yet... actully i have a better way to do what i want to do if i have thos | |
| 23:57:12 | sean-k-mooney | *those | |
| 23:57:45 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/rocky: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676290 | |
| 23:58:16 | sean-k-mooney | dpdk requried you to have hugepages to work with openstack | |
| 23:58:56 | sean-k-mooney | if we have a hugepage traits we could make it so your dpdk host would not allow vms without hugepages to be scheuled | |
| 23:59:22 | sean-k-mooney | for pinning we already have the dedicated cpus spec | |
| #openstack-nova - 2019-08-14 | |||
| 00:00:19 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/queens: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676291 | |
| 00:00:24 | sean-k-mooney | i would proably do this slightly differently however. e.g. not requrie the operator to confgiure aggreates and do it like my image metadtata tratis changes | |
| 00:02:09 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/pike: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676292 | |
| 00:02:49 | sean-k-mooney | melwitt: i havent reviewd the code for this but i hope they are caching the aggreate metadata | |
| 00:04:21 | sean-k-mooney | the only why the prefilter could work would eb to retrive all of the nova host aggreates and there metadata then calulate the aggrates that wont suppor the instance and generate the &member_of=!in:<agg1>,<agg2>,<agg3> query paramater | |
| 00:05:02 | melwitt | sean-k-mooney: here's the code, it's close to merging so get in on it while you can https://review.opendev.org/#/q/topic:bp/placement-req-filter-forbidden-aggregates+(status:open+OR+status:merged) | |
| 00:07:33 | sean-k-mooney | ya so no caching... | |
| 00:07:52 | sean-k-mooney | the proablem with caching is keping the cache valid | |
| 00:11:05 | sean-k-mooney | melwitt: the schduler does not directly talk to the db right? i.e. does the schduler acess the db via the conductor like teh compute service? | |
| 00:11:35 | melwitt | sean-k-mooney: I think it talks directly | |
| 00:11:39 | melwitt | only the compute service does not | |
| 00:12:00 | sean-k-mooney | ok so all the "control plane" service talk directly but not comptues | |
| 00:13:25 | melwitt | I think so | |
| 00:13:58 | sean-k-mooney | ok i was not sure if it was only the api,metadata service and conductor that went direct. | |
| 00:14:29 | sean-k-mooney | i guess it makes sense for the schduler to be able to take a fast path to the db | |
| 00:41:46 | melwitt | sean-k-mooney: you can see which services go through conductor by looking for 'indirection_api' under nova/cmd | |
| 00:42:23 | sean-k-mooney | indirection_api is new to me | |
| 00:43:32 | melwitt | yeah, that's the magic for objects going through conductor | |
| 00:44:15 | sean-k-mooney | ah ok. | |
| 00:46:47 | sean-k-mooney | the way the prefilter is workign bugs me | |
| 00:47:15 | sean-k-mooney | i need to re read the spec but i dont think it workign the way i expect it too | |
| 00:47:21 | sean-k-mooney | it may be equivalent | |
| 00:48:44 | sean-k-mooney | if the image has no trait request i think it would allow all aggreates | |
| 00:49:10 | sean-k-mooney | and i think you would have to list all traits in the imgae/flavor on the aggreate for it to match. | |
| 00:49:38 | sean-k-mooney | i proably should be less tired when reviewing this but this seams reversed to me | |
| 00:52:44 | melwitt | yeah, I think it will get all the aggregates, add any without the trait to the "no" bin, and then pass the "no" bin to placement and say "these are the forbidden aggregates, don't return hosts in them" | |
| 00:53:02 | sean-k-mooney | i think the logic is inverted. | |
| 00:53:13 | melwitt | which does seem backward but I assume it's the only way to do it | |
| 00:53:26 | sean-k-mooney | its treating the image and flaovr as the attoritive source nto the metadata | |
| 00:53:40 | sean-k-mooney | this is broken in the same way the existing filter is | |
| 00:53:58 | sean-k-mooney | and it the different betten the out of tree one and the in tree one | |
| 00:55:01 | sean-k-mooney | for this prefilter to work every trait that is in the image+flavor would have to be on the host_aggreate metadata | |
| 00:55:02 | melwitt | it does specifically say (in the proposed doc change) that image/flavor need not be set and will still not land on hosts in aggregates with =required. I just don't know how that's done | |
| 00:55:34 | sean-k-mooney | its done by invertin the relationship | |
| 00:56:10 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/ocata: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676295 | |
| 00:56:14 | sean-k-mooney | so what you need to do is get the metadata for all host_aggreates. then for each aggreate you check all the keys are present itn the flvore/image requst | |
| 00:56:32 | sean-k-mooney | if a key is set on the aggreate but not in the flaor/imgae then you add that to the forbindin list | |
| 00:56:34 | melwitt | oh ok | |
| 00:56:38 | melwitt | yeah | |
| 00:56:56 | sean-k-mooney | again exactly what the out of tree filter does :) | |
| 00:57:09 | sean-k-mooney | ok ill comment on the review | |
| 00:58:03 | sean-k-mooney | as implement this would work exactly the same as the aggregate_instance_extra_specs filter | |
| 00:58:24 | sean-k-mooney | well it sort of | |
| 00:58:41 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/ocata: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676295 | |
| 00:58:41 | sean-k-mooney | it use traits instead of any proerty and also looks at the image | |
| 00:58:47 | sean-k-mooney | but conceptuly its the same | |
| 01:01:14 | openstackgerrit | Takashi NATSUME proposed openstack/nova stable/ocata: Fix misuse of nova.objects.base.obj_equal_prims https://review.opendev.org/676295 | |
| 01:04:05 | sean-k-mooney | melwitt: actully it might be correct https://review.opendev.org/#/c/671074/6/nova/objects/aggregate.py@476 | |
| 01:04:58 | sean-k-mooney | in need to figure out what sql that generates | |