| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-01 | |||
| 18:37:27 | sean-k-mooney | technicall the api does not enforce that as we dont have owner of resouce proivers in the api however | |
| 18:38:28 | mriedem | what i'm hearing is we can't remove this todo https://github.com/openstack/nova/blob/1e823f21997018bcd197057ebd4d6207a5c54403/nova/scheduler/client/report.py#L1011 even if we know we didn't just create the root compute node, because we need to call it anyway to determine if there are new nested providers under that pre-existing compute node | |
| 18:38:40 | mriedem | s/remove/resolve/ | |
| 18:38:50 | mriedem | iow, the todo should be removed b/c we can't do anything about it | |
| 18:38:56 | mriedem | even if we *know* the compute node record was just created | |
| 18:39:23 | sean-k-mooney | am i dont know if we need to know if there are new nested resouce providers | |
| 18:39:38 | mriedem | that's the whole in_tree thing i thought | |
| 18:39:41 | sean-k-mooney | in fact i would asser as the compute node we dont need to know that | |
| 18:40:48 | mriedem | ok well what i'm saying is we (the RT) know when we created a new compute node record, and thus need to create its resource provider, i guess i'll wait for someone to tell me if that's worth doing so we can avoid the GET /resource_providers?in_tree=<uuid of the thing we just created and thus doesn't exist yet> case | |
| 18:42:00 | sean-k-mooney | mriedem: in that case i think you are right wew dont need the /resource_providers?in_tree=<thing i just created> call | |
| 18:43:11 | sean-k-mooney | the placement api will not allow me to create a resouce provider with a parrent uuid that does not exist | |
| 18:44:18 | efried | sean-k-mooney: We *do* need to know if there are new nested providers. | |
| 18:44:52 | sean-k-mooney | efried: there cant be nested resouce providers of a compute node if we have not created the compute node yet right | |
| 18:45:08 | efried | That's what ?in_tree is about, though. | |
| 18:45:26 | efried | ?in_tree=$compute_rp gives me the compute RP and any descendants. | |
| 18:45:35 | sean-k-mooney | yes | |
| 18:46:06 | sean-k-mooney | but if the compute RP does not exist yet then cyborge cant create nested resources under it | |
| 18:46:14 | efried | So at T0, it gives me nothing, so I create the compute RP. At T1 it gives me the compute RP. At T2, cyborg creates a child provider for a device. At T3 ?in_tree=$compute_rp gives me both providers. | |
| 18:46:28 | efried | If I didn't call ?in_tree I would never know about that device RP | |
| 18:46:44 | efried | and I need to know about that device RP e.g. from my virt driver so I can white/blacklist it and/or deploy it. | |
| 18:46:48 | sean-k-mooney | efried: sure you dont own that and as nova you cant directly modify it | |
| 18:47:17 | efried | Unclear whether blacklisting happens at cyborg or at nova | |
| 18:47:41 | efried | but what you say makes sense (single ownership) so it would have to be at cyborg. | |
| 18:48:14 | efried | Which means virt driver-esque code would need to be invoked by cyborg to do discovery in the first place. | |
| 18:48:30 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove TODO from get_provider_tree_and_ensure_root https://review.openstack.org/614835 | |
| 18:48:41 | sean-k-mooney | efried: well not nessisarily | |
| 18:49:30 | sean-k-mooney | we did suggest that in update_provider_tree we could call os-acc to do that | |
| 18:49:40 | efried | yes | |
| 18:49:45 | efried | wait | |
| 18:50:06 | sean-k-mooney | but what cyborg need to do is 1 lookup the root provider for the compute node | |
| 18:50:15 | efried | update_provider_tree would call os_acc with a list of discovered-and-already-whitelist-scrubbed devices so that cyborg can create the providers? | |
| 18:50:31 | sean-k-mooney | efried: no | |
| 18:50:55 | sean-k-mooney | if nova is doing the deicovery we are rebuiling cyborg in nova | |
| 18:51:33 | sean-k-mooney | the idea was that we woudl pass in the current tree to cyborg and it woudl do the discovery itslf and append to that tree | |
| 18:51:38 | sean-k-mooney | but the other approch | |
| 18:51:46 | sean-k-mooney | which is what we were going to do | |
| 18:52:03 | sean-k-mooney | was cybroge poll placement for compute node to be created. | |
| 18:52:24 | sean-k-mooney | then it would add child resouce providers to the tree created by nova | |
| 18:52:42 | sean-k-mooney | but not modify any resouce provider it did not created | |
| 18:57:33 | sean-k-mooney | efried: today are we doing the provider tree update by put or patch. if put what would it take to make it a patch so nova can do a partial update and merge it on the placement side | |
| 18:58:27 | efried | mriedem: quick fix pls | |
| 18:59:29 | efried | sean-k-mooney: patch is only applicable if you're talking about modifying part of a single provider. Which I think we're not considering. | |
| 18:59:56 | efried | sean-k-mooney: IIUC, you're suggesting modifying some providers in the tree, but not others. That's still PUT - one per provider to be modified. | |
| 19:00:05 | efried | And it's what we do today, see update_from_provider_tree | |
| 19:00:17 | sean-k-mooney | efried: ok cool | |
| 19:00:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove TODO from get_provider_tree_and_ensure_root https://review.openstack.org/614835 | |
| 19:00:51 | efried | +2 ^ | |
| 19:03:31 | sean-k-mooney | what i was actully suggesting was make it a single patch call to placement to update all resouce providers in a tree owned by service x but thats a different conversation | |
| 19:03:46 | efried | totally | |
| 19:04:48 | sean-k-mooney | i am still not aware of an usecase that woudl require nova to be aware of a resouce provider created by another service by the way | |
| 19:05:10 | sean-k-mooney | when i say nova i sepcfically mean the compute agent | |
| 19:07:59 | dansmith | efried: so was there some outcome? | |
| 19:08:21 | efried | dansmith: Remember that thing they did where they disabled the refresh_associations? | |
| 19:08:51 | dansmith | oh they reverted that? | |
| 19:08:54 | dansmith | accidentally | |
| 19:08:59 | efried | dansmith: That's why all those calls were zeroes in queens and nonzero once they upgraded (because that hack was no longer there). Yeah. | |
| 19:09:04 | dansmith | ah cool. | |
| 19:09:17 | efried | So belmiro is going to reinstate that and come back at us. | |
| 19:09:23 | dansmith | right on | |
| 19:09:32 | efried | but it's still a shit ton of calls | |
| 19:10:06 | efried | Matt and Sean and I brainstormed briefly on whether we could just get rid of the cache completely (and whether that would actually help). | |
| 19:10:22 | efried | And what we actually ended up doing was getting rid of a comment: https://review.openstack.org/614835 :( | |
| 19:13:04 | sean-k-mooney | efried: well i think we could maybe get rid of the cache but i think it needs a spec not irc ideas | |
| 19:13:24 | efried | sean-k-mooney: I would rather see a PoC in code for that one than a spec. | |
| 19:14:11 | sean-k-mooney | efried: well that a possiblity but it would be similar to neutrons notifications for prot/network events | |
| 19:15:14 | efried | yeah, a subscribable notification framework at placement itself would be cool. | |
| 19:15:46 | sean-k-mooney | yep that is what i was about to type but had not decided how to phase it | |
| 19:15:46 | mriedem | there was a blueprint for that at one point i think | |
| 19:16:07 | mriedem | https://blueprints.launchpad.net/nova/+spec/placement-notifications | |
| 19:16:13 | sean-k-mooney | well this is a much more positive resonce to this idea then i had expected | |
| 19:17:29 | sean-k-mooney | if we had an owner attribute on every provider and a api to register owner with call backs then each service could register a subsciption to the rps they own | |
| 19:19:20 | mriedem | and if ifs and buts were candy and nuts we'd all have a merry christmas | |
| 19:19:35 | efried | I was thinking much simpler to start. | |
| 19:19:49 | efried | You could register for a notification any time $rp_uuid is touched. | |
| 19:20:11 | efried | which includes "create a resource provider with $rp_uuid as a root", which solves the use case we were discussing. | |
| 19:20:20 | sean-k-mooney | efried: i considered that but that could be a lot of RPs | |
| 19:20:32 | efried | Only one per host | |
| 19:20:45 | sean-k-mooney | that said i guess you would only have to do that on creating the rp the first time | |
| 19:20:51 | sean-k-mooney | at least for clean deployments | |
| 19:20:52 | cfriesen | sean-k-mooney: does nova-compute need to know about the child resource providers that cyborg created? | |
| 19:21:05 | sean-k-mooney | cfriesen: i dont think so | |
| 19:21:24 | sean-k-mooney | cfriesen: not in any of the interaction specs i have seen | |
| 19:21:25 | efried | cfriesen: Yeah, we talked about that above; the virt driver needs to know about them for purposes of whitelisting, deploying/attaching, etc. | |
| 19:21:39 | sean-k-mooney | efried: no it does not | |
| 19:21:48 | sean-k-mooney | the whitelisting is cyborges job | |
| 19:22:33 | sean-k-mooney | and deploy/attaching will be not dont in term of the VARs or whatever the equivalent of a port bining has become | |
| 19:22:59 | cfriesen | assuming nova owns a specific set of resource providers, and it's the only thing consuming from those resource providers (can we assume that) then it should only have to update inventory once at startup. | |
| 19:23:03 | openstackgerrit | Merged openstack/os-vif master: Do not import pyroute2 on Windows https://review.openstack.org/614728 | |
| 19:23:05 | ldau | Hi, somebody has installed all-in-one openstack using vmware as hypervisor? | |
| 19:23:38 | sean-k-mooney | cfriesen: that is the assumtion we stated in denver so yes i think that is still ture | |
| 19:23:39 | cfriesen | now if anything else can consume those resources, then we need the periodic inventory update | |
| 19:23:46 | efried | sean-k-mooney, cfriesen: if all of that is true, then we can indeed resolve the TODO that mriedem just blew away. | |
| 19:24:13 | efried | cfriesen: You mocking this up? | |
| 19:24:24 | cfriesen | not me. :) | |
| 19:24:52 | efried | cfriesen: "Consume the resources" doesn't matter. Changing inventory matters, but only for the providers I onw. | |
| 19:24:53 | efried | own | |
| 19:25:06 | efried | we don't cache allocation/usage data. | |
| 19:25:44 | cfriesen | efried: agreed. I guess it'd have to be something like CPU/RAM hotplug where it actually changes the inventory | |
| 19:26:11 | efried | But that would be noticed by the virt driver, which would update_provider_tree, which the rt would flush back. | |
| 19:26:13 | sean-k-mooney | cfriesen: it would be the virtdirver that would do that however | |