| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-01 | |||
| 19:20:32 | efried | Only one per host | |
| 19:20:45 | sean-k-mooney | that said i guess you would only have to do that on creating the rp the first time | |
| 19:20:51 | sean-k-mooney | at least for clean deployments | |
| 19:20:52 | cfriesen | sean-k-mooney: does nova-compute need to know about the child resource providers that cyborg created? | |
| 19:21:05 | sean-k-mooney | cfriesen: i dont think so | |
| 19:21:24 | sean-k-mooney | cfriesen: not in any of the interaction specs i have seen | |
| 19:21:25 | efried | cfriesen: Yeah, we talked about that above; the virt driver needs to know about them for purposes of whitelisting, deploying/attaching, etc. | |
| 19:21:39 | sean-k-mooney | efried: no it does not | |
| 19:21:48 | sean-k-mooney | the whitelisting is cyborges job | |
| 19:22:33 | sean-k-mooney | and deploy/attaching will be not dont in term of the VARs or whatever the equivalent of a port bining has become | |
| 19:22:59 | cfriesen | assuming nova owns a specific set of resource providers, and it's the only thing consuming from those resource providers (can we assume that) then it should only have to update inventory once at startup. | |
| 19:23:03 | openstackgerrit | Merged openstack/os-vif master: Do not import pyroute2 on Windows https://review.openstack.org/614728 | |
| 19:23:05 | ldau | Hi, somebody has installed all-in-one openstack using vmware as hypervisor? | |
| 19:23:38 | sean-k-mooney | cfriesen: that is the assumtion we stated in denver so yes i think that is still ture | |
| 19:23:39 | cfriesen | now if anything else can consume those resources, then we need the periodic inventory update | |
| 19:23:46 | efried | sean-k-mooney, cfriesen: if all of that is true, then we can indeed resolve the TODO that mriedem just blew away. | |
| 19:24:13 | efried | cfriesen: You mocking this up? | |
| 19:24:24 | cfriesen | not me. :) | |
| 19:24:52 | efried | cfriesen: "Consume the resources" doesn't matter. Changing inventory matters, but only for the providers I onw. | |
| 19:24:53 | efried | own | |
| 19:25:06 | efried | we don't cache allocation/usage data. | |
| 19:25:44 | cfriesen | efried: agreed. I guess it'd have to be something like CPU/RAM hotplug where it actually changes the inventory | |
| 19:26:11 | efried | But that would be noticed by the virt driver, which would update_provider_tree, which the rt would flush back. | |
| 19:26:13 | sean-k-mooney | cfriesen: it would be the virtdirver that would do that however | |
| 19:26:28 | efried | IOW, we're not getting rid of the cache. We're getting rid of all the cache *refreshing*. | |
| 19:26:39 | sean-k-mooney | efried: yes | |
| 19:26:46 | efried | sean-k-mooney: you mocking this up? | |
| 19:26:52 | efried | or is mriedem? | |
| 19:27:19 | mriedem | i stopped paying attention, what now? | |
| 19:27:20 | sean-k-mooney | efried: if we just disable the refesh in the config dose that not effectivly mock it up | |
| 19:27:51 | efried | mriedem: We're operating on the hypothesis that nova does *not* in fact need to know if outside agents create child providers that they will continue to own. | |
| 19:28:06 | efried | mriedem: And if that's true, we do *not* in fact need to refresh the cache, ever. | |
| 19:28:20 | efried | mriedem: So we *can* in fact resolve the TODO you just removed. | |
| 19:28:42 | efried | sean-k-mooney: More or less, yeah. Which is what CERN did. Which they seem to have had success with. | |
| 19:28:50 | mriedem | how about someone poop this out in the ML and get it sorted out there when gibi, jaypipes and cdent can also weigh in | |
| 19:29:00 | efried | sean-k-mooney: I think there's more we can do, though. | |
| 19:29:27 | mriedem | i think in general, we should default to *not* cache b/c of the cern issue, and only allow caching if you want to opt-in b/c you have a wildly busy env where inventory is changing a lot | |
| 19:29:29 | efried | mriedem: What was your idea to get the compute RP creation happening only once? | |
| 19:29:31 | mriedem | which i guess is a powervm thing | |
| 19:29:38 | sean-k-mooney | efried: yes proably | |
| 19:29:59 | mriedem | efried: yes, create the compute node root rp when the compute node is created, otherwise don't attempt to do the _ensure_resource_provider thing again | |
| 19:30:45 | mriedem | i.e. is it a powervm thing to be swapping out disk and such on the fly and expect nova-compute to just happily handle that? | |
| 19:30:57 | cfriesen | mriedem: changing inventory, or changing usage? | |
| 19:31:02 | mriedem | inventory | |
| 19:31:17 | mriedem | anyway, that probably doesn't matter here, | |
| 19:31:27 | cfriesen | do we expect anyone to have wildly changing inventory? I would have thought inventory is relatively stable. | |
| 19:31:27 | mriedem | we get the inventory regardless to know if it changed so we can push updates back to placement | |
| 19:31:35 | mriedem | cfriesen: that's what i said about 2 hours ago | |
| 19:31:37 | sean-k-mooney | mriedem: if its somthing that is discoverd by the virt driver its not an issue if it chagnes | |
| 19:32:26 | mriedem | i also don't think we really need to worry about refreshing aggregate relationships in the compute, | |
| 19:32:29 | efried | mriedem: I contend it doesn't matter if powervm (or any driver) changes inventory every single periodic. Because update_provider_tree is getting whatever the previous state of placement was (because placement isn't changed yet) and then update_from_provider_tree is flushing that back to placement *and* updating the cache accordingly. | |
| 19:32:30 | mriedem | since we don't do anything with those yet | |
| 19:32:38 | efried | yeah, what sean-k-mooney said, only bigger. | |
| 19:33:38 | sean-k-mooney | so do we all agree we dont need to refesh the cache in any case that is atleas emitly obvious to us | |
| 19:33:53 | efried | tentatively yes | |
| 19:34:35 | mriedem | i would have thought a lot of prior discussion about why we even have a cache in the first place has happene | |
| 19:34:37 | mriedem | *happened | |
| 19:34:57 | mriedem | therefore it seems pretty severe to just all of a sudden say, "oh i guess we don't" | |
| 19:35:19 | mriedem | and if there are reasons, do those reasons justify us caching by default | |
| 19:35:35 | mriedem | anyway, those are questions for the ML, not irc | |
| 19:35:37 | sean-k-mooney | if so then should we start with a patch to default the refresh to off. and a mailing list post to see what operators think/other feedback | |
| 19:35:55 | mriedem | cern is the only deployment big enough and new enough that i've heard complain about that refresh | |
| 19:36:01 | mriedem | not sure if mnaser is doing anything about it | |
| 19:36:20 | mnaser | hi | |
| 19:36:42 | mriedem | mnaser: tl;dr do you turn this way down? https://docs.openstack.org/nova/latest/configuration/config.html#compute.resource_provider_association_refresh | |
| 19:37:01 | mriedem | to avoid computes DDoS'ing placement every 5 minutes | |
| 19:37:49 | mnaser | i didn't know about that but i do feel like placement gets waaaaay too much unnecessary traffic | |
| 19:38:05 | mnaser | an idle cloud (aka literally no new vms being created/deleted) will constantly hit placement | |
| 19:38:22 | efried | so mriedem, to do the thing you were talking about earlier, we would add is_new_compute_node as a kwarg from _update_available_resource into _update => _update_to_placement and then only call _get_provider_tree_and_ensure_root if it's true? | |
| 19:38:44 | mriedem | mnaser: inventory updates baby! | |
| 19:38:47 | mnaser | i'm all in favour of minimizing the amount of traffic that placement gets however a lot of times we end up seeing weird stuff happen in placement db | |
| 19:38:58 | mnaser | so if we have to edit stuff via the api | |
| 19:39:06 | mnaser | it'd be nice to just know they work | |
| 19:39:17 | mriedem | efried: that's what i was thinking yeah, and nearly starting writing it, but then you guys all said in_tree was mega importante | |
| 19:39:42 | efried | mnaser: True story. If you edit something by hand, and we've switched off all this cache refreshing, the only way you're going to pick it up again is to restart the compute service. | |
| 19:39:52 | efried | or maybe we can work a HUP in there or something. | |
| 19:40:11 | mriedem | HUP is how we refresh the enabled/disabled cell cache in the scheduler | |
| 19:40:34 | sean-k-mooney | and any of the mutable config stuff if we have it so HUP makes sense | |
| 19:40:36 | mnaser | but really the only editing ive had to do was delete/remove allocations | |
| 19:40:42 | mnaser | that were stale for $reasons | |
| 19:40:50 | sean-k-mooney | mnaser: oh if its allocation that fine | |
| 19:40:54 | efried | mriedem: based on having toggled some kind of setting, right? I.e. HUP is a no-op if you haven't changed anything. | |
| 19:40:58 | mnaser | so forgive me for my silly question but | |
| 19:41:22 | mnaser | why do computes care about that information | |
| 19:41:31 | efried | mnaser: which information specifically? | |
| 19:41:37 | efried | allocations? | |
| 19:41:46 | mnaser | well, whatever api hits all the time, i think its allocations? | |
| 19:41:57 | efried | what hits the API all the time is inventory, providers, aggregates. | |
| 19:42:18 | mnaser | but inventory can be a one time hit on start up because afaik the state of that is pretty darn static right | |
| 19:42:37 | sean-k-mooney | no this check aggretes and traits on the provider not the assoications | |
| 19:42:38 | efried | mnaser: And what we're discussing is, the virt drivers need to know what that stuff looks like so they can *modify* the provider layout, inventory, etc. | |
| 19:42:43 | sean-k-mooney | *allocations | |
| 19:42:50 | efried | BUT that stuff shouldn't change unless the virt driver changes it | |
| 19:42:55 | efried | ...or if you muck with it in the CLI :) | |
| 19:43:16 | mnaser | yeah usually my mucking around is around allocations, that's where things get out of sync usually | |
| 19:43:38 | efried | mnaser: Placement for dummies won't help you. And for placement-in-nova, there's no such thing as for-dummies. | |
| 19:43:49 | mnaser | but i mean the "traits" change dynamically? | |
| 19:43:59 | mnaser | anyways, i wont let your discusion diverge too much | |
| 19:44:09 | efried | no, traits is a good point, /me thinks... | |
| 19:44:15 | mriedem | mnaser: you are asking and saying the same thing i've been saying for an hour or so, | |