| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-01 | |||
| 19:40:36 | mnaser | but really the only editing ive had to do was delete/remove allocations | |
| 19:40:42 | mnaser | that were stale for $reasons | |
| 19:40:50 | sean-k-mooney | mnaser: oh if its allocation that fine | |
| 19:40:54 | efried | mriedem: based on having toggled some kind of setting, right? I.e. HUP is a no-op if you haven't changed anything. | |
| 19:40:58 | mnaser | so forgive me for my silly question but | |
| 19:41:22 | mnaser | why do computes care about that information | |
| 19:41:31 | efried | mnaser: which information specifically? | |
| 19:41:37 | efried | allocations? | |
| 19:41:46 | mnaser | well, whatever api hits all the time, i think its allocations? | |
| 19:41:57 | efried | what hits the API all the time is inventory, providers, aggregates. | |
| 19:42:18 | mnaser | but inventory can be a one time hit on start up because afaik the state of that is pretty darn static right | |
| 19:42:37 | sean-k-mooney | no this check aggretes and traits on the provider not the assoications | |
| 19:42:38 | efried | mnaser: And what we're discussing is, the virt drivers need to know what that stuff looks like so they can *modify* the provider layout, inventory, etc. | |
| 19:42:43 | sean-k-mooney | *allocations | |
| 19:42:50 | efried | BUT that stuff shouldn't change unless the virt driver changes it | |
| 19:42:55 | efried | ...or if you muck with it in the CLI :) | |
| 19:43:16 | mnaser | yeah usually my mucking around is around allocations, that's where things get out of sync usually | |
| 19:43:38 | efried | mnaser: Placement for dummies won't help you. And for placement-in-nova, there's no such thing as for-dummies. | |
| 19:43:49 | mnaser | but i mean the "traits" change dynamically? | |
| 19:43:59 | mnaser | anyways, i wont let your discusion diverge too much | |
| 19:44:09 | efried | no, traits is a good point, /me thinks... | |
| 19:44:15 | mriedem | mnaser: you are asking and saying the same thing i've been saying for an hour or so, | |
| 19:44:19 | mriedem | that inventory is pretty static | |
| 19:44:32 | sean-k-mooney | mnaser: we technicall propised that operators should be able to add traits to RPs but currently the virt dirver just overrite them i think | |
| 19:44:32 | mnaser | its 100% static.. things in inventory are | |
| 19:44:36 | efried | once again, if traits change due to external factors, you could HUP to get that flushed. | |
| 19:44:37 | mriedem | traits could be changed out of band if you're decorating capabilities on your compute node for scheduling, | |
| 19:44:41 | mnaser | memory.. disk.. vcpus.. | |
| 19:44:43 | mriedem | and aggregates aren't used in the compute service (yet) | |
| 19:45:00 | mnaser | yeah unless someone is hot plugging in memory/disk/cpu | |
| 19:45:05 | mnaser | i dont see inventory changing | |
| 19:45:22 | mnaser | and yes i agree traits make sense if you wanna say this compute node is special.. but also, does that compute node really care to know if its special? | |
| 19:45:27 | mnaser | only the scheduler cares that it's special.. | |
| 19:45:35 | sean-k-mooney | mriedem: well the only ones that are are the ones that are created form nova host aggreates(assuming jays stuff laned last cycle) | |
| 19:45:40 | efried | you could run into some interesting race conditions. If you muck with traits at the same time as the virt driver is mucking with traits, whoever gets there last will win. | |
| 19:45:40 | mriedem | mnaser: almost correct | |
| 19:45:59 | mriedem | right, we try to merge in what the compute is reporting for traits with what was put on the node resource provider for traits externally | |
| 19:46:13 | sean-k-mooney | mriedem: sorry you said compute service ignore that | |
| 19:46:31 | mnaser | ok so its like | |
| 19:46:34 | mriedem | the *only* thing on the compute that would care about aggregates in placement is shared storage providers, | |
| 19:46:36 | mriedem | which we don't support yet | |
| 19:46:37 | mnaser | self reported traits + "user" decorated traits | |
| 19:46:44 | mriedem | mnaser: yes | |
| 19:47:13 | mnaser | but the nova-compute reported traits.. i feel like those are pretty static right? maybe i just don't know any wild use cases | |
| 19:47:18 | mriedem | i think i've been saying since pike, at least queens, we don't need to refresh aggregate info in compute | |
| 19:47:26 | mriedem | mnaser: proably depends on the driver | |
| 19:47:30 | mnaser | but i feel like most of the time, if a system trait changes, you probably have nova-compute restart things | |
| 19:47:31 | mnaser | ah ok | |
| 19:47:44 | mriedem | vmware would love to be able to randomly proxy traits from vcenter through nova-compute to placement | |
| 19:47:45 | cfriesen | mnaser: the one exception would be something like vTPM where the driver uses the presence of the requested trait to decide to do something with the instance. | |
| 19:47:48 | mriedem | for changes in vcenter | |
| 19:47:57 | sean-k-mooney | mnaser: the intent with the user decorated traits was to let the operator tag node with stuff the virt dirver cant discover or to express policy | |
| 19:48:11 | cfriesen | mnaser: but that's really looking at the requested trait, not the trait on the resource provider | |
| 19:48:28 | sean-k-mooney | cfriesen: but that is in the instance request | |
| 19:48:40 | mnaser | yeah, i dunno, i feel like those will not change much, and i think just calling a method *once* when you make some changes rather than all the time isnt probelmatic | |
| 19:48:42 | sean-k-mooney | cfriesen: it does not need the RP info | |
| 19:48:43 | cfriesen | sean-k-mooney: yah, that's what I realized after typing the first sentence. :) | |
| 19:48:53 | mnaser | i mean lets be honest, we don't "refresh" allocations and those can go pretty stale | |
| 19:49:09 | mnaser | are "system" and "user" traits distingusable or in a race they can wipe each other out? | |
| 19:49:30 | mriedem | system traits might be 'standard' traits | |
| 19:49:35 | mriedem | user traits would be CUSTOM traits | |
| 19:49:42 | sean-k-mooney | efried: i assume we update the resouce provider generattion when updating traits? | |
| 19:49:52 | mriedem | but a user could put standard traits on a provider that the virt driver doesn't report | |
| 19:49:56 | mriedem | sean-k-mooney: yes | |
| 19:49:57 | efried | sean-k-mooney: Placement does | |
| 19:50:17 | cfriesen | sean-k-mooney: on a totally different topic, have you ever run into a scenario where qemu has a thread sitting at 100% cpu but not making any forward progress? I'm assuming it's a livelock somehow, just not sure how. | |
| 19:50:22 | sean-k-mooney | ya so one of either the user or the virtdriver will fail in that case and have to retry | |
| 19:50:23 | mriedem | i think the only virt driver today that reports any traits is the libvirt driver reporting cpu features | |
| 19:50:24 | efried | so yeah, this is where re-GET-and-redrive comes into play. | |
| 19:50:28 | sean-k-mooney | so there is not race | |
| 19:50:41 | mnaser | i still don't see the need for constantly updating, sounds like this is something that you can just report once on start up or when it changes | |
| 19:50:42 | sean-k-mooney | well unless the clinet is coded badly | |
| 19:50:49 | efried | mnaser: Yes | |
| 19:51:03 | mnaser | MAYBE pull it in when a new VM gets spun up | |
| 19:51:18 | mnaser | if it means a different codepath | |
| 19:51:51 | sean-k-mooney | so currntly resource_provider_association_refresh has a min vlaue of 1. can we allow 0 and define that to mean update cache only on startup or sig hup | |
| 19:53:44 | sean-k-mooney | mriedem: did you not have a propsal to report things like support_migration as traits too | |
| 19:53:48 | cfriesen | dunno what's going on today, I keep disconnecting | |
| 19:54:06 | sean-k-mooney | mriedem: or is that handeled by the compute manager above the virt dirver level | |
| 19:54:28 | mnaser | anyways thats my 2 cents | |
| 19:56:12 | efried | mriedem, sean-k-mooney: Is ComputeManager.reset() the right hook for that SIGHUP thing? | |
| 19:56:14 | mriedem | sean-k-mooney: https://review.openstack.org/#/c/538498/ | |
| 19:56:31 | mriedem | efried: yeah | |
| 19:56:48 | sean-k-mooney | mriedem: ah ya that is what i was thinking of | |
| 19:57:42 | openstackgerrit | Merged openstack/nova master: PowerVM upt parity for reshaper, DISK_GB reserved https://review.openstack.org/614643 | |
| 19:58:02 | efried | to that method | |
| 19:58:02 | efried | self.reportclient = self.scheduler_client.reportclient | |
| 19:58:02 | efried | self.scheduler_client = scheduler_client.SchedulerClient() | |
| 19:58:02 | efried | mriedem: So like if I wanted to "clear the cache" I could add | |
| 19:58:33 | efried | or if I wanted to be narrower about it, I could add a reset() to the report client and invoke self.reportclient.reset() from there instead. | |
| 19:59:01 | mriedem | i reckon | |
| 19:59:03 | sean-k-mooney | efried: reset might be better | |
| 19:59:48 | mriedem | this just seems like a 180 in attitude about how important it is to having nova-compute be totally self-healing on every periodic | |
| 19:59:54 | mriedem | which i'm sure was debated to death in releases past | |
| 20:00:57 | sean-k-mooney | mriedem: well nothing is stopping us also reviving the notificaiton also to move the healing to a push model | |
| 20:01:42 | mriedem | there are plenty of things stopping me from doing anything | |
| 20:01:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clean up cpu_shared_set config docs https://review.openstack.org/614864 | |
| 20:02:41 | efried | mriedem: I think a lot of the reasoning in the past was because information was coming from several different places while we were transitioning to placement but not fully there yet. | |
| 20:02:55 | efried | We're getting pretty close to fully there at this point, so I think a lot of this stuff is going to get to be cleaned up. | |
| 20:03:04 | efried | Like _normalize_allocation_and_reserved_bullshit() | |