| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-01 | |||
| 19:45:27 | mnaser | only the scheduler cares that it's special.. | |
| 19:45:35 | sean-k-mooney | mriedem: well the only ones that are are the ones that are created form nova host aggreates(assuming jays stuff laned last cycle) | |
| 19:45:40 | efried | you could run into some interesting race conditions. If you muck with traits at the same time as the virt driver is mucking with traits, whoever gets there last will win. | |
| 19:45:40 | mriedem | mnaser: almost correct | |
| 19:45:59 | mriedem | right, we try to merge in what the compute is reporting for traits with what was put on the node resource provider for traits externally | |
| 19:46:13 | sean-k-mooney | mriedem: sorry you said compute service ignore that | |
| 19:46:31 | mnaser | ok so its like | |
| 19:46:34 | mriedem | the *only* thing on the compute that would care about aggregates in placement is shared storage providers, | |
| 19:46:36 | mriedem | which we don't support yet | |
| 19:46:37 | mnaser | self reported traits + "user" decorated traits | |
| 19:46:44 | mriedem | mnaser: yes | |
| 19:47:13 | mnaser | but the nova-compute reported traits.. i feel like those are pretty static right? maybe i just don't know any wild use cases | |
| 19:47:18 | mriedem | i think i've been saying since pike, at least queens, we don't need to refresh aggregate info in compute | |
| 19:47:26 | mriedem | mnaser: proably depends on the driver | |
| 19:47:30 | mnaser | but i feel like most of the time, if a system trait changes, you probably have nova-compute restart things | |
| 19:47:31 | mnaser | ah ok | |
| 19:47:44 | mriedem | vmware would love to be able to randomly proxy traits from vcenter through nova-compute to placement | |
| 19:47:45 | cfriesen | mnaser: the one exception would be something like vTPM where the driver uses the presence of the requested trait to decide to do something with the instance. | |
| 19:47:48 | mriedem | for changes in vcenter | |
| 19:47:57 | sean-k-mooney | mnaser: the intent with the user decorated traits was to let the operator tag node with stuff the virt dirver cant discover or to express policy | |
| 19:48:11 | cfriesen | mnaser: but that's really looking at the requested trait, not the trait on the resource provider | |
| 19:48:28 | sean-k-mooney | cfriesen: but that is in the instance request | |
| 19:48:40 | mnaser | yeah, i dunno, i feel like those will not change much, and i think just calling a method *once* when you make some changes rather than all the time isnt probelmatic | |
| 19:48:42 | sean-k-mooney | cfriesen: it does not need the RP info | |
| 19:48:43 | cfriesen | sean-k-mooney: yah, that's what I realized after typing the first sentence. :) | |
| 19:48:53 | mnaser | i mean lets be honest, we don't "refresh" allocations and those can go pretty stale | |
| 19:49:09 | mnaser | are "system" and "user" traits distingusable or in a race they can wipe each other out? | |
| 19:49:30 | mriedem | system traits might be 'standard' traits | |
| 19:49:35 | mriedem | user traits would be CUSTOM traits | |
| 19:49:42 | sean-k-mooney | efried: i assume we update the resouce provider generattion when updating traits? | |
| 19:49:52 | mriedem | but a user could put standard traits on a provider that the virt driver doesn't report | |
| 19:49:56 | mriedem | sean-k-mooney: yes | |
| 19:49:57 | efried | sean-k-mooney: Placement does | |
| 19:50:17 | cfriesen | sean-k-mooney: on a totally different topic, have you ever run into a scenario where qemu has a thread sitting at 100% cpu but not making any forward progress? I'm assuming it's a livelock somehow, just not sure how. | |
| 19:50:22 | sean-k-mooney | ya so one of either the user or the virtdriver will fail in that case and have to retry | |
| 19:50:23 | mriedem | i think the only virt driver today that reports any traits is the libvirt driver reporting cpu features | |
| 19:50:24 | efried | so yeah, this is where re-GET-and-redrive comes into play. | |
| 19:50:28 | sean-k-mooney | so there is not race | |
| 19:50:41 | mnaser | i still don't see the need for constantly updating, sounds like this is something that you can just report once on start up or when it changes | |
| 19:50:42 | sean-k-mooney | well unless the clinet is coded badly | |
| 19:50:49 | efried | mnaser: Yes | |
| 19:51:03 | mnaser | MAYBE pull it in when a new VM gets spun up | |
| 19:51:18 | mnaser | if it means a different codepath | |
| 19:51:51 | sean-k-mooney | so currntly resource_provider_association_refresh has a min vlaue of 1. can we allow 0 and define that to mean update cache only on startup or sig hup | |
| 19:53:44 | sean-k-mooney | mriedem: did you not have a propsal to report things like support_migration as traits too | |
| 19:53:48 | cfriesen | dunno what's going on today, I keep disconnecting | |
| 19:54:06 | sean-k-mooney | mriedem: or is that handeled by the compute manager above the virt dirver level | |
| 19:54:28 | mnaser | anyways thats my 2 cents | |
| 19:56:12 | efried | mriedem, sean-k-mooney: Is ComputeManager.reset() the right hook for that SIGHUP thing? | |
| 19:56:14 | mriedem | sean-k-mooney: https://review.openstack.org/#/c/538498/ | |
| 19:56:31 | mriedem | efried: yeah | |
| 19:56:48 | sean-k-mooney | mriedem: ah ya that is what i was thinking of | |
| 19:57:42 | openstackgerrit | Merged openstack/nova master: PowerVM upt parity for reshaper, DISK_GB reserved https://review.openstack.org/614643 | |
| 19:58:02 | efried | to that method | |
| 19:58:02 | efried | self.reportclient = self.scheduler_client.reportclient | |
| 19:58:02 | efried | self.scheduler_client = scheduler_client.SchedulerClient() | |
| 19:58:02 | efried | mriedem: So like if I wanted to "clear the cache" I could add | |
| 19:58:33 | efried | or if I wanted to be narrower about it, I could add a reset() to the report client and invoke self.reportclient.reset() from there instead. | |
| 19:59:01 | mriedem | i reckon | |
| 19:59:03 | sean-k-mooney | efried: reset might be better | |
| 19:59:48 | mriedem | this just seems like a 180 in attitude about how important it is to having nova-compute be totally self-healing on every periodic | |
| 19:59:54 | mriedem | which i'm sure was debated to death in releases past | |
| 20:00:57 | sean-k-mooney | mriedem: well nothing is stopping us also reviving the notificaiton also to move the healing to a push model | |
| 20:01:42 | mriedem | there are plenty of things stopping me from doing anything | |
| 20:01:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clean up cpu_shared_set config docs https://review.openstack.org/614864 | |
| 20:02:41 | efried | mriedem: I think a lot of the reasoning in the past was because information was coming from several different places while we were transitioning to placement but not fully there yet. | |
| 20:02:55 | efried | We're getting pretty close to fully there at this point, so I think a lot of this stuff is going to get to be cleaned up. | |
| 20:03:04 | efried | Like _normalize_allocation_and_reserved_bullshit() | |
| 20:04:11 | mriedem | efried: maybe, but i specifically remember you bringing up something once about how powervm shared storage pools can have disk swapped in and out on a whim and nova-compute should be cool with reporting that as it changes - but maybe that's unrelated to this, idk | |
| 20:04:59 | efried | mriedem: 1) that's not implemented yet, but 2) even when it is, that gels just fine with this, precisely because it's being node by virt.powervm.update_provider_tree and *not* out of band. | |
| 20:05:25 | efried | s/node/done/ | |
| 20:05:41 | mriedem | ok i thought it was to handle some out of band thing | |
| 20:06:04 | mriedem | but it was awhile ago and i've been high on ether since then | |
| 20:06:35 | sean-k-mooney | well in additon to the sig hup stuff + disableing the cache refush via config =0 if we inject a sleep(random(refersh interval)) seconds to the specific periodic task once the jitter should sperad out the update over the entire interval smoothly on average | |
| 20:07:36 | sean-k-mooney | so for those that dont turn this off the same amount of update to placement will happen jsut not all at once every x seconds | |
| 20:10:25 | mriedem | i imagine cern would appreciate a way to disable the refresh altogether since they are already doing that out of tree | |
| 20:11:37 | efried | I'm working something up now. | |
| 20:11:57 | sean-k-mooney | efried: code or ml post or spec | |
| 20:13:05 | efried | sean-k-mooney: code | |
| 20:13:19 | efried | If it gets traction, I can spec it. | |
| 20:13:56 | sean-k-mooney | cool | |
| 20:15:03 | sean-k-mooney | mriedem: by the way i know your busy with other stuff but do you plan to revive https://review.openstack.org/#/c/538498/ at somepoint | |
| 20:16:48 | mriedem | it's pretty low priority | |
| 20:18:18 | sean-k-mooney | mriedem: ok i stared it. so in the unlikely event i run out of things to do i might take a look at it if you dont get back to it. but ya ther are many things ahead of it | |
| 20:27:36 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: High Precision Event Timer (HPET) on x86 guests https://review.openstack.org/607989 | |
| 20:29:04 | mriedem | cfriesen: jackding: ^ i cleaned that up, +2 | |
| 20:30:01 | cfriesen | mriedem: sweet, thanks. any chance you could take another look at the vtpm one? | |
| 20:30:09 | cfriesen | sean-k-mooney: you too | |
| 20:30:29 | sean-k-mooney | cfriesen: am sure | |
| 20:30:47 | sean-k-mooney | cfriesen: im respining a patch but ill take a look after | |
| 20:30:49 | mriedem | ffs yes you know i'd love t | |
| 20:30:50 | mriedem | *to | |
| 20:30:58 | cfriesen | you're so sweet | |
| 20:32:16 | mriedem | i have my moments | |
| 20:32:21 | mriedem | once per quarter | |
| 20:32:27 | KeithMnemonic | mriedem: can someone help move this along https://review.openstack.org/#/c/611326/1 ? | |
| 20:32:59 | mriedem | KeithMnemonic: umm, melwitt and/or dansmith could probably hammer that through | |
| 20:34:46 | mriedem | KeithMnemonic: how far back do you need that fix? | |
| 20:34:47 | KeithMnemonic | thanks melwitt: dansmith: can you help here ? | |
| 20:34:56 | KeithMnemonic | just pike | |