| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-11-19 | |||
| 15:53:38 | sean-k-mooney | vcpu_pin_set is nothign to do with pinning | |
| 15:53:39 | stephenfin | nope. That _should_ have been the case, but it wasn't | |
| 15:53:45 | gibi | dansmith: yeah, that make sense | |
| 15:53:49 | sean-k-mooney | its the set of host you can boot a vm on | |
| 15:53:54 | sean-k-mooney | *cpus | |
| 15:53:58 | sean-k-mooney | if unset its all cpus | |
| 15:53:59 | stephenfin | artom: within the constraints of NUMA, yes | |
| 15:54:16 | stephenfin | pinning means NUMA, which means a guest NUMA node is constrained to a host NUMA node | |
| 15:54:31 | stephenfin | yeah, what sean-k-mooney said | |
| 15:54:49 | artom | Aha, thanks for bringing me up to speed | |
| 15:55:04 | stephenfin | I said we should have required it because pinning to e.g. host core 0 isn't a great idea, but there's no relationship between the option and pinning, as sean-k-mooney says | |
| 15:55:19 | artom | Well, there is if it's set :P | |
| 15:55:47 | stephenfin | sure, but unpinned instances booted to the same host will be bound to that set too | |
| 15:55:47 | artom | Are the semantics the same with cpu_dedicated_set? IOW, if that's None, can we still pin to anything? | |
| 15:56:08 | artom | Ah, it was just the set of CPUs usable by instances | |
| 15:56:09 | stephenfin | artom: at the moment, yes, because of upgrade reasons | |
| 15:56:09 | gibi | dansmith: having just a controller config enough? or we need a separate scheduler config, conductor config, etc? | |
| 15:56:10 | artom | Of any kind | |
| 15:56:22 | artom | I feel like I should know this stuff | |
| 15:56:24 | stephenfin | yes, exactly. Not specific to pinning policy | |
| 15:56:28 | gibi | dansmith: like the scheduler only need the API database I guess | |
| 15:56:35 | dansmith | gibi: well, with cells there needs to be some differences, that's why we're here | |
| 15:56:59 | stephenfin | artom: once we drop support for vcpu_pin_set, you'll have to define 'cpu_dedicated_set' | |
| 15:57:05 | sean-k-mooney | artom: vcpu_pin_set applied to unpinned instance too | |
| 15:57:14 | sean-k-mooney | we set it in the xml | |
| 15:57:36 | sean-k-mooney | artom: which si still a gap in live migration by the way | |
| 15:57:44 | sean-k-mooney | we dont update it for non numa guests | |
| 15:58:10 | stephenfin | I have a draft patch, but I can't submit it because doing so will require removal of the reshape. I'm hoping bauzas gets that offline reshape spec sorted this cycle | |
| 15:58:32 | bauzas | my spec is still open | |
| 15:58:59 | sean-k-mooney | stephenfin: well you mean once we drop the fallback query | |
| 15:59:08 | gibi | nova meeting starts in 1 minute | |
| 15:59:13 | stephenfin | sean-k-mooney: right | |
| 15:59:32 | stephenfin | which we'll do at the same time as we drop vcpu_pin_set | |
| 15:59:32 | sean-k-mooney | once that is gon you can only boot pinned vms with pcpus | |
| 15:59:38 | stephenfin | correct | |
| 15:59:40 | sean-k-mooney | yep | |
| 15:59:51 | sean-k-mooney | but even removing vcpu_pin_set would not be enough | |
| 16:00:21 | sean-k-mooney | since without cpu_dedicated_set but with the fallback enabeld you can boot pinned gusts with vcpus | |
| 16:00:34 | sean-k-mooney | so its really the fallback that is imporant | |
| 16:01:23 | artom | sean-k-mooney, oh yeah, I remember the live migration for cpu_shared_set | |
| 16:01:28 | artom | Should get to it, at some point :P | |
| 16:04:52 | sean-k-mooney | artom: is this reated to a bug or something by the way | |
| 16:04:59 | sean-k-mooney | or just better testing | |
| 16:05:00 | artom | sean-k-mooney, the whitebox CI failures | |
| 16:05:08 | artom | I haven't managed to reproduce the delete failure | |
| 16:05:10 | sean-k-mooney | oh well this is not a bug | |
| 16:05:16 | artom | (In functional tests, at any rate) | |
| 16:05:34 | sean-k-mooney | it might be a missconfiguration or a bug in the white box tests | |
| 16:05:39 | artom | But it's happening because it's deleting the pinned instance when cpu_dedicated_set is None | |
| 16:05:45 | artom | So it complains about unpinning CPUs | |
| 16:05:48 | artom | Well... | |
| 16:05:56 | sean-k-mooney | anytime we define cpu_dedicated_set you should also disable the fallback | |
| 16:06:06 | sean-k-mooney | we likely are not doing the latter | |
| 16:06:09 | artom | We don't define it | |
| 16:06:18 | sean-k-mooney | its enabled by default | |
| 16:06:32 | artom | I mean cpu_dedicated_set | |
| 16:06:42 | artom | What I'm seeing is the test_resize_unpinned_server_to_pinned tearDown is failing | |
| 16:06:51 | artom | Because it can't delete the pinned instance | |
| 16:07:08 | artom | Because nova.exception.CPUUnpinningInvalid: CPU set to unpin [2, 6] must be a subset of pinned CPU set [] | |
| 16:07:46 | artom | AFAICT there's no race or anything, it's just booting the resized pinned instance on the host without cpu_dedicated_set or vcpu_pin_set configured | |
| 16:07:54 | artom | Pinning it to 2 random CPUs | |
| 16:07:59 | artom | Then complaining when it can't delete it | |
| 16:08:11 | artom | Which... does kinda smell like a minor bug... | |
| 16:08:11 | sean-k-mooney | well we need delete teh vms before we reconfigure | |
| 16:08:18 | artom | We *don't* reconfigure | |
| 16:08:31 | artom | This is all without any config changes or service restarts | |
| 16:08:46 | sean-k-mooney | no i mean im not sure we are waiting for all vms from the previous test to be deleted | |
| 16:09:08 | artom | AFAICT, yes | |
| 16:09:54 | sean-k-mooney | artom: we should never see nova.exception.CPUUnpinningInvalid: CPU set to unpin [2, 6] must be a subset of pinned CPU set [] | |
| 16:10:12 | sean-k-mooney | if we do it means the config have chagned | |
| 16:10:31 | sean-k-mooney | or we are hitting the race which stephen fixed | |
| 16:10:40 | artom | But... what happens in the case of having booted an instance with pinned CPUs on a host without cpu_dedicated_set configured? | |
| 16:10:46 | artom | And then deleting that | |
| 16:10:52 | artom | Wouldn't we expect that error? | |
| 16:10:57 | sean-k-mooney | if its still not set its fine | |
| 16:11:07 | artom | Or does it use the database and not the config? | |
| 16:11:17 | sean-k-mooney | its not vallide to add cpu_dedicate_set to a host with vms | |
| 16:11:22 | artom | I know | |
| 16:11:26 | artom | That's not what we're doing here | |
| 16:11:38 | sean-k-mooney | can you point me to the test | |
| 16:11:43 | artom | sean-k-mooney, gmeet? :) | |
| 16:11:49 | sean-k-mooney | sure | |
| 16:13:03 | artom | sean-k-mooney, https://meet.google.com/eqm-jjcx-fjw | |
| 16:13:04 | sean-k-mooney | im going to make a cup of coffee but say 15 past | |
| 16:13:15 | sean-k-mooney | ill join when i get back | |
| 16:13:20 | artom | Take your time :) | |
| 16:16:24 | sean-k-mooney | just back :) | |
| 16:24:21 | dansmith | gibi: the distro-ness also affects our ability to describe what should go in each config file, | |
| 16:24:42 | gibi | dansmith: do you have an example? | |
| 16:25:04 | dansmith | gibi: because the problem on the ML is making all the distro packages work both across multiple nodes, as well as AIO, and currently there's no nova-conductor-cellN vs nova-superconductor service(s) | |
| 16:25:23 | dansmith | gibi: so I'm not quite sure how to write this to make sense for both | |
| 16:26:04 | dansmith | gibi: I'm going to try, but we'll probably need some input from people like owalsh .. maybe that means we need to use something other than the install doc to describe things, I dunno | |
| 16:26:14 | dansmith | writing docs is hard :( | |
| 16:27:13 | gibi | so the problem is that distros map packages to services but the nova-conductor package needs to be mapped to two type of services with different config? | |
| 16:27:52 | dansmith | yeah | |
| 16:27:58 | dansmith | and also, | |
| 16:28:02 | owalsh | hey | |
| 16:28:05 | dansmith | and this is what he was complaining about, | |
| 16:28:15 | dansmith | normally they'd all point to nova.conf, assuming they'd be on different hosts, | |
| 16:28:39 | dansmith | but if they're all on the same host, I'd have to describe changing/overriding unit files to get the right config to conductor vs. api, etc | |