| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-04 | |||
| 15:24:19 | kashyap | Oh, actually zhu.boxiang has picked it up lately | |
| 15:25:20 | cfriesen | kashyap: yeah, was just going to say. :) | |
| 15:30:23 | sean-k-mooney | dansmith: i think the cross rpc migration is working. i have not looked at the logs to confim but it succeed via horizon | |
| 15:30:49 | dansmith | sean-k-mooney: okay, with a numa flavor or non-numa flavor? | |
| 15:31:00 | sean-k-mooney | with cpu pinning enabled | |
| 15:31:03 | sean-k-mooney | so numa | |
| 15:31:31 | dansmith | okay, so the behavior we expect is for it to work, not claim, and just translate the topo directly over to the other side, for better or worse, correct? | |
| 15:31:40 | dansmith | artom: ^ | |
| 15:31:50 | openstackgerrit | Adam Spiers proposed openstack/nova master: Ensure q35 machine type is used when booting with SEV https://review.opendev.org/680065 | |
| 15:31:58 | aspiers | efried, kashyap, sean-k-mooney: ^^^ this implements what we agreed yesterday | |
| 15:32:01 | artom | dansmith, yep, aka, what we have now | |
| 15:32:21 | dansmith | yeah, okay | |
| 15:32:24 | kashyap | aspiers: Already reading the code for a couple of mins (while on a call) | |
| 15:32:34 | aspiers | kashyap: awesome, I just uploaded PS2 though | |
| 15:32:42 | aspiers | realised I missed a few little bits out | |
| 15:32:50 | sean-k-mooney | dansmith: do i need to test the sucess case or was it the revert case that you were worried about | |
| 15:32:52 | dansmith | sean-k-mooney: so, one more test you could do is pop all the patches off from one node and just validate that you can still migrate between them (with the version pinned) | |
| 15:32:53 | artom | dansmith, because as soon as we start claiming, it's all or nothing, we need to handle applying the migration context, dropping the claim on rollback, etc | |
| 15:33:15 | dansmith | sean-k-mooney: yeah, that'd be good too | |
| 15:33:19 | kashyap | aspiers: I see that you've put the check in hardware.py | |
| 15:33:34 | aspiers | kashyap: yes. This new approach is actually quite a lot cleaner | |
| 15:33:46 | aspiers | so I'm glad we had the discussions | |
| 15:34:19 | kashyap | aspiers: Okay, then. Because there's a prereq on hardware feature, hardware.py? | |
| 15:35:11 | aspiers | kashyap: it's just adding another check alongside the existing 2 extra specs / image props checks | |
| 15:35:17 | openstackgerrit | Adam Spiers proposed openstack/nova master: Apply SEV-specific guest config when SEV is required https://review.opendev.org/644565 | |
| 15:35:21 | stephenfin | mriedem: What exception type should I be raising in ComputeManager.init_host if my checks fail. I'm thinking 'InternalError' since it's an operator misconfiguration | |
| 15:35:23 | sean-k-mooney | dansmith: ok so before i check artom code vs master or the revert case. was there somehting in partcal you wanted me to grab form the logs or db | |
| 15:35:58 | kashyap | aspiers: Right; so I can probably put my _guest_machine_type_for_secure_boot() still in utils.py. | |
| 15:36:06 | dansmith | sean-k-mooney: just want to make sure it's not doing numa migration I guess.. there are a couple logs that indicate it's doing the new stuff | |
| 15:36:18 | aspiers | kashyap: now I realise why you were asking that | |
| 15:36:25 | kashyap | aspiers: :-) | |
| 15:36:26 | dansmith | sean-k-mooney: you don't really have to pastebin, I trust you, just check to make sure you don't see those on either side | |
| 15:36:35 | artom | sean-k-mooney, dansmith, presence of LibvirtLiveMigrateNUMAInfo is a big sign | |
| 15:36:46 | dansmith | artom: like this one: https://review.opendev.org/#/c/634606/73/nova/virt/libvirt/driver.py@7465 | |
| 15:36:51 | aspiers | kashyap: yeah I guess yours could still go in utils.py | |
| 15:36:51 | dansmith | artom: yep | |
| 15:37:03 | sean-k-mooney | artom: are you expecting it to regenerat the xml when the RPC version is pinned? | |
| 15:37:12 | artom | We log the migrate_data all over the place, so if that's not in it, we haven't NUMA live-migrated | |
| 15:37:16 | artom | sean-k-mooney, no | |
| 15:37:22 | sean-k-mooney | ok ill check that so | |
| 15:37:30 | mriedem | stephenfin: i see the libvirt driver init_host raises Invalid, but we already have a InvalidConfiguration so that seems more appropriate | |
| 15:37:32 | artom | Any RPC pin on either source, dest or cond should cause the current half-broken behaviour | |
| 15:37:34 | mriedem | it doesn't really matter though | |
| 15:37:35 | kashyap | aspiers: It's a similar method. But in your case, I see you're clubbing it all together with the relevant mem encryption- / SEV-related checks | |
| 15:37:38 | stephenfin | ack | |
| 15:37:48 | sean-k-mooney | the config i have would require the xml to be update to be correct so i shoudl see the xml updated in only 1 direction | |
| 15:38:01 | sean-k-mooney | or not at all | |
| 15:38:25 | aspiers | kashyap: right | |
| 15:39:05 | kashyap | aspiers: I'll just Depends-On yours for the exception class | |
| 15:39:23 | aspiers | kashyap: or just make it the child commit | |
| 15:39:31 | aspiers | kashyap: isn't Depends-On more for cross-repo stuff? | |
| 15:39:55 | sean-k-mooney | artom: dansmith: pinned source => unpinned dest no xml updates on migration | |
| 15:40:26 | kashyap | aspiers: True, Depends-On is indeed normally used for cross-repo things | |
| 15:41:28 | sean-k-mooney | artom: and i get erros in the compute log because the vm xml is invalid for the new host vcpu_pin_set so that is working as we expect | |
| 15:41:53 | artom | "and i get erros in the comput" *panic* "so that is working as we expect" *relief | |
| 15:42:04 | artom | u kill me man | |
| 15:42:35 | sean-k-mooney | its compliing the vm cpu pinning is not valid for the host in the periodici task that caulates resouce usage | |
| 15:42:51 | artom | Which is a known bug I think | |
| 15:42:56 | sean-k-mooney | its compliing the vm cpu pinning is not valid for the host in the periodici task that caulates resouce usage | |
| 15:43:05 | sean-k-mooney | wrong terminal | |
| 15:43:34 | sean-k-mooney | well its not really a bug so much as use say hay you vm pinning is invalid | |
| 15:45:24 | artom | I mean there's a launchpad bug for it somewhere | |
| 15:46:25 | sean-k-mooney | ya there is a patch that sugges we skip updating the resouse if its invalid | |
| 15:46:35 | sean-k-mooney | which i think we both nack'd | |
| 15:47:06 | sean-k-mooney | artom: dansmith same behavior for new source to old dest. no xml update | |
| 15:47:17 | artom | sean-k-mooney, yep | |
| 15:47:20 | dansmith | cool | |
| 15:47:47 | artom | dansmith, well, figured out why my func tests weren't doing the expected thing | |
| 15:47:57 | artom | I was stubbing stuff on the ClientRouter, which is a global singleton | |
| 15:48:06 | artom | So *all* 3 services got capped | |
| 15:48:07 | dansmith | ah yeah | |
| 15:49:18 | artom | I shouldn't hit the same problem if I replace the actual compute_rpcapi objects, though, right? | |
| 15:49:40 | dansmith | you can get access to the managers in the services you start right? | |
| 15:49:48 | dansmith | I think | |
| 15:49:48 | artom | I couldn't figure out how to either cap the objects directly, or create new ones and replace. Looks like I have no choice to do that now. | |
| 15:49:48 | dansmith | so manager.rpcapi = $newthing | |
| 15:50:09 | artom | Yeah, it's "$newthing" that needs thinking about | |
| 15:50:49 | artom | There's no __init__, so how do I cap it? | |
| 15:50:52 | mriedem | compute = self.start_service('compute', host='host1') | |
| 15:50:58 | mriedem | compute.manager.compute_rpcapi = my_stub | |
| 15:51:25 | mriedem | or something like that | |
| 15:51:37 | mriedem | stub the compute_rcpapi on one of the services in the func test | |
| 15:51:43 | dansmith | that ^ | |
| 15:52:23 | dansmith | I think you could also not stub the full rpcapi, and just stub out can_send_version on the one it sets up for you | |
| 15:53:17 | openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | |
| 15:53:19 | artom | can_send_version is on the client though | |
| 15:53:20 | sean-k-mooney | so ... there is one issue. if we live migrate the server to a host where the pinning is invalide. | |
| 15:53:26 | sean-k-mooney | then we cold migrate it | |
| 15:53:35 | sean-k-mooney | the cold migrtion fails in resize confim | |
| 15:53:58 | artom | sean-k-mooney, is pinning involved, or master to master? | |
| 15:54:08 | sean-k-mooney | yes | |
| 15:54:15 | artom | Yes to...? | |
| 15:54:17 | sean-k-mooney | nova.exception.CP│ | |
| 15:54:19 | sean-k-mooney | UUnpinningUnknown: CPU set to unpin [12, 15, 4, 7] must be a subset of known CPU set [] | |
| 15:54:40 | artom | Wouldn't that be a latent thing? | |
| 15:54:54 | sean-k-mooney | im going to triple check but i think cold migration of a pinned instnace is fine | |
| 15:55:19 | sean-k-mooney | jsut not one that has been live migrtation to a host with out regenrting the xml if the xml is invalid for that host | |
| 15:55:48 | artom | That's still a different bug though, and why we moved to block numa live migration by default? | |
| 15:55:51 | sean-k-mooney | i can check but i think this would happen on master too | |
| 15:57:00 | sean-k-mooney | by the way i was kind of expecting that might fail im just reporing what im finding as i try to break it | |