| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-04 | |||
| 15:42:35 | sean-k-mooney | its compliing the vm cpu pinning is not valid for the host in the periodici task that caulates resouce usage | |
| 15:42:51 | artom | Which is a known bug I think | |
| 15:42:56 | sean-k-mooney | its compliing the vm cpu pinning is not valid for the host in the periodici task that caulates resouce usage | |
| 15:43:05 | sean-k-mooney | wrong terminal | |
| 15:43:34 | sean-k-mooney | well its not really a bug so much as use say hay you vm pinning is invalid | |
| 15:45:24 | artom | I mean there's a launchpad bug for it somewhere | |
| 15:46:25 | sean-k-mooney | ya there is a patch that sugges we skip updating the resouse if its invalid | |
| 15:46:35 | sean-k-mooney | which i think we both nack'd | |
| 15:47:06 | sean-k-mooney | artom: dansmith same behavior for new source to old dest. no xml update | |
| 15:47:17 | artom | sean-k-mooney, yep | |
| 15:47:20 | dansmith | cool | |
| 15:47:47 | artom | dansmith, well, figured out why my func tests weren't doing the expected thing | |
| 15:47:57 | artom | I was stubbing stuff on the ClientRouter, which is a global singleton | |
| 15:48:06 | artom | So *all* 3 services got capped | |
| 15:48:07 | dansmith | ah yeah | |
| 15:49:18 | artom | I shouldn't hit the same problem if I replace the actual compute_rpcapi objects, though, right? | |
| 15:49:40 | dansmith | you can get access to the managers in the services you start right? | |
| 15:49:48 | dansmith | so manager.rpcapi = $newthing | |
| 15:49:48 | artom | I couldn't figure out how to either cap the objects directly, or create new ones and replace. Looks like I have no choice to do that now. | |
| 15:49:48 | dansmith | I think | |
| 15:50:09 | artom | Yeah, it's "$newthing" that needs thinking about | |
| 15:50:49 | artom | There's no __init__, so how do I cap it? | |
| 15:50:52 | mriedem | compute = self.start_service('compute', host='host1') | |
| 15:50:58 | mriedem | compute.manager.compute_rpcapi = my_stub | |
| 15:51:25 | mriedem | or something like that | |
| 15:51:37 | mriedem | stub the compute_rcpapi on one of the services in the func test | |
| 15:51:43 | dansmith | that ^ | |
| 15:52:23 | dansmith | I think you could also not stub the full rpcapi, and just stub out can_send_version on the one it sets up for you | |
| 15:53:17 | openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | |
| 15:53:19 | artom | can_send_version is on the client though | |
| 15:53:20 | sean-k-mooney | so ... there is one issue. if we live migrate the server to a host where the pinning is invalide. | |
| 15:53:26 | sean-k-mooney | then we cold migrate it | |
| 15:53:35 | sean-k-mooney | the cold migrtion fails in resize confim | |
| 15:53:58 | artom | sean-k-mooney, is pinning involved, or master to master? | |
| 15:54:08 | sean-k-mooney | yes | |
| 15:54:15 | artom | Yes to...? | |
| 15:54:17 | sean-k-mooney | nova.exception.CP│ | |
| 15:54:19 | sean-k-mooney | UUnpinningUnknown: CPU set to unpin [12, 15, 4, 7] must be a subset of known CPU set [] | |
| 15:54:40 | artom | Wouldn't that be a latent thing? | |
| 15:54:54 | sean-k-mooney | im going to triple check but i think cold migration of a pinned instnace is fine | |
| 15:55:19 | sean-k-mooney | jsut not one that has been live migrtation to a host with out regenrting the xml if the xml is invalid for that host | |
| 15:55:48 | artom | That's still a different bug though, and why we moved to block numa live migration by default? | |
| 15:55:51 | sean-k-mooney | i can check but i think this would happen on master too | |
| 15:57:00 | sean-k-mooney | by the way i was kind of expecting that might fail im just reporing what im finding as i try to break it | |
| 15:57:26 | artom | Totally, this sort of testing is great | |
| 15:57:51 | openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | |
| 15:58:07 | artom | Wait, would live migrating to an invalid pinning situation go through? | |
| 15:58:07 | openstackgerrit | Merged openstack/nova master: Refactor MigrationTask._execute https://review.opendev.org/679651 | |
| 15:58:17 | openstackgerrit | Merged openstack/nova master: unit test: do not fill rp mapping for failed re-schedule https://review.opendev.org/679653 | |
| 15:58:34 | sean-k-mooney | ok cold migration of a new instance works fine | |
| 15:58:47 | artom | I guess as long as the host CPUs are online, we don't check whether they're part of vcpu_pin_set during the LM | |
| 15:58:59 | sean-k-mooney | ill check reverting to master on the compute node then master only | |
| 15:59:01 | artom | So it'd make sense for that to fail in a subsequent operation | |
| 15:59:46 | sean-k-mooney | artom: right libvirt checks they are are a thing but libvirt is not aware of the vcpu_pin_set | |
| 16:00:10 | sean-k-mooney | and i have intentionally configred theyse hsot so the vcpu_pin_sets on each host does not overlap | |
| 16:00:44 | artom | Right, to make sure the old broken behaviour is happening | |
| 16:01:13 | sean-k-mooney | yes | |
| 16:01:25 | sean-k-mooney | and to give a really hard edgecase | |
| 16:02:05 | aspiers | kashyap: I've just built an image with an experimental kernel and grub secure boot fails due to "invalid signature" | |
| 16:02:16 | aspiers | kashyap: any idea how to tell it the signature is trustworthy? | |
| 16:02:17 | sean-k-mooney | one server is set up to only allow resouce on numa node 0 the other for node 1 only and it have the cpu_shared_set configure to use the opistie numa node to the vcpu_pin_set | |
| 16:03:07 | kashyap | aspiers: Yeah, "experimental kernal" is the keyword :-) | |
| 16:03:55 | sean-k-mooney | aspiers: it has not been sgined with a key that is in the secure boot keystore would be my guess | |
| 16:04:10 | kashyap | So, there is a manual way to make it "trustworthy" | |
| 16:04:27 | kashyap | aspiers: But I don't see a document for it. Last I checked with an admin who does it, said it's quite tedious | |
| 16:05:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add delete_on_termination to volume-attach API https://review.opendev.org/673133 | |
| 16:05:35 | sean-k-mooney | dansmith: artom os i have a proablem with 1 node on master and the other node on artoms code | |
| 16:05:56 | sean-k-mooney | on startup the agent on master code is raising "ServiceTooOld: This service is older (v39) than the minimum (v40) version of the rest of the deployment." | |
| 16:06:05 | donnyd | Is there a way to ask nova for a specific trait on an instance without configuring a flavor first? | |
| 16:06:17 | openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | |
| 16:06:28 | artom | sean-k-mooney, update the database I think | |
| 16:06:35 | kashyap | aspiers: If Secure Boot is not a reqiurement for your test, then I'd suggest to disable it | |
| 16:06:36 | sean-k-mooney | i guess i need to pin artoms code to the older version? | |
| 16:06:42 | aspiers | kashyap: how? | |
| 16:06:47 | mriedem | donnyd: required traits are defined on the flavor or image | |
| 16:06:50 | artom | sean-k-mooney, I hit that a few times when I git pulled a new service version, but the version in the database for the service was old | |
| 16:06:56 | sean-k-mooney | artom: well i didnt use the head of master. i used the version of master your code was bassed on | |
| 16:06:58 | kashyap | aspiers: You mean, "disable how"? | |
| 16:07:10 | aspiers | kashyap: yes, how do I disable it | |
| 16:07:13 | sean-k-mooney | e.g. this is the frist commit before your code | |
| 16:07:23 | dansmith | sean-k-mooney: you'll have to delete some service records | |
| 16:07:33 | sean-k-mooney | ah right | |
| 16:07:37 | dansmith | sean-k-mooney: that message is coming because you've got entries in the db that say everything should be upgraded already | |
| 16:08:07 | sean-k-mooney | if i just stop the agent and do a compute servcie delete that should fix it right | |
| 16:08:18 | sean-k-mooney | alternitivel i can restack that node | |
| 16:08:44 | donnyd | Is there any way to place a custom request in via the sdk? | |
| 16:08:44 | donnyd | The scenario is I use some sort of discovery to populate placement with traits, and I don't want to configure a flavor for each specific type | |
| 16:08:53 | kashyap | aspiers: Don't know top off my head; but let me check with a dev who does this more often | |
| 16:09:36 | mriedem | stephenfin: looking at https://review.opendev.org/#/c/621476/ alex is +2 and you asked for docs fixes in PS61 which are done now, so you might want to hit that again soonish, | |
| 16:09:51 | mriedem | stephenfin: especially b/c i'm +2 on https://review.opendev.org/#/c/673133/15 which is competing for the same microversion | |
| 16:09:51 | kashyap | aspiers: In the BIOS menu, it depends on your vendor | |
| 16:10:06 | aspiers | kashyap: we are talking about a VM in nova here, remember | |
| 16:10:16 | stephenfin | mriedem: Sure, I can do that before I leave | |
| 16:10:23 | aspiers | it's some virtual BIOS thing | |
| 16:11:39 | kashyap | aspiers: Right, it's a VM; a quick check -- you're using one of those enrolled "VARS" files shipped by your distro I guess? | |
| 16:12:34 | kashyap | aspiers: If so, the quickest way to disable it is to just start your guest with a non-enrolled (i.e. not signed by MS) "VARS" file. | |
| 16:12:40 | aspiers | kashyap: right, as per my comment on https://review.opendev.org/#/c/348394/ | |
| 16:13:00 | sean-k-mooney | we need to update osc to better support service deleteion | |
| 16:13:02 | aspiers | kashyap: in that final comment there is a list of all the "code" files | |
| 16:13:10 | aspiers | kashyap: which have corresponding "vars" files | |