| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-12 | |||
| 17:44:25 | efried | but how would I know that on the compute without looking at placement? :P | |
| 17:44:42 | sean-k-mooney | efried: know what? | |
| 17:45:00 | efried | know that I can SEV | |
| 17:45:11 | sean-k-mooney | because its asked for in teh flavor/image | |
| 17:45:17 | efried | no | |
| 17:45:24 | sean-k-mooney | if we got to the compute node we know we were allocated everything we asked for | |
| 17:45:32 | efried | well, yes, okay | |
| 17:45:44 | efried | that's kind of a lot of assuming. | |
| 17:46:00 | efried | including assuming nothing changed on the host that makes it no longer work | |
| 17:46:02 | efried | but that's reasonable. | |
| 17:46:02 | sean-k-mooney | assuming that if placment could not fulfil a request it returned no allcoation candates? | |
| 17:46:23 | sean-k-mooney | that kind of fundemental no? | |
| 17:46:32 | dansmith | definitely don't think that being sent an instance with sev required is good enough to assume we have it | |
| 17:46:36 | dansmith | because we don't know what the scheduler policy is | |
| 17:46:41 | dansmith | or if that node is old | |
| 17:46:49 | efried | or got force migrated? | |
| 17:46:52 | dansmith | or if the request has been sitting in a rabbit queue across config changes | |
| 17:47:04 | sean-k-mooney | well the virt direver is going to check compatiblity | |
| 17:47:14 | sean-k-mooney | right | |
| 17:47:32 | efried | so then yeah, in order to truly fulfill "can and should do SEV" I would need to check the flavor/image *and* sysfs, every time. | |
| 17:47:45 | efried | which is still going to be cheaper than calling placement across the wire | |
| 17:47:58 | sean-k-mooney | well libvirt capablitys api | |
| 17:48:08 | sean-k-mooney | rather then sysfs but same effect | |
| 17:48:30 | efried | yeah, using "sysfs" to encompass "introspecting the host in whatever way is necessary" | |
| 17:48:49 | efried | which should be all "local" one way or another. | |
| 17:49:09 | efried | "local" to the system; not necessarily local to whatever's in process memory already, which would be even better. | |
| 17:51:37 | sean-k-mooney | so back to https://review.opendev.org/#/c/664420/4/nova/scheduler/request_filter.py | |
| 17:52:42 | sean-k-mooney | if we just move thos three funtion into nova/virt/hardware.py wiht all the cpu pinning and numa stuff and just call | |
| 17:52:43 | dansmith | artom: I think maybe too much got removed, but see my comments and hopefully prove me wrong | |
| 17:53:16 | artom | dansmith, looking | |
| 17:53:34 | sean-k-mooney | required_encryped_memory support in both the dirver and filter that shoudl be fine right | |
| 17:53:36 | efried | sean-k-mooney: that and the sysfs pawing part, yes. Not sure if hardware.py is the right place | |
| 17:54:18 | sean-k-mooney | i think this is similarly to hugepages,realtime,cpupinning and numa | |
| 17:54:28 | sean-k-mooney | all of which are in the hardware.py module | |
| 17:54:47 | efried | does hardware.py also do the flavor interpretation bits associated with that? | |
| 17:54:57 | sean-k-mooney | efried: yes | |
| 17:55:04 | efried | I kind of object to hardware.py not being nova/virt/libvirt/hardware.py | |
| 17:55:18 | efried | but that soapbox is gathering dust in my closet at this point. | |
| 17:55:19 | sean-k-mooney | its used in other driver too | |
| 17:55:28 | sean-k-mooney | its used in the hyperv driver | |
| 17:55:33 | sean-k-mooney | and its used in the api | |
| 17:55:39 | sean-k-mooney | and the scheduler | |
| 17:55:56 | aspiers | I can never drop bad habits :-( | |
| 17:56:20 | sean-k-mooney | the trick is to replace them with other sligly less bad habits | |
| 17:56:28 | aspiers | true | |
| 17:57:49 | sean-k-mooney | anyway it could go somewhere else | |
| 17:57:51 | artom | Agile self-improvement ;) | |
| 17:58:34 | efried | aspiers: not sure if you've been following along, but I think the consensus is to centralize the methods that do a) flavor/image parsing/validation, and b) host capability introspection; and call those from all the places (update_provider_tree calls b to expose inventory; request filter calls a to add to the request; virt driver lifecycle ops call a & b to decide whether SEV-y things should be done or not) | |
| 17:59:05 | aspiers | OK | |
| 17:59:28 | efried | but if you want to just play around, I made this for you: https://review.opendev.org/664986 | |
| 18:00:13 | efried | ...for lifecycle ops to use to look for MEM_ENC_CTX in the alloc to bypass at least a). | |
| 18:02:36 | mriedem | umm, i just got back | |
| 18:02:53 | mriedem | not going to read all of the scrollback | |
| 18:03:09 | mriedem | without knowing more, i'd say i don't want to replicate allocation data per instance in nova | |
| 18:03:31 | mriedem | source of truth is the external service and all that | |
| 18:05:46 | sean-k-mooney | mriedem: the context was how to determin if sev is need. efried summerised where we landed a few lines up | |
| 18:07:00 | mriedem | isn't sev a required trait on the flavor? | |
| 18:07:16 | sean-k-mooney | its a resouce class in the flavor | |
| 18:07:17 | mriedem | if so, that is already information stored on the instance | |
| 18:07:23 | mriedem | ok either way | |
| 18:07:35 | efried | "stored on the instance" you mean via the flavor (and/or image in this case)? | |
| 18:07:42 | sean-k-mooney | yes efried didnt want to duplicate the logic between the filter an the virt driver | |
| 18:07:48 | mriedem | not the image | |
| 18:07:52 | mriedem | if it's a resource class it's not on the image | |
| 18:08:00 | mriedem | but yes if it's on the flavor it's embedded on the instance | |
| 18:08:03 | efried | The flavor doesn't use a resource class directly | |
| 18:08:06 | mriedem | instance.flavor.extra_spec | |
| 18:08:10 | mriedem | it's an extra spec | |
| 18:08:11 | mriedem | right? | |
| 18:08:15 | efried | In this case we decide whether to SEV based on flavor extra spec *or* image meta prop | |
| 18:08:17 | mriedem | resources:SEV=1 | |
| 18:08:25 | mriedem | what image meta prop? | |
| 18:08:30 | mriedem | i thought the image only had traits? | |
| 18:08:33 | efried | no, no explicit placement-ese in flavor or image | |
| 18:08:52 | efried | hw:memory_encryption=true <= flavor | |
| 18:08:52 | efried | hw_memory_encryption=true <= img | |
| 18:08:55 | mriedem | i have been deliberately ignoring / avoiding the sev talk going on in here for the last several months so idk wtf we are now | |
| 18:09:04 | efried | (I may have spelled it wrong, but that's the idea) | |
| 18:09:17 | efried | request filter interprets those into placement-ese | |
| 18:09:28 | mriedem | as a trait? or a resource class? | |
| 18:09:29 | mriedem | or both? | |
| 18:09:33 | openstackgerrit | Eric Fried proposed openstack/nova master: Raise if flavor and image disagree on hide_hypervisor_id https://review.opendev.org/663365 | |
| 18:09:37 | efried | a resource class | |
| 18:09:46 | mriedem | with what amount? 1 | |
| 18:09:46 | efried | required=MEM_ENCRYPTION_CONTEXT:1 | |
| 18:09:47 | mriedem | ? | |
| 18:09:59 | sean-k-mooney | yes | |
| 18:10:03 | efried | we got here because there's a limited number of SEVs you can do on a host | |
| 18:10:09 | efried | and | |
| 18:10:11 | mriedem | gimme all the sevs | |
| 18:10:29 | efried | we want the solution to be generic for memory encryption technologies, so e.g. MKTME can play in this space. | |
| 18:10:43 | mriedem | is mktme that band from the 90s? | |
| 18:10:47 | mriedem | with the cool album covers? | |
| 18:10:49 | efried | yes | |
| 18:10:50 | aspiers | :) | |
| 18:11:15 | aspiers | whereas SEV is just terrible Euro-pop | |
| 18:11:19 | mriedem | let me guess, mktme is intel something | |
| 18:11:35 | aspiers | yes, Intel multi-key total memory encryption | |
| 18:12:27 | edleafe | which sed | |
| 18:12:33 | edleafe | doh! | |