| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-12 | |||
| 17:11:02 | efried | ComputeDriver.virtapi... | |
| 17:11:06 | dansmith | but, as I said, it matters only what mriedem thinks, so ... no point in even discussing until he opines | |
| 17:11:28 | aspiers | RPC bloat sounds like a very valid concern | |
| 17:16:23 | sean-k-mooney | RPC bloat being the reason not to put it in the instance? | |
| 17:16:40 | aspiers | that's how I interpreted what dansmith said | |
| 17:16:47 | efried | This virtapi thing is pretty small right now, basically just has wait_for_instance_event | |
| 17:16:51 | aspiers | yeah | |
| 17:16:59 | aspiers | also, I don't see allocations cached in the manager | |
| 17:17:06 | sean-k-mooney | dansmith: the allcoation should not change over the lifetime of the instance without a server action to modify it | |
| 17:17:10 | dansmith | efried: it used to have a bunch of services the driver used, but has whittled down | |
| 17:17:29 | sean-k-mooney | e.g. it will only cahnge on move operattion/resize/rebulds/or attaches/detaches | |
| 17:17:57 | dansmith | sean-k-mooney: in some future where I can allocate non-compute resources for the instance, the instance's allocation can change not from nova right? | |
| 17:18:02 | efried | but if dansmith likes the idea of extending it with allocations_for_instance, which we can cache in the manager, I'm down with that idea. | |
| 17:18:15 | dansmith | aspiers: no, we cache inventory AFAIK, but not allocations at the moment | |
| 17:18:26 | sean-k-mooney | dansmith: im notsure about that | |
| 17:18:43 | sean-k-mooney | are you thinking baout things like changing the bandwith allcoation via neutron | |
| 17:18:51 | sean-k-mooney | e.g. by changing the qos policy | |
| 17:19:17 | efried | aspiers: The inventory (basically all the provider tree information for all the hosts managed by this compute) is cached in the SchedulerReportClient today. But yeah, not the allocations. | |
| 17:19:40 | sean-k-mooney | i guess that could happen in the future | |
| 17:19:55 | dansmith | sean-k-mooney: I'm thinking of allocating things like a new volume, or even ephemeral things like networked secure enclave keys or something | |
| 17:20:17 | sean-k-mooney | dansmith: well a new volume would be a volume attach right | |
| 17:20:19 | efried | as long as those ^ things happen in nova, we're good, but yeah, if they happen outside of nova, weirdness. | |
| 17:20:22 | dansmith | sean-k-mooney: it also means our rpc messages and database footprint scales with whatever else we add into the allocation in the future | |
| 17:20:48 | efried | I think we have other problems to solve ("heal allocations"??) if we start allowing outside entities to modify instance allocations | |
| 17:20:56 | dansmith | sean-k-mooney: not always in the case of manilla or similar higher level services | |
| 17:21:01 | sean-k-mooney | that is true altheough we have a 64k userdata blob in there already | |
| 17:21:11 | efried | I'm happy with the virtapi solution personally. | |
| 17:21:18 | efried | I don't even think we need to bother caching it, at least initially. | |
| 17:21:28 | sean-k-mooney | for mania the filesystem is not own by the instnace | |
| 17:21:30 | efried | Since it's on demand and should be a pretty rare ask. | |
| 17:21:53 | sean-k-mooney | the instace can have acess too it but its not tied to the lifetime of the instnace | |
| 17:22:30 | sean-k-mooney | efried: i do kindof dislike the idea that if placement goes offline you cant reboot an instnace | |
| 17:22:32 | dansmith | sean-k-mooney: okay, you understand I'm talking about high-level services that sit on top of infra that would have per-instance allocations right? | |
| 17:22:45 | sean-k-mooney | dansmith: yes | |
| 17:23:01 | sean-k-mooney | and i get that it could be extendted that way in the future | |
| 17:24:51 | sean-k-mooney | without storing the allcoation somewhere in nova then placment becmoes a singel point of failture for starting or stroping vms if they use sev or vgpus | |
| 17:25:33 | dansmith | I definitely don't like that we'd depend on placement to reboot an instance or do other things, but I'd rather us not depend on the placement *data* in that case, | |
| 17:25:38 | dansmith | and not just solve it by replicating everything everywhere | |
| 17:26:01 | efried | yeah, when I found out we rebuild the domxml every time we reboot I was like whaaa? | |
| 17:26:16 | sean-k-mooney | well that makes sense in some ways | |
| 17:26:26 | efried | it must | |
| 17:26:30 | efried | in some ways | |
| 17:26:31 | sean-k-mooney | that is not really the root cause of the dependcy in the vgpu case | |
| 17:26:32 | dansmith | is this all coming up because we've already translated a request into a "will have SEV" and the only way we can think to know that later is persist the whole allocation? | |
| 17:26:46 | sean-k-mooney | no | |
| 17:26:46 | efried | dansmith: more or less | |
| 17:26:49 | efried | heh | |
| 17:26:53 | efried | anti-jinx | |
| 17:26:58 | sean-k-mooney | we can look at the request spec | |
| 17:27:03 | dansmith | efried: well, that's a bad reason to do that, IMHO | |
| 17:27:05 | efried | dansmith: It's not "the only way we can think to know that later" | |
| 17:27:06 | sean-k-mooney | and use the request for sev | |
| 17:27:17 | efried | we could reinvoke the logic we used to decide the first time | |
| 17:27:32 | dansmith | efried: point being, let's store what we need about that decision and not just couple the guts of the virt driver into the placement data | |
| 17:27:35 | efried | by looking at the flavor & image - which we have access to via the Instance obj, right? | |
| 17:27:46 | sean-k-mooney | ya we can compute it form the info in the request spec which these fucntion already have acess too | |
| 17:28:09 | dansmith | because the other thing is, allocations are somewhat tied to a microversion in placement, and while it seems hard to imagine the core structure changing, we've changed a lot of the external data structures of placement since we started | |
| 17:29:45 | sean-k-mooney | so once the comment is address i personally would have check the requestd_resouces object in the request spec https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L93-L100 | |
| 17:30:06 | sean-k-mooney | but we also have acess to the image and flavor via the instance yes | |
| 17:31:14 | dansmith | efried: re: [10:27:05] <efried>dansmith: It's not "the only way we can think to know that later" | |
| 17:31:36 | dansmith | efried: I know not the only way, but "the way", collapsed to "only" for dramatic effect | |
| 17:36:56 | sean-k-mooney | so the instance is already passed to all the places we need to check if the instace requested sev | |
| 17:37:47 | sean-k-mooney | so why dont we jsut hav ea single function that takes the instance object and determins if sev is needed and reuse that everywhere we care. | |
| 17:38:22 | sean-k-mooney | that could live in nova.virt.hardware.yp with the numa stuff? | |
| 17:38:58 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: Include direct-physical in compute manager events check https://review.opendev.org/664431 | |
| 17:38:58 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Revert resize: wait for events according to hybrid plug https://review.opendev.org/644881 | |
| 17:38:59 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [DNM] testing bug/1813789 revert resize events https://review.opendev.org/664442 | |
| 17:40:11 | sean-k-mooney | like is_realtime_enabled https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1436-L1438 | |
| 17:40:34 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: ComputeVirtAPI.allocations_for_instance https://review.opendev.org/664986 | |
| 17:40:39 | efried | aspiers: for grins, you can try this out ^ | |
| 17:40:45 | efried | dansmith: this what you had in mind? | |
| 17:41:23 | dansmith | efried: mostly | |
| 17:41:43 | efried | I explained why no caching in the commit message | |
| 17:41:46 | sean-k-mooney | efried: but is there any reason to have to check the allotion the athoritive info is stored in the image/flavor not the allcoation | |
| 17:42:09 | efried | sean-k-mooney: just to not have to duplicate that logic. | |
| 17:42:20 | efried | but in this case, no, not really. | |
| 17:42:24 | sean-k-mooney | an api call is a lot more expensive | |
| 17:42:28 | dansmith | is a single call to that less expensive than calling the placement? | |
| 17:42:29 | efried | Accessing the flavor/image meta from Instance is cheap | |
| 17:42:54 | efried | dansmith: assuming we already have the flavor & image available in the Instance object, it's all local, yeah. | |
| 17:43:00 | efried | well | |
| 17:43:06 | efried | some of it involves pawing at sysfs | |
| 17:43:13 | sean-k-mooney | we do | |
| 17:43:14 | dansmith | it better not, | |
| 17:43:24 | dansmith | because the api nodes' sysfs wouldn't be the same as compute | |
| 17:43:58 | sean-k-mooney | we dont need to look at sysfs for schduling | |
| 17:44:04 | efried | right, the compute pawed at sysfs to expose the inventory, so by virtue of having the MEM_ENC_CTX inventory at all, we know we did the sysfs stuff already. | |
| 17:44:09 | sean-k-mooney | we may or may not for xml generation | |
| 17:44:21 | dansmith | efried: ack, I see | |
| 17:44:25 | efried | but how would I know that on the compute without looking at placement? :P | |
| 17:44:42 | sean-k-mooney | efried: know what? | |
| 17:45:00 | efried | know that I can SEV | |
| 17:45:11 | sean-k-mooney | because its asked for in teh flavor/image | |
| 17:45:17 | efried | no | |
| 17:45:24 | sean-k-mooney | if we got to the compute node we know we were allocated everything we asked for | |
| 17:45:32 | efried | well, yes, okay | |
| 17:45:44 | efried | that's kind of a lot of assuming. | |
| 17:46:00 | efried | including assuming nothing changed on the host that makes it no longer work | |
| 17:46:02 | sean-k-mooney | assuming that if placment could not fulfil a request it returned no allcoation candates? | |
| 17:46:02 | efried | but that's reasonable. | |