| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2018-09-26 | |||
| 14:53:03 | openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_cyborg/2018/openstack_cyborg.2018-09-26-14.05.html | |
| 14:53:04 | openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_cyborg/2018/openstack_cyborg.2018-09-26-14.05.txt | |
| 14:53:05 | openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_cyborg/2018/openstack_cyborg.2018-09-26-14.05.log.html | |
| 14:53:06 | wangzhh | Bye | |
| 14:53:11 | Li_Liu | 88 | |
| 14:53:24 | xinran | bye | |
| 14:53:26 | Coco_gao | Bye | |
| 19:33:32 | Sundar | efried: Please ping me when you can | |
| 19:38:01 | efried | Sundar: Hi there. | |
| 19:42:33 | Sundar | Hi Eric, in the nova spec 603955, you said that having Cyborg claim a device by default (without explicit whitelisting) is a security risk. Can you elaborate? | |
| 19:45:02 | Sundar | Line 108 in https://review.openstack.org/#/c/603955 | |
| 20:26:41 | Sundar | @efried: I'll be around for another 90 minutes. I'll step out after that and should be back by 5 pm PDT. | |
| 20:27:31 | efried | sorry I missed your response earlier. ffr best to tag me | |
| 20:27:52 | efried | Not claiming, but exposing | |
| 20:28:34 | efried | If you expose, say, the controller that's handling the root disk of the management partition, and someone manages to attach it to the VM, your end user (the owner of the VM) could wreak havoc. | |
| 20:28:48 | efried | Sundar: ^^ :) | |
| 20:29:32 | Sundar | Ah, yes, will make sure to tag you :) | |
| 20:30:18 | Sundar | efried: My statement was "if the operator has installed and configured Cyborg drivers, he has explicitly enabled the devices managed by those drivers." | |
| 20:30:38 | Sundar | Why is that a security risk? | |
| 20:31:02 | efried | Sundar: that should be fine. | |
| 20:31:40 | efried | sorry | |
| 20:31:43 | efried | strike that. | |
| 20:31:56 | Sundar | efried: NP. Will respond in the spec. Thanks. | |
| 20:32:12 | efried | Sundar: as worded, it implies that by installing cyborg, you've enabled the devices | |
| 20:32:47 | efried | Sundar: Is the "and configured" part where I would list which devices I wanted cyborg to manage? | |
| 20:33:30 | Sundar | Well, "installed Cyborg drivers". Installing a driver is a dleiberate act, right? | |
| 20:33:41 | Sundar | *deliberate | |
| 20:33:59 | efried | Yes. But installing a driver capable of handling X does *not* mean that I want *all* instances of X to be handled by cyborg and made available for attachment to VMs. | |
| 20:34:35 | Sundar | I don't follow why that is a security risk. You may want to disable some, but that could be an explicit blacklist | |
| 20:35:11 | Sundar | If the admin doesn't want all those devices, why would he install them? | |
| 20:36:03 | efried | I'm no expert, but that's not how security works. | |
| 20:37:18 | Sundar | If some of those devices come pre-installed in the physical servers, like built-in NICs, I can understand. | |
| 20:37:56 | Sundar | I don;t think GPUs are pre-installed in data center class servers. (Desktops are a different story) | |
| 20:39:15 | efried | Sundar: I think it's less of a matter of what's *likely* to happen, and more about it just being a bad policy in general. | |
| 20:39:20 | efried | edmondsw: you're a security guy... | |
| 20:39:44 | efried | we're talking about whether, by installing a cyborg driver, you should automatically expose any device that driver can handle. | |
| 20:39:54 | efried | ...expose for attachability to VMs. | |
| 20:40:09 | efried | ...and if you want to reserve/restrict one, you have to explicitly blacklist it. | |
| 20:40:42 | edmondsw | yeah, that's probably not a great idea | |
| 20:41:03 | Sundar | efried, edmondsw: If some of those devices come pre-installed in the physical servers, like built-in NICs, I can understand the need to blacklist by default. That is not the case for accelerators in the data center. | |
| 20:41:57 | efried | I mean, I'm having a tough time coming up with a practical scenario where it could cause you a real problem, it just feels wrong. | |
| 20:42:43 | edmondsw | do we think there will never be a case where it makes sense for an accelerator to be owned by the hypervisor itself? | |
| 20:43:24 | Sundar | edmondsw: Yes. That could be an explicit Cyborg configuration too. | |
| 20:45:18 | Sundar | Isn't one of the pain points of PCI whitelisting that there is so much operator labor required? That may be unavoidable for common elements like NICs, but hopefully can be avoided in most cases for accelerators. | |
| 20:45:59 | edmondsw | but accelerators are going to become more and more common | |
| 20:46:54 | edmondsw | could we compromise with a conf setting that allows the user to specify whether they want whitelist-by-default behavior or not? | |
| 20:47:59 | Sundar | edmondsw: That seems reasonable. That is a single setting for an entire deployment. | |
| 20:49:13 | Sundar | We still need that to coexist with Nova's PCI whitelists. The accelerator's PCI ID may be in te white list, due to operator error, or because an upgrade moved the device from Nova to Cyborg, or whatever | |
| 20:49:59 | Sundar | We can help that by providing a tool that the operator runs, wherein he feeds the Nova's PCI whitelist file and the output says which PCI IDs are in conflict | |
| 20:50:23 | Sundar | edmondsw, efried: What do you think? | |
| 20:51:45 | efried | Sundar: That sounds like an ambitious idea, but I'm sure it would be welcomed if you could pull it off :) | |
| 20:52:25 | efried | TBC, coexisting with the nova PCI whitelist in the sense that the set of devices managed by each are mutually exclusive. | |
| 20:52:30 | edmondsw | I'm not sure every conflict would be an error... but they could flick that conf setting to *not* whitelist-by-default in cyborg if that's the case | |
| 20:53:18 | edmondsw | I put that badly... I meant it wouldn't necessarily be a nova error... could be that they need to blacklist it in cyborg | |
| 20:54:07 | Sundar | edmondsw: Sure. | |
| 20:55:08 | Sundar | efried: Do you see practical issues with such a tool? It seems to me that the tool is merely scanning CYborg's db for PCI IDs and the operator-provided PCI whitelist file for conflicts. I suppose wild cards need careful handling | |
| 20:55:36 | efried | heh, yeah, good luck duplicating the logic nova uses to process that whitelist. | |
| 20:57:29 | Sundar | efried: I'm probably being naive. How does Nova handle conflicts within the whitelist? Say one entry enables a device and another blacklists it. | |
| 20:57:54 | efried | Sundar: Don't know off the top of my head. | |
| 20:58:23 | Sundar | efried: NP. I nly intend this to be a helper tool, not some official way that is fool-proof | |
| 20:58:49 | efried | Here are the notes I scribbled down a couple years ago when I was pawing through the code trying to understand how it's "working": https://etherpad.openstack.org/p/powervm-pci-passthrough-notes | |
| 20:59:40 | efried | gosh, maybe it was only a year ago | |
| 21:02:43 | Sundar | efried: Thanks for the notes. | |
| 21:02:47 | Sundar | efried, edmondsw: Thanks for the good feedback. :) | |
| #openstack-cyborg - 2018-09-27 | |||
| 03:14:01 | openstackgerrit | Xinran WANG proposed openstack/cyborg master: Allocation/Deallocation API Specification https://review.openstack.org/597991 | |
| 19:11:04 | Sundar | efried: Please LMK when you are here. | |
| 19:11:18 | efried | Sundar: ō/ | |
| 19:11:55 | Sundar | efried: On the Nova spec, you correctly noted that I am not asking to include acceN in Placement queries | |
| 19:12:21 | Sundar | These additional properties are needed to specify bitstreams, functions and things that Nova doesn't care about | |
| 19:12:42 | efried | I understand | |
| 19:12:49 | Sundar | So, I have proposed that they be bundled with the request groups in the same way as resources or traots | |
| 19:13:29 | Sundar | So, if we stick to that idea, do you expect pushback from other Nova developers? What does it take to get this through? | |
| 19:14:06 | efried | I could see some people objecting to the fact that it *looks* like they're tied together | |
| 19:14:10 | efried | which, of course, they are | |
| 19:15:19 | efried | I think you're going to have the same problem gibi is trying to solve via https://review.openstack.org/#/c/597601/ -- there's no way to map a request group to a piece of an allocation. | |
| 19:15:45 | efried | But that's orthogonal to the design decision of using accN to denote such information. | |
| 19:17:13 | Sundar | The alternative is to split the device profile entries in each VAR and store it in Cyborg. The problem then is that the information gets duplicated. That can lead to issues with consistency. | |
| 19:17:49 | efried | Which information gets duplicated? | |
| 19:18:20 | efried | you mean include stuff like bitstream info in the device profile? | |
| 19:18:29 | efried | that seems... reasonable. | |
| 19:18:34 | efried | at least for phase 1 | |
| 19:19:29 | Sundar | For e.g., say the device profile says: { resources:ACCEL_GPU=2; trait:GPU_FOO=required}. Then we would create 2 separate VARs. Each one could contain a copy like: {resources: ACCEL_GPU=1, trait:GPU_FOO=required} | |
| 19:19:39 | Sundar | So, there are 2 VARs, each requesting 1 resource | |
| 19:20:09 | efried | yes, that's as it should be. One VAR, one accelerator. | |
| 19:20:17 | Sundar | That means, the resources/traits go duplicated. Also, the semantics that both resources need to come from the same resource provider is lost from Cyborg's perspective | |
| 19:20:24 | Sundar | *got | |
| 19:20:36 | efried | So | |
| 19:20:53 | efried | I was thinking a device profile would be for *one* device. | |
| 19:20:54 | efried | But | |
| 19:21:17 | efried | then you can't, as you say, demand that two devices come from the same provider | |
| 19:21:28 | efried | or more generally, take advantage of granular syntax. | |
| 19:21:34 | efried | unless | |
| 19:22:02 | Sundar | Yes. Nova would have that info, and presumably can colocate them. But, if VCyborg needs that info to connect the accelerators together, or whatever, that is ruled out | |
| 19:22:03 | efried | we number the extra spec key using the same pattern as we do for the other resources | |
| 19:22:12 | efried | which is actually probably a better, more composable design. | |
| 19:22:18 | Sundar | Yes! | |
| 19:22:24 | Sundar | +1 | |
| 19:22:38 | Sundar | That is the proposal in the spec | |
| 19:22:48 | efried | it is? | |
| 19:23:02 | efried | I got the impression you had set up device profiles so they could contain more than one accelerator | |