| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-11-28 | |||
| 14:33:12 | bauzas | efried: yup, I know | |
| 14:33:25 | bauzas | efried: I was more commenting about the fact we discussed that | |
| 14:33:27 | efried | bauzas The one I remember was for "preferred traits", not for optional resource classes. | |
| 14:33:33 | bauzas | yeah that | |
| 14:33:37 | mdbooth | So if you have an instance with a root disk and a config disk, then you attach a volume, you'll have root disk, config disk, volume | |
| 14:33:49 | mdbooth | But in rebuild we always put the config disk last | |
| 14:33:58 | mdbooth | So on rebuild you'll have root disk, volume, config disk | |
| 14:33:59 | jianghuaw_ | bauzas, I think this case makes sense. What I'm think is that a flavor is asking only VGPU, but it placement may return RPs which contains display heads also. | |
| 14:34:10 | mdbooth | And in fact we have no way of knowing what the previous order was | |
| 14:34:13 | jianghuaw_ | it will result into the display heads leaking. | |
| 14:34:16 | bauzas | jianghuaw_: right | |
| 14:34:37 | bauzas | what do you mean by leaking ? | |
| 14:34:43 | efried | mdbooth For us, the dev order is determined by which virtual "slot" the disk is attached via. So we maintain a cache of which devices are associated with which slots, and then we make sure to restore that on rebuild. | |
| 14:34:50 | bauzas | jianghuaw_: that's depending on the virt driver, right ? | |
| 14:35:00 | mdbooth | efried: Right. Where do you store the cache, though? | |
| 14:35:15 | efried | mdbooth In our case, in order to support that, the user has to set up swift :$ | |
| 14:35:27 | mdbooth | Ah... | |
| 14:35:31 | efried | indeeed. | |
| 14:35:35 | jianghuaw_ | bauzas, I mean it automatically consumed display heads. but placement don't it. | |
| 14:35:51 | mdbooth | See, I actually think there's a lot of merit in having persistent driver state | |
| 14:36:14 | mdbooth | I think it should be local to the hypervisor, though | |
| 14:36:34 | efried | mdbooth We don't go crazy with persistent state, for sure. We save these slot mappings, and we save the partition's NVRAM. | |
| 14:36:55 | jianghuaw_ | bauzas, don't it => doesn't know of it. | |
| 14:37:16 | mdbooth | efried: Definitely in favour of the concept. | |
| 14:37:48 | efried | mdbooth I have to say, our implementation ain't pretty. | |
| 14:38:10 | jianghuaw_ | bauzas, virt driver can only stop a booting by checking the resoureces in the allocation. | |
| 14:38:21 | efried | mdbooth It was one of these rush jobs where we realized the problem (devices attached out of order) late in a cycle, and threw together a solution at the last minute. | |
| 14:38:55 | mdbooth | efried: Out of curiosity, why did you consider it a sufficiently big problem for a big rush job? | |
| 14:39:01 | jianghuaw_ | bauzas, or is there a way to correct the allocation if the allocated vGPU also supports display heads? | |
| 14:39:21 | mdbooth | Device ordering is also inconsistent in a Linux guest OS | |
| 14:39:29 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] re-use existing conf with auth token middleware https://review.openstack.org/523403 | |
| 14:39:38 | efried | mdbooth I don't remember all the reasons, but e.g. maybe you don't boot. | |
| 14:39:47 | efried | mdbooth Or possibly worse, boot to the wrong disk. | |
| 14:41:20 | efried | mdbooth Also could have something to do with applications in guests referring to devices by name - and if that name changes, you're kaput. | |
| 14:41:43 | mdbooth | efried: That last point isn't fixed by consistent device ordering | |
| 14:41:47 | efried | mdbooth Keep in mind we're running not just Linux, but also AIX and IBMi. | |
| 14:41:56 | mdbooth | Because as I say, Linux itself is also non-deterministic | |
| 14:42:31 | mdbooth | The first should be handled by the hypervisor specifying a boot order for its BIOS/UEFI | |
| 14:43:27 | mdbooth | Or whatever hopefully much better boot solution is in Power :) | |
| 14:43:43 | efried | mdbooth That's exactly the point: The boot list is specified based on hardware addresses, which are discovered based on slots. | |
| 14:45:04 | efried | mdbooth The big conceptual divide here is that PowerVM has a real hypervisor and the guests are real partitions. | |
| 14:46:10 | jianghuaw_ | bauzas, I think efried's opinion is that we shouldn't make inventory for VGPU_DISPLAY_HEAD as that's possible to be consumed automatically by requesting VGPU. | |
| 14:46:54 | efried | jianghuaw_ Only if display heads are *only* consumed automatically by requesting VGPU | |
| 14:47:29 | edleafe | jianghuaw_: if it can't be consumed independently, and is always consumed automatically, then it shouldn't be tracked as inventory. | |
| 14:47:45 | efried | jianghuaw_ If it's possible to request them separately, then you need to inventory them separately; and then have extra logic in scenarios where it would happen automatically such that, if the flavor doesn't request the appropriate number of each, you throw an error. | |
| 14:47:46 | bauzas | jianghuaw_: I agree with him | |
| 14:48:18 | bauzas | jianghuaw_: until we have a way to track optional resources classes, we shouldn't be providing inventories for display heads IMHO | |
| 14:48:42 | bauzas | and just do the display head consumption solely in the virt driver, if you can | |
| 14:48:59 | bauzas | tbh, libvirt doesn't support that yet, so I don't give a bit of that :p | |
| 14:49:10 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Refined fix for validating image on rebuild https://review.openstack.org/523427 | |
| 14:50:14 | jianghuaw_ | bauzas, efried Ok. Agreed. I will drop that patch for inventoring display heads. | |
| 14:51:05 | jianghuaw_ | Per my understand, the display heads will always be consumed automatically as long as the VGPUs supporting display heads. | |
| 14:53:11 | jianghuaw_ | Actually different vGPU types may support different display heads. And users may need some vGPUs which support specified amount of display heads. That's why I said it seems somehow traits for the vGPUs. | |
| 14:54:00 | efried | jianghuaw_ Yes, that sounds like it might be suitable for traits. | |
| 14:55:00 | jianghuaw_ | efried, but jaypipes seems don't agree ^ :-) | |
| 14:55:24 | bauzas | jianghuaw_: well, requesting a display head is not quantitative | |
| 14:55:35 | efried | jianghuaw_ It depends whether it can be consumed independently, or is always tied to the VGPU. | |
| 14:55:43 | efried | Which I'm still not clear on. | |
| 14:56:24 | jianghuaw_ | efried, yes, it's always tied to VGPU. | |
| 14:56:35 | efried | Okay. If it is *always* the case that requesting a specific VGPU results in a specific number (possibly zero) of display heads being consumed, then there's no reason to inventory separately. | |
| 14:57:55 | efried | jianghuaw_ And how does the user specify which type of VGPU they're getting? | |
| 14:58:02 | bauzas | by the trait | |
| 14:58:15 | jianghuaw_ | yes, by traits. | |
| 14:58:18 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Nits from Ic3ab7d60e4ac12b767fe70bef97b327545a86e74 https://review.openstack.org/523432 | |
| 14:58:30 | bauzas | given we don't support traits yet, we only ask for the operator to provide *one* type | |
| 14:58:47 | efried | So a given trait is in fact overloading more than one aspect of the VGPU's characteristics. | |
| 14:59:15 | bauzas | well | |
| 14:59:22 | efried | In other words, by saying I need a VGPU of type X, I'm also implicitly stating that I'm going to get two display heads (or whatever) | |
| 14:59:33 | bauzas | that is a correct assumption to me | |
| 14:59:59 | efried | Okay. Then if that correlation always holds true | |
| 15:00:12 | efried | that is, a given VGPU type always maps to a given number of display heads | |
| 15:00:21 | jianghuaw_ | the plan is to associate traits like display_solutions; supported features after we support traits. | |
| 15:00:26 | bauzas | efried: see the spec | |
| 15:00:33 | bauzas | there are some examples | |
| 15:00:40 | efried | then you shouldn't have inventory for the display heads. | |
| 15:00:45 | bauzas | I agree | |
| 15:00:52 | bauzas | if that's correlated | |
| 15:01:10 | efried | And there's even no need for extra traits - though those could be included and it wouldn't hurt anything. | |
| 15:01:12 | bauzas | tbh, I haven't thought that much on that problem, since libvirt doesn't return the heads :p | |
| 15:01:20 | bauzas | (yet) | |
| 15:02:00 | jianghuaw_ | efried, the display heads is one factor can be used to determine a vGPU type. | |
| 15:02:26 | efried | jianghuaw_ Ah, okay, if you want to be able to go in the other direction, then it makes sense. | |
| 15:03:10 | efried | jianghuaw_ I can imagine that you don't want to be getting particular about VGPU types; rather, you want to be enumerating capabilities of each type, so that the scheduler could conceivably pick one of any number of types as long as they fit the capabilities you request. | |
| 15:03:15 | mriedem | gibi: replied in https://review.openstack.org/#/c/523353/ and explained the issue | |
| 15:03:24 | jianghuaw_ | But the possible value is a continuous number which makes it more like quantitative. | |
| 15:03:58 | gibi | mriedem: looking | |
| 15:04:07 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/newton: Refined fix for validating image on rebuild https://review.openstack.org/523434 | |
| 15:04:54 | mriedem | instance actions/events are mythical beasts | |
| 15:04:57 | mriedem | easily broken | |
| 15:05:02 | mriedem | rarely tested | |
| 15:05:19 | jianghuaw_ | efried, yes. That's also the result of the discussion with jaypipes to use capabilities instead of specifying vGPU type. | |
| 15:05:27 | efried | jianghuaw_ But a small number of discrete possible values | |
| 15:05:29 | mriedem | dansmith: i've got those backports all up now https://review.openstack.org/#/q/I1a46ef1503be2febcd20f4594f44344d05525446 | |
| 15:05:39 | jianghuaw_ | efried, indeed. | |
| 15:05:39 | efried | jianghuaw_ Like 1, 2, 4, 8 | |
| 15:05:45 | gibi | mriedem: thanks, I | |
| 15:05:49 | gibi | mriedem: thanks, I'm +2 now | |
| 15:06:09 | jianghuaw_ | efried, it looks like you agree with me to make it as traits. right? | |
| 15:06:52 | efried | jianghuaw_ With the caveat that I don't fully understand the different ways you can request/configure these things, yes, I believe I agree. | |
| 15:07:26 | mriedem | sdague: could use some review on https://review.openstack.org/#/c/521391/ which is a fix for a regression introduced by a recent cve fix, so it's going to have to be backported through to newton - there is a regression test patch underneath it | |