| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-04-21 | |||
| 14:57:11 | gibi | so I assume nova also uses that config option for its of file lock | |
| 14:57:12 | sean-k-mooney | elodilles: gibi any prefernce on how to proceed. | |
| 14:57:20 | sean-k-mooney | yes | |
| 14:57:23 | sean-k-mooney | we do | |
| 14:57:42 | gibi | so somewhere in the nova test env we already provide a default for that option | |
| 14:57:44 | sean-k-mooney | and os-brick is runnign in novas process space so its reading our config value | |
| 14:58:04 | sean-k-mooney | gibi: that or we mock the locking code | |
| 14:58:09 | gibi | yeah | |
| 14:58:09 | sean-k-mooney | so it does not write to disk | |
| 14:58:27 | gibi | we already do something so that the nova usage of that opt is covered | |
| 14:58:28 | sean-k-mooney | in this case since its failing its actully trying to create a file systm lock in the unit test | |
| 14:58:31 | sean-k-mooney | which it shoudl not be | |
| 14:58:48 | sean-k-mooney | i woudl guess we are just missing a fixture in those tets | |
| 14:59:00 | gibi | yeah that is what I think too | |
| 14:59:09 | gibi | we need to find what is missing and extend the fixture | |
| 15:13:02 | kashyap | gibi: To tie up the lose end on caps vs domCaps -- the guidance from the libvirt folks is (a) no, we can't treat the caps == domCaps w/ 'host-model' mode; and (b) we should use domCaps wherever possible. | |
| 15:13:23 | kashyap | gibi: So that means, we should introduce a new config object | |
| 15:13:32 | gibi | kashyap: ack, make sense | |
| 15:14:02 | kashyap | gibi: I might need some help on this XML parsing ... I'll take a stab | |
| 15:14:21 | kashyap | ... at it. | |
| 15:14:42 | gibi | kashyap: just ping me if you need another set of eyes | |
| 15:14:57 | kashyap | Will do; thx | |
| 15:59:14 | mnaser | i would appreciate some reviews on https://review.opendev.org/c/openstack/nova/+/830646 (wrt allowing addition of viommu to vms) | |
| 16:13:35 | bauzas | mnaser: you're not the first one to ask for reviews on that stephenfin's old patch, will mark it as review priority for the team | |
| 16:14:10 | mnaser | bauzas: cool thanks, the other one might have been ricolin but thats coming from the same side, so not sure if thats super fair ;) haha | |
| 16:14:34 | bauzas | that being said, the bp isn't validated yet | |
| 16:15:14 | bauzas | and not sure stephenfin will unghost himself :) | |
| 16:15:39 | stephenfin | bauzas: I think mnaser and ricolin are taking care of it now? | |
| 16:16:00 | bauzas | good question | |
| 16:16:14 | mnaser | yeah it's ready (imho) from a code perspective, but if there's something else we have to do, we can take care of | |
| 16:17:20 | bauzas | mnaser: this is just a paperwork question | |
| 16:17:33 | bauzas | since a new extraspec is added + the api validation | |
| 16:17:42 | bauzas | this has to be tracked correctly | |
| 16:17:55 | bauzas | and we need to balance in a meeting whether we need a spec or not | |
| 16:18:40 | bauzas | mnaser: that being said, I'm more than glad to welcome you as a new feature contributor ! :p | |
| 16:19:03 | bauzas | we don't have badges but I can create one for Berlin :p | |
| 16:19:23 | stephenfin | I can't say if it needs a spec, but it definitely needs a blueprint and some discussion in the meeting | |
| 16:19:36 | stephenfin | given there's now an extra spec (I think? | |
| 16:19:38 | stephenfin | ) | |
| 16:20:34 | bauzas | yeah and yeah | |
| 16:20:45 | bauzas | (needs a blueprint and probably don't need a spec) | |
| 16:21:26 | bauzas | stephenfin: have you just added some fancy gerrit topic for some non-existing related blueprint ? | |
| 16:21:43 | bauzas | if so, you're bragging. | |
| 16:22:00 | bauzas | :p | |
| 16:37:19 | opendevreview | Kashyap Chamarthy proposed openstack/nova master: libvirt: Add workaround to remove compareCPU() check on the destination https://review.opendev.org/c/openstack/nova/+/838926 | |
| 16:40:37 | opendevreview | Kashyap Chamarthy proposed openstack/nova master: libvirt: Add workaround to skip compareCPU() check on the destination https://review.opendev.org/c/openstack/nova/+/838926 | |
| 17:08:48 | sean-k-mooney | mnaser: viommu is not useful without a lot of extra work | |
| 17:08:57 | sean-k-mooney | stephenfin: that definetly needs a spec | |
| 17:09:24 | sean-k-mooney | viommu is trivial to enable i have implemeted that locally before | |
| 17:09:43 | sean-k-mooney | mnaser: the issue is all devices will be in the same iommu group | |
| 17:10:20 | sean-k-mooney | so either you disable the memory isolation in the guest vfio-pci module | |
| 17:10:35 | sean-k-mooney | or nova has to take full contol of the pcie toplogy | |
| 17:10:52 | sean-k-mooney | mnaser: stephenfin did some poc work related to this | |
| 17:11:03 | sean-k-mooney | but i dont think its complete | |
| 17:11:15 | sean-k-mooney | and it had upgrade impacts potentially | |
| 17:11:21 | stephenfin | far from complete | |
| 17:11:35 | sean-k-mooney | so it needs a spec to figure that out and more expirece playing with the poc | |
| 17:12:00 | sean-k-mooney | mnaser: by the way apprently the prefomcne of the viommu is really bad with passthough devices | |
| 17:12:13 | sean-k-mooney | im not sure why but that is what the virt team told me in the past | |
| 17:12:58 | sean-k-mooney | hopefully that has changed mnaser have you done any tests locally with livbirt? | |
| 17:13:27 | mnaser | sean-k-mooney: so, my knowledge level to this is that we have a company which ships pci cards which are hardware accelerators | |
| 17:13:58 | mnaser | sean-k-mooney: as we tried to get them to use native openstack, we figured out the extra qemu parameters they had and it included the iommu stuff | |
| 17:14:10 | mnaser | Supposedly, the cards won’t work without them in PCI pass through in the guests. | |
| 17:14:22 | sean-k-mooney | ack | |
| 17:14:31 | sean-k-mooney | what driver to you use in the guest | |
| 17:14:41 | sean-k-mooney | is it vfio-pci or a normal kernel dirver | |
| 17:14:58 | sean-k-mooney | if they dont need to be in there own iommu group the stephenfin patch would enable your usecase | |
| 17:15:04 | mnaser | I think you do bring up a good point that we’re adding it by default to q35 machine types, so I think making another flag to make it opt in might be good | |
| 17:15:17 | sean-k-mooney | so we could perhaps start with the simpel turn this one feature | |
| 17:15:44 | sean-k-mooney | we could possble add it by default | |
| 17:16:01 | sean-k-mooney | but what we cant do by defualt is change the pci layout to put each device in its own iommu group | |
| 17:16:11 | sean-k-mooney | at least not without a lot of testing | |
| 17:16:18 | mnaser | Yeah I don’t think this is what the device needs in this case | |
| 17:16:32 | sean-k-mooney | havign a hw:viommu=on|off extra spec if simple | |
| 17:17:14 | sean-k-mooney | ok stephenfin i would be ok with a specless bluepint for just adding the extraspec/image propety for this | |
| 17:17:38 | sean-k-mooney | but if we were to do the iommu group split out that would need the spec | |
| 17:18:38 | sean-k-mooney | mnaser: tldr if we want each device to be in its own iommu group we need to create a pcie expantion bridge per device. and to do that we need to assign virtual pci devivie adress to every device in the xml | |
| 17:19:10 | sean-k-mooney | mnaser: so that is the bit that has upgade concerns as basically every device in the vm would change its address potentially | |
| 17:21:04 | sean-k-mooney | mnaser: have you tried stephenfin's patch with that card? | |
| 17:22:02 | sean-k-mooney | mnaser: im not sure im comfortable with exposeing the adress with by the way | |
| 17:22:08 | sean-k-mooney | that feel a bit too low level | |
| 17:30:50 | mnaser | sean-k-mooney: i haven't yet, but it's kindof a reimplementation of how they've done things with flat libvirt | |
| 17:31:05 | mnaser | but i am thinknig hw:viommu=on|off is a good idea | |
| 17:31:11 | sean-k-mooney | im about to -1 the patch | |
| 17:31:21 | mnaser | in terms of controlling the address, i'm thinking since it's a flavor extra spec and not a image extra spec, it's an operator-level decision | |
| 17:31:25 | mnaser | okay great | |
| 17:31:32 | sean-k-mooney | no | |
| 17:31:39 | mnaser | we'll discuss further there and ill try to poc something as well | |
| 17:31:47 | sean-k-mooney | so image extra specs are ment to be used for contoleing emulated hardware | |
| 17:31:49 | sean-k-mooney | not extra specs | |
| 17:32:01 | sean-k-mooney | so in generaly this shoudl only be exposed as an image property | |
| 17:32:17 | sean-k-mooney | but if we are to expose it in the image i would be open to having it in both | |
| 17:33:02 | sean-k-mooney | so i think we should have somehtin like hw_viommu_modle=none|intel|smmuv3|virtio in the image to contol turing this on | |
| 17:33:22 | sean-k-mooney | and i woudl be open to also haveing hw:viommu_modle=none|intel|smmuv3|virtio for partiy in the flavor | |
| 17:33:49 | sean-k-mooney | the adress space unless its somethign we are going to schdule on i am not conviced we shoudl expose as a configurable | |
| 17:35:30 | mnaser | sean-k-mooney: you know this stuff better than we do, as long as at the end of the day, we can get the xml to show up on those virt guests, we're good to go | |
| 17:35:36 | mnaser | but i agree on the image extra specs controlling hw_viommu_module too | |
| 17:36:13 | sean-k-mooney | going forwad i think virtio is what we will want but that is very new | |
| 17:36:33 | sean-k-mooney | intel only works on x86 and smmuv3 only works on arm | |
| 17:36:36 | sean-k-mooney | virtio works on both | |