| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-04 | |||
| 09:39:12 | sean-k-mooney | nic and acclerator can expose mdevs | |
| 09:39:13 | bauzas | sure, but we have a spec fixing this | |
| 09:40:03 | bauzas | sean-k-mooney: in this spec, you explicitly say which PCI IDs *are* GPUs | |
| 09:40:06 | bauzas | sean-k-mooney: https://review.openstack.org/#/c/602474/ | |
| 09:40:22 | bauzas | I need to go, bbiabn | |
| 09:40:33 | sean-k-mooney | bauzas: cool that will work for the terible hack i want to do. ill let you know when you get back | |
| 09:51:10 | gmann | pooja_jadhav: hope that worked. | |
| 10:12:00 | bauzas | sean-k-mooney: just able to discuss for 5 mins | |
| 10:12:20 | bauzas | sean-k-mooney: but also, without the spec, it also only works for GPU devices | |
| 10:12:45 | bauzas | sean-k-mooney: because libvirt provides us all the mdev capable devices, but we look at which supports the provided GPU type | |
| 10:12:51 | bauzas | say 'nvidia-11' | |
| 10:13:27 | bauzas | so, in case you have mdev capable devices that are not GPU devices (say a nic one), they don't support the type | |
| 10:13:33 | bauzas | so we don't use them | |
| 10:14:03 | bauzas | also, say you have an Intel GPU not supporting the 'nvidia-11' type, then we don't have inventory for it too | |
| 10:15:06 | sean-k-mooney | ok so your assumeing good nameing of the mdev types without the spec | |
| 10:15:16 | sean-k-mooney | e.g. not small medium large | |
| 10:15:28 | sean-k-mooney | but ya i think that is fine | |
| 10:15:59 | sean-k-mooney | basically i found out that in kernel 4.16 there is a new driver called netdevsim that simulates a nic | |
| 10:16:15 | sean-k-mooney | it supprots sriov without any hardware and mdevs | |
| 10:16:41 | sean-k-mooney | so i think i can use it to test pci passhtrouhg, sriov, and other things in the gate | |
| 10:17:00 | sean-k-mooney | i also may be able to use the mdev support to pretend to be a vgpu | |
| 10:17:32 | sean-k-mooney | not sure about the last point but i was planning to look into it more for sriov testing | |
| 10:22:34 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Move the evacuate action to admin action https://review.openstack.org/607896 | |
| 11:46:42 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: consumer gen: support claim_resources https://review.openstack.org/583667 | |
| 11:46:43 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Enable nested allocation candidates in scheduler https://review.openstack.org/585672 | |
| 11:46:44 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use provider tree in virt FakeDriver https://review.openstack.org/604083 | |
| 11:49:34 | gibi | jaypipes, efried: I replied in https://review.openstack.org/583667 for both of you fixed your comments. The runway slot for this patch ends today so I appreciate any feedback | |
| 11:49:42 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: [WIP] Refactor scatter-gather utility to return exception objects https://review.openstack.org/607934 | |
| 11:52:29 | jaypipes | gibi: yup, on it. | |
| 11:52:41 | gibi | jaypipes: thanks a lot | |
| 11:54:17 | bauzas | sean-k-mooney: back there | |
| 11:54:25 | bauzas | sean-k-mooney: if you wanna continue to discuss... | |
| 11:54:54 | bauzas | gibi: don't worry about the runway slot, now that I'm done with the reshaper change, your series is my top prio | |
| 11:55:10 | gibi | bauzas: cool, thanks | |
| 11:55:42 | bauzas | gibi: so, the above means that I can review it, right? | |
| 11:56:46 | gibi | bauzas: it means that tha last patch of consumer gen is updated with fixes based on jaypipes and efried comments, there are patches top of that that also up to date til https://review.openstack.org/604083 I'm fixing comment in the rest as we speak | |
| 11:56:58 | bauzas | gibi: okay, will look later today | |
| 11:57:26 | bauzas | gibi: FWIW I have a machine with GPU devices so I can test most of your series | |
| 11:57:38 | gibi | bauzas: sounds cool :) | |
| 11:58:26 | gibi | bauzas: so far I relied on functional test cases. Fortunately the resource allocation handling these patches change are possible to cover with functional | |
| 11:59:21 | gibi | bauzas: the physical consumpton of VGPUs are something where my functional test case wont be enough | |
| 11:59:36 | sean-k-mooney | jaypipes: bauzas gibi so in case you missed it earlier i think i figured out a way to test pci passthough, sriov, and some other things like livemigration with the same in the gate using the netdevsim kernel module | |
| 11:59:54 | gibi | sean-k-mooney: that is really good news! | |
| 11:59:58 | bauzas | gibi: I mean, I could just test your series with my reshape change | |
| 12:00:14 | bauzas | gibi: so that I could be sure we would be able to boot a flavor asking for VGPUs | |
| 12:00:24 | bauzas | (in a nested worldà | |
| 12:00:26 | sean-k-mooney | i also think that can be hacked to pretend to be vgpus i need to investgate that futher | |
| 12:00:30 | gibi | bauzas: yeah allocate, then rehape, then allocate again is something that needs coverage | |
| 12:00:44 | bauzas | gibi: I tested all of that with my single change | |
| 12:00:49 | gibi | bauzas: cool | |
| 12:00:51 | bauzas | gibi: but of course the boot failed | |
| 12:01:00 | bauzas | gibi: since I wasn't using your series | |
| 12:01:13 | bauzas | what I could do is just to pick your series, and do the same | |
| 12:01:19 | gibi | bauzas: I assume it will work with my series included | |
| 12:01:26 | bauzas | gibi: that's my assumption too | |
| 12:01:32 | bauzas | and the functional test shows it | |
| 12:01:39 | bauzas | but a real evidence is better, I guess | |
| 12:01:45 | gibi | bauzas: totally agree | |
| 12:02:05 | bauzas | sean-k-mooney: WDYM by netdevsim module ? | |
| 12:02:37 | sean-k-mooney | it is a kernel module adding in kernel 4.16 specifcally to allow testing of hardware offloads without hardwar | |
| 12:02:44 | jaypipes | sean-k-mooney: you'll excuse my skepticism until I see a real functional test of those items :) | |
| 12:03:00 | sean-k-mooney | bauzas: it allows you to create pci device that support mdevs and sriov | |
| 12:03:10 | bauzas | sean-k-mooney: but it requires 4.16, right? | |
| 12:03:10 | sean-k-mooney | and it support ebpf too | |
| 12:03:19 | sean-k-mooney | yep so a fedora 28 job | |
| 12:03:21 | bauzas | which could be a problem | |
| 12:03:37 | jaypipes | sean-k-mooney: that said, I'm still supportive of this netdevsim effort. ANYTHING is better than what we have now, which is pretty much nothing. | |
| 12:03:38 | sean-k-mooney | fedora comes with 4.17 | |
| 12:04:36 | bauzas | sean-k-mooney: but bionic comes with 4.15 https://packages.ubuntu.com/bionic/linux-image-generic | |
| 12:04:51 | bauzas | so it wouldn't be a classic upstream gate job | |
| 12:05:04 | sean-k-mooney | jaypipes: ya i just learned about it yesterday so still doing reasarch but since i plan on working on sirov livemigration this cycle i would like to try to create an experimental job with this moduel to test it | |
| 12:05:19 | bauzas | sean-k-mooney: I was also considering the use of fake libvirt | |
| 12:05:23 | sean-k-mooney | bauzas: we have fedroa 28 images avaliable in the upstream gate for testing | |
| 12:05:27 | bauzas | couldn't that be simplier ? | |
| 12:05:49 | sean-k-mooney | bauzas: it could be this would allow tempest testing not just functional | |
| 12:06:28 | bauzas | sean-k-mooney: for SR-IOV it could be interesting | |
| 12:06:42 | bauzas | sean-k-mooney: for VGPU, I just feel it's unnecessary thru tempest | |
| 12:06:50 | bauzas | a functional test in-tree is better IMO | |
| 12:07:02 | sean-k-mooney | yep it also support things like configring rate limiting on the vf and trusted vfs | |
| 12:07:32 | bauzas | sean-k-mooney: just to make it clear, I was considering use of https://libvirt.org/drvtest.html | |
| 12:08:24 | sean-k-mooney | ok but i doubt that will support the featue we want to test | |
| 12:09:21 | sean-k-mooney | on testing with real hardware there is also this change https://review.openstack.org/#/c/607686/ :) | |
| 12:09:22 | jaypipes | sean-k-mooney: isn't artom working on that as well? (sriov live migration...) | |
| 12:09:32 | jaypipes | sean-k-mooney: or is artom focused on NUMA stuffs? | |
| 12:09:39 | sean-k-mooney | jaypipes: artom is working on numa | |
| 12:09:42 | jaypipes | ah, gotcha | |
| 12:09:56 | sean-k-mooney | im goning to do the sriov part | |
| 12:10:11 | jaypipes | sean-k-mooney: you're both brave men. | |
| 12:10:20 | sean-k-mooney | or dumb | |
| 12:10:25 | jaypipes | sean-k-mooney: I wouldn't touch that stuff with a ten foot pole. | |
| 12:10:39 | jaypipes | sean-k-mooney: the closest I'll get is doing some reviews for ya ;) | |
| 12:11:36 | sean-k-mooney | well i keep finding bugs in livemigration that i have to fix first related to the multiple port bindings on both the nova and neutron sides | |
| 12:12:50 | sean-k-mooney | so first im going to try and fix all those bugs then sriov live migration. i think neutron forgot to update teh sriovnic agent and driver to support multiple port binding so i have to fix that first | |
| 12:13:47 | artom | jaypipes, since you mentioned it, review pretty please https://review.openstack.org/#/c/599587/ ? ;) | |
| 12:14:00 | artom | But yeah, there's going to be some overlap/cooperation | |
| 12:14:13 | artom | Also, the mellanox guys were interested in SRIOV live migration, not sure what happened with that | |
| 12:14:53 | sean-k-mooney | artom: i talked to moshele last week. ill try and work with them if they want to help. | |
| 12:15:05 | artom | sean-k-mooney, ah, cool | |
| 12:15:09 | sean-k-mooney | artom: moshele said someone on his team was working on a poc | |
| 12:15:28 | artom | Hrmm, I thought they were going to propose a spec first? | |