| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-12-17 | |||
| 10:37:40 | bauzas | yes | |
| 10:37:43 | zigo | Ah ok... | |
| 10:37:44 | bauzas | nvidia T4s | |
| 10:37:52 | zigo | bauzas: That's the model we have, indeed. | |
| 10:38:06 | bauzas | yeah, we had a problem with nvidia | |
| 10:38:08 | zigo | bauzas: Is this for both the alias in the scheduler and in the compute ? | |
| 10:38:36 | bauzas | they only say that SR-IOV is experimental for T4, but given this, you actuallly need to change nova for it | |
| 10:38:46 | bauzas | zigo: hem, yes AFAIK | |
| 10:38:49 | bauzas | stephenfin: right ? ^ | |
| 10:40:09 | lyarwood | gibi: odd, so one of the tests makes two attempts to detach the same volume | |
| 10:40:30 | stephenfin | zigo, bauzas: correct - if the device reports SR-IOV capabilities, you need to set the device type | |
| 10:42:14 | zigo | bauzas: Thanks a lot. I'll add a per-compute option to select PCI / PF then. | |
| 10:42:32 | stephenfin | On that note, I still have two patches open to improve SR-IOV testing, if anyone has a few minutes. Happy to review other stuff in exchange https://review.opendev.org/q/topic:%2522sriov-functional-tests%2522+status:open | |
| 10:43:22 | lyarwood | gibi: https://review.opendev.org/c/openstack/tempest/+/760357 - this added a cleanup to detach the volume that these tests already detaches | |
| 10:44:17 | lyarwood | I'll write up a bug shortly | |
| 10:49:15 | zigo | bauzas: IMO, it'd be really nice to automate this, with placement reporting the correct type and the scheduler being auto-magic. It took us 2 days to figure out what was going on... | |
| 10:50:18 | elod | thx lyarwood ! | |
| 10:51:32 | gibi | stephenfin: ack, I can add that to today's agenda | |
| 10:53:28 | gibi | lyarwood: thanks! (sorry I was pulled after I pinged you) | |
| 10:53:41 | sean-k-mooney | zigo: we do not report pci devices in placement yet | |
| 10:54:00 | zigo | sean-k-mooney: But that would be a nice improvement ! :) | |
| 10:54:04 | sean-k-mooney | zigo: also this is because of a upgrade in your t4 device firmware | |
| 10:54:16 | sean-k-mooney | zigo: yes its on our todolist for a few years | |
| 10:54:32 | sean-k-mooney | there is a series that does most of it that was abandoned 2 years ago | |
| 10:54:37 | sean-k-mooney | we just need to revive it | |
| 10:54:56 | sean-k-mooney | it was blocked because placmenet did not support nested resouce providers proerly at the time | |
| 10:55:10 | sean-k-mooney | that has now meen fixed on the placment side | |
| 10:55:11 | gibi | lyarwood: i don't see the double detach here: tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario | |
| 10:55:42 | zigo | sean-k-mooney: Is type-PCI and type-PF the only possible values? | |
| 10:55:52 | gibi | lyarwood: nvm, I see it now | |
| 10:55:56 | sean-k-mooney | no also type-VF | |
| 10:55:59 | zigo | (I'm implementing this in my deployment software and need to validate input) | |
| 10:56:04 | zigo | Ok, thanks. | |
| 10:56:12 | gibi | nova_volume_attach adds a cleanup and the test case itself adds the same cleanup | |
| 10:56:21 | sean-k-mooney | we are adding type-VDPA but i plan to not allow that in the alias | |
| 10:56:53 | sean-k-mooney | VDPA devices will be VF that cant be use with alias based passthough as they require an additonal contolplain to function | |
| 10:57:33 | sean-k-mooney | zigo: by the way are you currently supporting devname in the pci whitelist? | |
| 10:57:43 | sean-k-mooney | zigo: we woudl really like to remove that in the future | |
| 10:57:57 | zigo | sean-k-mooney: Yeah, I do ! | |
| 10:58:34 | sean-k-mooney | ok what we would prefer is for installer tools to convert that to the pci adress | |
| 10:58:39 | zigo | For each compute node, I can set: use_gpu, gpu_name, gpu_vendor_id, gpu_product_id and gpu_device_type. | |
| 10:58:52 | sean-k-mooney | gpu_name? | |
| 10:58:53 | zigo | I just need to expose gpu_device_type in my API which I didn't do... | |
| 10:59:02 | zigo | Yeah, a free name which goes in the alias. | |
| 10:59:13 | sean-k-mooney | oh that is differnt | |
| 10:59:21 | sean-k-mooney | im not talking about the alias | |
| 10:59:38 | sean-k-mooney | im talking about the whitelist where we support devname for nics e.g. eht1 | |
| 10:59:43 | sean-k-mooney | eth1 | |
| 10:59:50 | zigo | We don't do SR-IOV yet. | |
| 11:00:01 | zigo | Just GPUs... | |
| 11:00:47 | sean-k-mooney | vGPU or pci passthough | |
| 11:01:22 | sean-k-mooney | if your using pci passthough via the alias and pci whitelist that is most of the config for neutron sriov | |
| 11:01:50 | sean-k-mooney | well sriov is jsut a tech its not related to networkign some gpus supprot it too | |
| 11:02:22 | sean-k-mooney | e.g. you can create VF on some gpus and passthough those to the guest | |
| 11:02:29 | sean-k-mooney | instead of the PF | |
| 11:04:55 | zigo | sean-k-mooney: Maybe for later, I'll implement it then... Thanks for the details! | |
| 11:05:42 | sean-k-mooney | no worries i just want to make sure you dont implement someitng we want to remove and then have to deal with migratin users off the use of devname | |
| 11:05:49 | sean-k-mooney | in the pci whitelist | |
| 11:07:26 | openstackgerrit | Merged openstack/nova stable/queens: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/767364 | |
| 11:47:52 | stephenfin | sean-k-mooney: fyi I closed this as WONTFIX https://bugs.launchpad.net/nova/+bug/1908507 | |
| 11:47:52 | openstack | Launchpad bug 1908507 in OpenStack Compute (nova) "vif quotas not set for tap interface" [Undecided,Won't fix] | |
| 11:48:00 | stephenfin | lemme know if you disagree | |
| 11:48:55 | sean-k-mooney | hum for calico that should technically work | |
| 11:49:32 | sean-k-mooney | i agree we are unlikely to fix it and they shoudl just use neutron qos | |
| 11:54:07 | openstackgerrit | Elod Illes proposed openstack/nova stable/rocky: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/766492 | |
| 12:07:59 | elod | lyarwood, sorry, one more thing o:) the ussuri gate fixing patch looks good now... if you have a minute: https://review.opendev.org/c/openstack/nova/+/766738 | |
| 12:13:45 | lyarwood | elod: LGTM, could you +2/+W as I own it | |
| 12:14:12 | elod | lyarwood: ACK | |
| 12:23:24 | gibi | stephenfin: thanks for the lot of bugtriage the numbers now looks really good | |
| 12:23:35 | stephenfin | np | |
| 12:27:48 | gibi | I think we have an all time low untriaged bug count | |
| 12:27:56 | gibi | (all time low since I'm looking) | |
| 12:56:10 | Yumeng | hi gibi, good afternoon. | |
| 12:56:16 | gibi | Yumeng: hi | |
| 12:56:35 | Yumeng | are you almost in holiday? :) | |
| 12:57:32 | Yumeng | I saw nova is going to cancel the next two weekly meeting. | |
| 12:57:33 | gibi | I'm off from next week | |
| 12:57:46 | gibi | Yumeng: yes, today is the last meeting this yera | |
| 12:57:48 | gibi | year | |
| 12:58:12 | Yumeng | wow, sounds excited and happy. | |
| 12:58:54 | Yumeng | I'm gonna catch you to discuss vGPU before you off. lol | |
| 12:59:03 | gibi | OK | |
| 13:01:01 | Yumeng | current nova code has this to support get accel_info to resume guest state when a host is booted. https://review.opendev.org/c/openstack/nova/+/767273/1/nova/virt/libvirt/driver.py#3490 | |
| 13:01:45 | Yumeng | and your concern is if cyborg-agent service starts after nova-compute, nova can not get acc_info as expected，how should we solve this issue? | |
| 13:02:47 | sean-k-mooney | we shoudl not in nova, at least not entirely | |
| 13:03:15 | sean-k-mooney | the deployment tools shoudl use systemd before/after to order the service starts | |
| 13:03:30 | gibi | I guess we need to make clear in the doc that we have an service restart ordering dependency on cyborg | |
| 13:04:05 | gibi | other than that nova could simply fail to reboot those VMs during service startup that needs cyborg | |
| 13:04:05 | sean-k-mooney | well ideally the cyborg agent shoudl have a "before: nova-compute.service" requirement | |
| 13:04:10 | sean-k-mooney | not the other way around | |
| 13:04:44 | sean-k-mooney | gibi: well we can and should call cyborgs api and do the async wait for the arq bidnigns | |
| 13:05:13 | sean-k-mooney | if the agent comes online in that tiem and responds it would be fine but yes we coudl skip them if not | |
| 13:05:33 | sean-k-mooney | the resume guest on host reboot feature does not work unless you use system to force the ordering anyway | |
| 13:05:53 | sean-k-mooney | libvirtd and openvswitch for example both need to be started before nova-compute | |
| 13:06:03 | Yumeng | "the resume guest on host reboot feature does not work unless you use system to force the ordering anyway" +1 | |
| 13:06:13 | gibi | Today we don't do async in this code path | |
| 13:06:35 | gibi | at least nova.compute.manager.ComputeManager._get_accel_info does not do that | |
| 13:06:59 | sean-k-mooney | are we just getting the info or rebining on reboot | |
| 13:07:06 | gibi | just getting infor | |
| 13:07:13 | gibi | no rebind as far as I see | |