| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-23 | |||
| 12:10:31 | sean-k-mooney | then they create the mig instance "nvidia-smi mig -cgi 2g.10gb,2g.10gb,2g.10gb -C" | |
| 12:10:48 | sean-k-mooney | and the vf had inventories | |
| 12:10:50 | sean-k-mooney | cat /sys/class/mdev_bus/0000\:04\:01.0/mdev_supported_types/nvidia-475/available_instances | |
| 12:11:07 | sean-k-mooney | so they added | |
| 12:11:09 | sean-k-mooney | [devices] | |
| 12:11:11 | sean-k-mooney | enabled_vgpu_types = nvidia-475,nvidia-1 | |
| 12:11:13 | sean-k-mooney | [vgpu_nvidia-475] | |
| 12:11:15 | sean-k-mooney | device_addresses = 0000:04:00.4,0000:04:00.5,0000:04:00.6 | |
| 12:11:17 | sean-k-mooney | [vgpu_nvidia-1] | |
| 12:11:19 | sean-k-mooney | device_addresses = 0000:00:00.1 | |
| 12:11:21 | sean-k-mooney | to the nova.conf | |
| 12:12:26 | sean-k-mooney | songwenping: the steps bauzas did were | |
| 12:12:45 | sean-k-mooney | nvidia-smi -i 0 -mig 1 | |
| 12:12:47 | songwenping | thanks sean-k-mooney | |
| 12:12:49 | sean-k-mooney | to enable mig | |
| 12:12:54 | sean-k-mooney | which they checked with | |
| 12:13:02 | sean-k-mooney | nvidia-smi -i 0 --query-gpu=pci.bus_id,mig.mode.current --format=csv | |
| 12:13:23 | sean-k-mooney | then "/usr/lib/nvidia/sriov-manage -e 00:04:0000.0" to enable the vfs | |
| 12:13:45 | sean-k-mooney | and then you can use " ll /sys/class/mdev_bus/" | |
| 12:13:54 | sean-k-mooney | to verify that hte device are capable of hosting mdevs | |
| 12:14:23 | sean-k-mooney | then they configugred vgpu instance per mig instance | |
| 12:14:25 | sean-k-mooney | nvidia-smi mig -cgi 2g.10gb,2g.10gb,2g.10gb -C | |
| 12:14:54 | sean-k-mooney | and after that they had 1 mdev inventory per vf | |
| 12:15:02 | sean-k-mooney | which can be checked with "cat /sys/class/mdev_bus/0000\:04\:01.0/mdev_supported_types/nvidia-475/available_instances" | |
| 12:15:19 | sean-k-mooney | songwenping: hopefully that helps | |
| 12:15:44 | songwenping | thanks, this is useful. | |
| 12:16:41 | sean-k-mooney | our downstream docs bz to write this up is still pending | |
| 12:34:58 | sean-k-mooney | dansmith: those keystone select number really are an order of magnitude higher then everything else https://paste.opendev.org/show/bsG9exlvGqaNZrbhEnJp/ | |
| 12:59:49 | gibi | melwitt: can I help somehow to convince you to +2 https://review.opendev.org/c/openstack/nova-specs/+/791047 ? :) | |
| 13:03:04 | sean-k-mooney | gibi: she may have been leaving that to stephenfin to +2w but if melwitt is happy withthe proposal i think stephenfin will be happy that you have incoperated there feedback with regards to the dynmaic alloctaiton fo VF/PFs | |
| 13:05:24 | gibi | ahh OK. | |
| 13:05:42 | gibi | stephenfin: if you have time, I think https://review.opendev.org/c/openstack/nova-specs/+/791047 is ready for a +2+A :) | |
| 13:08:37 | gibi | I'm not sure I understand what the sitting down part means ^^ | |
| 13:09:06 | sean-k-mooney | preparing my self for the shock of us agreeing to actully track pci device in placement after like what 5 years | |
| 13:09:18 | gibi | ahh :d | |
| 13:09:19 | gibi | :D | |
| 13:09:34 | sean-k-mooney | placement was still in nova wehn we started this converastion and nested resouce providers did not exist | |
| 13:10:07 | gibi | yeah it is a long journey | |
| 13:10:14 | gibi | and it is not over yet :) | |
| 13:32:54 | opendevreview | ribaudr proposed openstack/nova-specs master: libvirt: Allow Manila shares to be directly attached to instances https://review.opendev.org/c/openstack/nova-specs/+/833669 | |
| 13:49:53 | dansmith | sean-k-mooney: I know.. something has to be wrong right? | |
| 13:50:44 | sean-k-mooney | presumable. i would expect keysone to be higher then the rest but not 10x | |
| 13:51:01 | sean-k-mooney | and that 10x neutron which is also higher then the rest | |
| 13:51:38 | dansmith | right | |
| 13:52:11 | dansmith | neutron goes over keystone quickly during tempest tho | |
| 14:11:03 | sean-k-mooney | dansmith: while your here we use 2 allocation for live migration correct? one is held by the instnace an the other by the migration uuid? | |
| 14:11:19 | dansmith | yeah | |
| 14:11:39 | sean-k-mooney | do you remember when that change was made it was 5 or more cycles ago right | |
| 14:11:43 | sean-k-mooney | as in its not recent | |
| 14:11:50 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1975490 just in the context of that | |
| 14:11:58 | dansmith | oh jeez, no idea, but I'm sure it was 5+ ago | |
| 14:12:32 | sean-k-mooney | right it was well before train with the new vcpu code | |
| 14:13:23 | sean-k-mooney | they did not say what version of openstack they were using but im guessign it was quite an old version if they are seeing placment check the source allocations like they describe | |
| 14:14:58 | sean-k-mooney | i dont think any supported branch including the ones in EM still use a signel allocation for both hosts | |
| 14:15:15 | dansmith | yeah I'm sure not | |
| 14:15:41 | sean-k-mooney | we will see if they respond i guess | |
| 14:30:32 | opendevreview | ribaudr proposed openstack/nova-specs master: Allow unshelve to a specific host https://review.opendev.org/c/openstack/nova-specs/+/831506 | |
| 15:14:46 | kashyap | melwitt: When you're about, can you review this, please: https://review.opendev.org/c/openstack/nova/+/838926 (It already has a +2 from gibi) | |
| 15:15:12 | kashyap | (And it is successfully tested by a Red Hat contributor downstream) | |
| 15:18:16 | kashyap | bauzas: --^ | |
| 15:42:05 | melwitt | bauzas, elodilles, gmann: I started an etherpad for stable branch ci here. will fill it in with more info https://etherpad.opendev.org/p/nova-stable-branch-ci | |
| 15:43:08 | melwitt | kashyap: thanks for noticing :) | |
| 15:44:39 | melwitt | gibi, sean-k-mooney: yeah I thought stephenfin may want to look at the current spec bc he had a lot of feedback in previous PS | |
| 15:44:54 | stephenfin | oh, I'm looking at it at the moment | |
| 15:44:56 | kashyap | melwitt: I recall debugging it with you in the past; thanks for bringing it to its logical conclusion | |
| 15:45:17 | kashyap | Looks good to me to workaround it. (Also, the CI failure seems unrelated) | |
| 15:45:20 | melwitt | stephenfin: great :) | |
| 15:46:17 | melwitt | kashyap: will also review your patch | |
| 15:46:29 | kashyap | melwitt: It's also a workaround :D | |
| 15:46:36 | melwitt | haha nice | |
| 15:46:53 | elodilles | melwitt: thanks \o/ it was on my TODO as well o:) | |
| 15:47:51 | kashyap | melwitt: But it is functionally tested by a Red Hat support eng downstream | |
| 15:48:01 | kashyap | (And it works) | |
| 15:48:06 | gmann | melwitt: ack, thanks. | |
| 15:48:21 | melwitt | kashyap: ack | |
| 15:49:35 | kashyap | Thank you | |
| 15:54:18 | elodilles | melwitt: i proposed quite many bandit capping patches in the past, but did not notice that placement needs it :-o ( https://review.opendev.org/q/topic:cap-bandit-1.6.2 ) | |
| 15:56:37 | melwitt | elodilles: yeah I remembered that being a thing and I think I found your patches for other projects. placement had been working ok for awhile, not actually sure when this cropped up. backports don't get proposed there often | |
| 15:58:10 | bauzas | Uggla: made a review with a French link on https://review.opendev.org/c/openstack/nova-specs/+/831506/ | |
| 15:58:35 | bauzas | either way, +2d | |
| 15:58:52 | bauzas | cores, a spec seems very close to be accepted https://review.opendev.org/c/openstack/nova-specs/+/831506/4.55 | |
| 15:58:58 | bauzas | cores, a spec seems very close to be accepted https://review.opendev.org/c/openstack/nova-specs/+/831506/4..5 even | |
| 16:00:36 | elodilles | melwitt: yeah, and we have periodic-stable-jobs in placement only since xena: https://review.opendev.org/c/openstack/placement/+/775384 | |
| 16:01:44 | stephenfin | gibi: sean-k-mooney: melwitt: My only comment is on the config option. 'device_list' suggests it's a list but it's not - it's a MultiStrOpt (meaning you'd simply specify it multiple times in nova.conf) | |
| 16:02:13 | melwitt | elodilles: ah right | |
| 16:02:14 | elodilles | melwitt: we have the periodic-weekly for master, but maybe this worth a backport if we want to catch broken gates on older stable branches | |
| 16:02:14 | gmann | bauzas: FYI, i will be absent in nova meeting tomorrow as we have 'release naming meeting' at the same time - http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028618.html | |
| 16:02:19 | stephenfin | We can bikeshed on the correct name for that now or later, I don't mind. So long as we don't call it a plural thing when it's not actually plural | |
| 16:04:25 | gibi | stephenfin: hm | |
| 16:04:33 | sean-k-mooney | stephenfin: well eventully we shoudl make it a list instad of a multi opt | |
| 16:04:53 | sean-k-mooney | stephenfin: the alisa support json list syntax or mutlti opt repating | |
| 16:05:25 | sean-k-mooney | stephenfin: generally it enables multipel things by the way | |
| 16:05:47 | sean-k-mooney | it supprots regex and glob mathch or you can just use vender id and product id | |
| 16:05:48 | gibi | stephenfin: we can drop the _list postfix and simply call it [pci]devices | |
| 16:06:06 | sean-k-mooney | ya that would work | |
| 16:12:02 | sean-k-mooney | gibi: stephenfin if ye want to s/device_list/devices/ im fine with that | |
| 16:12:14 | sean-k-mooney | and or do it as a followup | |
| 16:12:42 | gibi | as I see stephenfin has things against the pluralness as well | |
| 16:13:04 | gibi | anyhow I will reply in the spec and we can discuss it tomorrow over IRC | |
| 16:13:32 | sean-k-mooney | well from my point of view it is plural as it generally refers to multiple devices via the vendor/product id or the regex support | |