Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-11-16
13:39:04 dmitriis ((Pdb)) request
13:39:10 dmitriis I had something like this ^
13:39:20 sean-k-mooney yep
13:39:26 sean-k-mooney that shoudl work
13:39:37 sean-k-mooney that is the python None
13:39:43 sean-k-mooney not its not quoted
13:39:48 sean-k-mooney *note
13:40:03 sean-k-mooney that will match physical_network=null
13:40:05 dmitriis ah, maybe that's an old note that I have. It was since fixed to use a string
13:40:50 sean-k-mooney you have to use "'physical_network':null" not "'physical_network':'null'" in the whitelist
13:41:32 sean-k-mooney like this passthrough_whitelist={ "vendor_id":"15b3", "product_id":"101e", "physical_network":null }
13:42:06 sean-k-mooney that enabels a connectx-6 dx for overlay networking
13:43:04 sean-k-mooney dmitriis: if you want to have some VF for vlan/flat and other for geneve tunnels you need to use the adress field to partion the vfs into groups
13:44:22 dmitriis sean-k-mooney: I suppose that could be one way to do it
13:44:32 sean-k-mooney dmitriis: this is because tunnels was never ment to be supported at all so we never impleted a way to allow a device to be part of multile physnets
13:45:20 sean-k-mooney dmitriis: if it was not for the fact that this was used as production we would have closed this as a secuirty bug and blocked the use of null the detail are in the bug
13:46:03 dmitriis sean-k-mooney: yeah, makes sense. I think that documenting this and suggesting address-based partitioning as a workaround is viable for now
13:46:52 sean-k-mooney dmitriis: the tl;dr is we use a json parser to parse the whitelist and in json unquoted null is mapped to the python None object which just happens to be what we get when we parse the phsynet form networks that dont have one
13:47:46 sean-k-mooney which is why physical_network=None in the pci request will actully match
13:48:09 sean-k-mooney sicne that is also python None object not the stirng 'None'
13:49:50 sean-k-mooney fyi hte docs for the whitelist are not greate but incase yo udont know we support both bash style globs and python regex expression in the adress filed
13:50:07 sean-k-mooney and we supprot both in either the sting or dict form
13:50:12 sean-k-mooney https://docs.openstack.org/nova/latest/configuration/config.html#pci.passthrough_whitelist has some examples
13:50:29 dmitriis sean-k-mooney: I recall some other place in Nova where I had to use a string instead (trying to find where so maybe I wrongly brought this up here).
13:51:28 sean-k-mooney there might be if you find it let me knwo and i might know the history or it might just be a bug
13:52:48 dmitriis passthrough_whitelist = [{"vendor_id": "15b3", "product_id": "101e", "physical_network": null, "remote_managed": "true"}]
13:52:48 dmitriis sean-k-mooney: that's what we used in the lab
13:53:13 dmitriis and for physnets: passthrough_whitelist = [{"vendor_id": "15b3", "product_id": "101e", "physical_network": "physnet1", "remote_managed": "true"}]
13:53:59 sean-k-mooney not at the same time right
13:54:23 sean-k-mooney if you add the adress filed you could use both but both look valid to me the first for geneve and the second for flat/vlan
13:55:07 sean-k-mooney huh interesting
13:55:51 sean-k-mooney the VFs for the connectx-6 on the bluefiled 2 have the same vendor and product id as a normal connectx-6
13:59:59 bauzas elodilles: sure, please do it, I'll just update the wikipage after you
14:06:46 dmitriis sean-k-mooney: ack on the address field usage.
14:07:57 dmitriis sean-k-mooney: I don't have a separate ConnectX-6 at hand but BF2 has ConnectX-6 in it. Let me check the PCI ID DB - I think I've seen different ids but maybe that's for something else.
14:11:10 elodilles bauzas: done, thanks (i might overused the info and link markers o:) feel free to edit :))
14:11:29 bauzas elodilles: ack, thanks
14:13:07 dmitriis Subsystem: 15b3:0061
14:13:07 dmitriis 82:00.0 0200: 15b3:a2d6 (rev 01)
14:13:07 dmitriis 82:00.0 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01)
14:13:07 dmitriis PF:
14:13:07 dmitriis sean-k-mooney: so the PF is different but VFs look like the ones from a "regular" ConnectX-6.
14:13:23 dmitriis 82:00.3 Ethernet controller: Mellanox Technologies ConnectX Family mlx5Gen Virtual Function (rev 01)
14:13:23 dmitriis VF:
14:13:24 dmitriis Subsystem: 15b3:0061
14:13:24 dmitriis 82:00.3 0200: 15b3:101e (rev 01)
14:14:15 dmitriis so careful "remote_managed" tagging is needed
14:14:44 sean-k-mooney ack good to know
14:29:21 sean-k-mooney dmitriis: you can use the adress of the pf and vendor id of the VF to whitelist all the VFs that belog to that pf
14:29:24 sean-k-mooney just so you know
14:29:44 sean-k-mooney dmitriis: that behavior is not well known
14:32:37 dmitriis sean-k-mooney: didn't know that (not surprisingly), thanks for the info.
14:33:14 dmitriis sean-k-mooney: btw, BF2 does bonding at the ARM CPU side transparently to the hypervisor
14:34:37 dmitriis and there's an option to hide the inactive PF for the hypervisor side: https://docs.mellanox.com/display/BlueFieldSWv24011082/BlueField%20Link%20Aggregation
14:35:27 dmitriis that makes it easier for OpenStack deployers/operators since only one PF needs to be taken into account
14:42:16 dmitriis sean-k-mooney: so this is the place where I had to use a string (instead of bool, not None so my reference was not correct) https://review.opendev.org/c/openstack/nova/+/812111/3/nova/network/neutron.py#2295 - that's where a device spec is dynamically generated (not based on flavor or image properties).
14:45:41 sean-k-mooney im on a call but ill look it up after thanks
14:47:31 dmitriis ack
15:03:22 Adri2000 hi, I've got a race condition issue on ussuri and victoria when resizing an instance... specifically this is with /var/lib/nova/instances on NFS, and the following happens sometimes when resizing an instance where a cold migration is triggered: `qemu-img resize` will be run on the new compute node, before the old compute node has fully released the lock on the disk file; this
15:03:25 Adri2000 will put the instance in ERROR state. does that ring a bell to anyone?
15:03:36 Adri2000 ERROR nova.compute.manager [req-...] [instance: 6ca672fd-8746-441f-bbca-6baa3234bb5e] Setting instance vm_state to ERROR: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Command: qemu-img resize /var/lib/nova/instances/6ca672fd-8746-441f-bbca-6baa3234bb5e/disk Exit code: 1 Stdout: ''
15:03:43 Adri2000 Stderr: "qemu-img: Could not open '/var/lib/nova/instances/6ca672fd-8746-441f-bbca-6baa3234bb5e/disk': Could not open '/var/lib/nova/instances/6ca672fd-8746-441f-bbca-6baa3234bb5e/disk': Permission denied\n"
15:04:09 sean-k-mooney Adri2000: are you using nfsv3
15:04:37 Adri2000 sean-k-mooney: `/var/lib/nova/instances type nfs4 (rw,relatime,vers=4.1...`
15:05:10 sean-k-mooney ok nfsv3 has locking issues v4.1 improves the situration but recommend v4.2+
15:05:21 sean-k-mooney lyarwood: does ^ seem familar to you
15:07:54 sean-k-mooney Adri2000: i belive there are some tunabel in the mount option that can be used to help resolve this
15:11:37 sean-k-mooney Adri2000: are you using raw images?
15:11:57 bauzas gibi: I'll have to hardstop our meeting by 5:50pm our TZ
15:12:15 gibi bauzas: ack
15:12:27 bauzas in case we have to continue discussing, could you be chairing it ?
15:13:32 sean-k-mooney dmitriis: oh there
15:13:42 sean-k-mooney str(self._is_remote_managed(vnic_type)),
15:13:53 sean-k-mooney dmitriis: ya that makes sense
15:13:57 Adri2000 sean-k-mooney: qcow3 images. one nfs option I have currently is local_lock=none, maybe I should look into this one.
15:14:18 sean-k-mooney dmitriis: technicaly the tags are defiend to be of type String
15:14:31 sean-k-mooney so its a dict of string to string
15:15:12 sean-k-mooney Adri2000: ack the reason i asked is apparently the lockign behavior is differnt in qemu for raw vs qcow
15:15:16 dmitriis sean-k-mooney: ack
15:17:26 sean-k-mooney dmitriis: https://github.com/openstack/nova/blob/master/nova/pci/devspec.py#L262-L263
15:18:27 dmitriis sean-k-mooney: yep, makes sense
15:18:51 dmitriis sean-k-mooney: btw, the ovn-vif repo is now up under ovn-org https://github.com/ovn-org/ovn-vif
15:19:06 sean-k-mooney yes i saw your comment
15:19:10 sean-k-mooney just looking at the code
15:19:17 dmitriis ack
15:19:44 sean-k-mooney am i right in assuming we do not want to allow these device to be used for flavor based pci passthouhg
15:20:38 dmitriis sean-k-mooney: yes, they won't be of much use without being plugged appropriately. Not the VFs at least.
15:21:00 sean-k-mooney ya
15:21:11 sean-k-mooney im wondering if we should explictly block that
15:21:44 sean-k-mooney unfortunetly i dont see a trivial way to do that
15:22:05 sean-k-mooney although we might already do that
15:22:20 dmitriis sean-k-mooney: I guess we could exclude devices from search results if remote_managed is present but not requestedd
15:22:46 sean-k-mooney yep
15:22:58 sean-k-mooney i was just going to provide an exmaple
15:23:04 sean-k-mooney we do this in other cases already
15:23:33 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/pci/stats.py#L411-L433
15:23:46 sean-k-mooney That filteres out PF if you did not ask for one
15:24:08 sean-k-mooney dmitriis: so you can copy past https://github.com/openstack/nova/blob/master/nova/pci/stats.py#L520-L535
15:24:27 sean-k-mooney and then add a new function that will filter out remote managed device if not requested
15:24:49 dmitriis sean-k-mooney: https://review.opendev.org/c/openstack/nova/+/812111/3/nova/network/neutron.py#2295

Earlier   Later