| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-02-07 | |||
| 16:56:19 | dmitriis | but, yes, it's tricky to rely on that | |
| 17:01:21 | sean-k-mooney | dmitriis: i dont think there is really anything and really it shoudl be soemthing on the port anyway | |
| 17:01:38 | sean-k-mooney | we could look at the flaovr but that is messy | |
| 17:02:39 | dmitriis | sean-k-mooney: ovs hardware offload uses binding:profile to differentiate between legacy SR-IOV and offload cases https://docs.openstack.org/neutron/latest/admin/config-ovs-offload.html#validate-open-vswitch-hardware-offloading | |
| 17:02:51 | dmitriis | been considering utilizing something like that as well | |
| 17:02:53 | sean-k-mooney | dmitriis: the port is not bound at this point | |
| 17:03:12 | sean-k-mooney | we cant use anything that depend on the host/neutron backend | |
| 17:03:27 | sean-k-mooney | since the host is only selected after the scheduling | |
| 17:03:58 | sean-k-mooney | dmitriis also --binding-profile '{"capabilities": ["switchdev"]}' i dot think was ever actully implemented | |
| 17:03:58 | dmitriis | sean-k-mooney: yeah, that's the unfortunate thing - I don't know the driver before scheduling happens | |
| 17:04:14 | dmitriis | sean-k-mooney: ah, right we discussed that way back | |
| 17:04:24 | sean-k-mooney | it also would be admin only since binind-profile is admin only | |
| 17:04:29 | sean-k-mooney | yep | |
| 17:04:43 | dmitriis | sean-k-mooney: yes, that would be a shame if it was admin-only | |
| 17:04:54 | dmitriis | sean-k-mooney: ok, I am convinced we need a new VNIC type then | |
| 17:05:13 | dmitriis | sean-k-mooney: I'll propose a change and, I suppose, spec changes too | |
| 17:05:26 | dmitriis | hopefully I'll get quick turnaround from the Neutron team | |
| 17:05:31 | dmitriis | fnordahl ^ FYI | |
| 17:05:42 | sean-k-mooney | yep im happy to review the spec change and approve once neutron agree | |
| 17:22:35 | dmitriis | sean-k-mooney: proposed a spec change for Neutron: https://review.opendev.org/c/openstack/neutron-specs/+/828173. Will upload a neutron-lib change and also raise a Nova spec update. | |
| 17:26:52 | sean-k-mooney | dmitriis: ack. | |
| 17:38:32 | chateaulav | sean-k-mooney: i think i found it, essentially have to tell nova to set `cpu = None` so that no topology is identified. https://usercontent.irccloud-cdn.com/file/EAK0qvLu/riscv | |
| 17:39:21 | sean-k-mooney | topology | |
| 17:39:26 | sean-k-mooney | oh pci topology | |
| 17:39:43 | sean-k-mooney | hum i dont know if None is a supported value or if that is stable | |
| 17:39:45 | chateaulav | https://usercontent.irccloud-cdn.com/file/5U9wBUXt/image.png | |
| 17:39:59 | sean-k-mooney | e.g. does that just mean "qemu you decide" | |
| 17:40:42 | sean-k-mooney | is that new code your addign to nova | |
| 17:40:49 | sean-k-mooney | settign the cpu element to None | |
| 17:40:58 | sean-k-mooney | for risxv64 | |
| 17:41:15 | sean-k-mooney | ]hum that might break other features | |
| 17:41:29 | sean-k-mooney | it will disable the topology but also numa affinity | |
| 17:41:55 | sean-k-mooney | have you tried settign the sockets and thread to 1 | |
| 17:42:11 | chateaulav | gonna do more testing, but that is locally currently. | |
| 17:42:40 | sean-k-mooney | ack. try hw:cpu_sockets=1 hw:cpu_threads=1 | |
| 17:42:41 | chateaulav | yeah i did no 'cpu.model = None' first and it did not like that | |
| 17:43:04 | sean-k-mooney | i think your current code is just not generatign the cpu element entirly | |
| 17:43:22 | sean-k-mooney | but if its just a toplocy issue perhaps you can generate a toplogy it will like | |
| 17:43:39 | chateaulav | yeah, i believe so. | |
| 17:44:04 | sean-k-mooney | when you dont set the falvor extra specs technially the toplogy is driver defined | |
| 17:44:18 | sean-k-mooney | so you can modify it to work based on the requirments | |
| 17:45:22 | sean-k-mooney | for legacy reason the libvirt driver genearte a toplogy of 1 socket per vcpu requested | |
| 17:45:30 | sean-k-mooney | which is bad for a number of reasons | |
| 17:45:56 | sean-k-mooney | it used to improve perfromance but i stongly doubt it does and it cause issue for windows for example | |
| 17:46:30 | sean-k-mooney | chateaulav: so if you need to change the default toplogy for the emulation usecase i think that is ok | |
| 17:46:33 | chateaulav | definetly, now that ive isolated the area, gonna tweak the topology settings | |
| 18:14:55 | opendevreview | Tobias Urdin proposed openstack/nova master: Cleanup old resize instances dir before resize https://review.opendev.org/c/openstack/nova/+/827865 | |
| 18:22:43 | opendevreview | Dmitrii Shcherbakov proposed openstack/nova-specs master: Late Amendments to the Off-path Backends Spec https://review.opendev.org/c/openstack/nova-specs/+/828177 | |
| 18:27:07 | dmitriis | sean-k-mooney: https://review.opendev.org/c/openstack/nova-specs/+/828177 - uploaded | |
| 18:33:21 | sean-k-mooney | dmitriis: ack am im not really happy with adding allow_remote_managed_ports to the whitelist | |
| 18:33:51 | sean-k-mooney | that feels semanticly incorrect to me | |
| 18:36:33 | sean-k-mooney | dmitriis: the docs issue on the spec repo is not related to your patch by the way | |
| 18:36:35 | dmitriis | sean-k-mooney: the alternative approach I considered was to expose a property on the whitelist which would tell us about whether remote_managed is used or not | |
| 18:36:53 | sean-k-mooney | dmitriis: that i think i coudl accpet yes | |
| 18:37:17 | sean-k-mooney | really the whitelist shoudl not really have awareness of the off-path feature | |
| 18:37:25 | dmitriis | sean-k-mooney: hmm, I felt it was slightly backwards because I am exposing state for somebody else to check but maybe I am wrong | |
| 18:37:28 | sean-k-mooney | a porperty i think woudl be ok | |
| 18:37:43 | dmitriis | I'd have to pass it through both on the whitelist and PciDeviceSpec though | |
| 18:37:58 | dmitriis | because I only know about the use of those tags in the spec object | |
| 18:38:15 | sean-k-mooney | the whitelist shoudl not be doign any validation of those tags | |
| 18:38:23 | sean-k-mooney | only the pci tracker should | |
| 18:39:01 | sean-k-mooney | the whitelist is a POD object | |
| 18:39:10 | sean-k-mooney | it does some basic validation fo the imports | |
| 18:39:48 | sean-k-mooney | but it should not be doing any filtering | |
| 18:41:09 | dmitriis | sean-k-mooney: there's some validation done here https://github.com/openstack/nova/blob/b6fe7521afa8d42febc68f5f79782f7bcc3b568f/nova/compute/manager.py#L1430 | |
| 18:41:16 | dmitriis | which is also very early | |
| 18:41:32 | dmitriis | and I have access to the whitelist here | |
| 18:41:46 | sean-k-mooney | that is in the compute manager | |
| 18:41:55 | sean-k-mooney | so its oke to do it there | |
| 18:42:05 | sean-k-mooney | but really it shoudl not be there either | |
| 18:42:14 | sean-k-mooney | it shoudl be in the resouce tracker or pci manager | |
| 18:42:34 | sean-k-mooney | in https://github.com/openstack/nova/blob/master/nova/pci/manager.py#L38 | |
| 18:43:20 | dmitriis | sean-k-mooney: yeah, this should be set up relatively early as well | |
| 18:43:44 | sean-k-mooney | you shoudl do any validation you need here https://github.com/openstack/nova/blob/master/nova/pci/manager.py#L78-L83 | |
| 18:44:49 | sean-k-mooney | its called as part of _setup_pci_tracker in teh resouce traceker | |
| 18:44:51 | sean-k-mooney | https://github.com/openstack/nova/blob/14bfbaa50ed3c000a691a37afbd20d86f431b3fb/nova/compute/resource_tracker.py#L764-L773 | |
| 18:45:04 | sean-k-mooney | which is invoked as part of _init_compute_node | |
| 18:45:58 | dmitriis | sean-k-mooney: ack. And when it comes to exposing the property, I could populate _has_remote_managed after this line https://github.com/openstack/nova/blob/b6fe7521afa8d42febc68f5f79782f7bcc3b568f/nova/pci/whitelist.py#L51 and expose a property | |
| 18:46:11 | dmitriis | and also expose a property on the spec objects themselves | |
| 18:46:42 | dmitriis | I'd have to do a search over spec objects then and see whether any one of those has a property | |
| 18:46:58 | dmitriis | if that's ok, I'll just do it | |
| 18:47:21 | sean-k-mooney | i think that shoudl be ok | |
| 18:48:01 | sean-k-mooney | it sound cleaner to me overall | |
| 18:48:29 | sean-k-mooney | where/why are you using this by the way | |
| 18:49:02 | dmitriis | sean-k-mooney: this is to make sure remote_managed tags aren't usable on a host where libvirt is too old (doesn't have VPD handling code) | |
| 18:49:25 | sean-k-mooney | right so you dont need this on the spec object or whitelist object | |
| 18:49:34 | sean-k-mooney | that is a check you can do in the libvirt driver at start up | |
| 18:49:54 | sean-k-mooney | you can just loop over the spec objects and check for the tag | |
| 18:50:04 | sean-k-mooney | and check the verisons | |
| 18:50:40 | sean-k-mooney | the device spec will already have the value in the PciDeviceSpec.tags field | |
| 18:50:49 | dmitriis | https://review.opendev.org/c/openstack/nova/+/827839/2/nova/virt/libvirt/driver.py#816 | |
| 18:51:13 | dmitriis | ^ so it's a host-based capability | |
| 18:51:20 | dmitriis | host-dependent * | |
| 18:51:36 | sean-k-mooney | yes | |
| 18:52:15 | sean-k-mooney | that does not need to you modify these data stuctures | |
| 18:52:36 | dmitriis | sean-k-mooney: I don't have access to the host object in those | |
| 18:52:46 | dmitriis | but maybe there's some other way to check the Libvirt version | |
| 18:52:53 | sean-k-mooney | you should not be checkign for them in these | |
| 18:53:03 | sean-k-mooney | dmitriis: you shoudl not be doing https://review.opendev.org/c/openstack/nova/+/827839/2/nova/pci/devspec.py#319 | |
| 18:53:23 | sean-k-mooney | that is not the correct place to raise that excption | |