Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-02-07
15:55:12 sean-k-mooney with the neutron agetn deployed on the smartnic
15:55:14 dmitriis sean-k-mooney: they have a special config option for the neutron-openvswitch-agent as well
15:55:24 sean-k-mooney yes
15:55:55 sean-k-mooney dmitriis: the possible problem i am seing is when we go to scedule the vm we have not bound the port
15:56:08 sean-k-mooney since we have not selected a host yet
15:56:23 sean-k-mooney so i dont think we will be able to tell the difference between the ironic usage and the new usage
15:56:33 sean-k-mooney and we wont know if we shoudl request a VF or not
15:57:01 sean-k-mooney so we might need to use a differnt off-path vnic type
15:57:04 sean-k-mooney instead
15:57:29 dmitriis sean-k-mooney: in Nova we decide if we want to request a remote_managed port or not
15:57:36 dmitriis and add that to a InstancePCIRequest
15:57:48 sean-k-mooney dmitriis: yes based on the vnic_type right
15:57:51 dmitriis yes
15:58:01 sean-k-mooney right so if we only look at that we need a new vnic type
15:58:27 sean-k-mooney since that decision need to be made before we schdule and therefor cannot depend on the host or driver
15:58:27 dmitriis but that's in Nova, trying to figure out how that affects ironic
15:58:43 sean-k-mooney booting ironic service via nova api
15:58:47 sean-k-mooney *servers
15:59:05 sean-k-mooney in both cases the vnic_type would be the same
15:59:20 sean-k-mooney the falvor woudl be differnt but we would not know its an ironic flavor
16:00:21 sean-k-mooney dmitriis: so i think we need to quickly add a new vnic in neutron-lib and update the nova and neutron code to use that
16:01:13 opendevreview Takashi Kajinami proposed openstack/nova master: Allow authorization by user_id for server resume action https://review.opendev.org/c/openstack/nova/+/828168
16:01:39 opendevreview Takashi Kajinami proposed openstack/nova master: Allow authorization by user_id for server resume action https://review.opendev.org/c/openstack/nova/+/828168
16:03:00 dmitriis sean-k-mooney: we could, trying to think if there's anything we've missed that could allow us to avoid that. We originally wanted to add a new VNIC type but then decided to reuse VNIC_TYPE_SMARTNIC after some reviews.
16:03:36 sean-k-mooney yep i proably suggested the reuse :) but i think its proably required
16:03:46 sean-k-mooney *the new vnic
16:03:59 sean-k-mooney i dont think there is anything else on the port we can use to differenciate
16:05:31 sean-k-mooney oh same
16:10:32 opendevreview Takashi Kajinami proposed openstack/nova master: Allow authorization by user_id for server resume action https://review.opendev.org/c/openstack/nova/+/828168
16:38:19 gibi so if we always translate the smartnic vnic_type to InstancePCIRequest but ironic support smartnic with ml2/ovs then we might have a problem
16:38:39 gibi do we require ironic to use some custom resource in the flavor or it is just an option?
16:42:14 sean-k-mooney am we require cpu, ram and disk to be overriden to resources:cpu=0
16:46:33 gibi that could be a clue
16:46:44 sean-k-mooney im not sure if we want to rely on that
16:46:47 sean-k-mooney we could
16:46:51 gibi yeah it is hackins
16:46:52 gibi hackis
16:47:11 gibi when we have the lib freeze?
16:47:20 sean-k-mooney i belive in a week or two
16:47:25 sean-k-mooney ill check
16:47:41 gibi feb 18
16:47:53 sean-k-mooney ya
16:48:00 sean-k-mooney actully proably 17th
16:48:03 gibi yeah
16:48:32 sean-k-mooney so we still have time to add the value
16:48:40 gibi lets try the new vnic_type way then
16:48:54 gibi that would be clean
16:49:04 sean-k-mooney so vnic_type=off-path
16:49:18 sean-k-mooney actully maybe vnic_type=off-path-direct
16:49:47 sean-k-mooney so that we can have vnic_type=off-path-vdpa or off-path-mev or whaterver later if needed
16:49:50 sean-k-mooney but keep the pattern
16:50:24 gibi yes, off-path or remote-managed whathever floats the neturon team's boat
16:51:07 sean-k-mooney remote-managed might be better ya
16:51:16 sean-k-mooney keep the continuity with the config option
16:56:11 dmitriis sean-k-mooney, gibi: been trying to find something to use as a clue (such as image or flavor metadata)
16:56:19 dmitriis but, yes, it's tricky to rely on that
17:01:21 sean-k-mooney dmitriis: i dont think there is really anything and really it shoudl be soemthing on the port anyway
17:01:38 sean-k-mooney we could look at the flaovr but that is messy
17:02:39 dmitriis sean-k-mooney: ovs hardware offload uses binding:profile to differentiate between legacy SR-IOV and offload cases https://docs.openstack.org/neutron/latest/admin/config-ovs-offload.html#validate-open-vswitch-hardware-offloading
17:02:51 dmitriis been considering utilizing something like that as well
17:02:53 sean-k-mooney dmitriis: the port is not bound at this point
17:03:12 sean-k-mooney we cant use anything that depend on the host/neutron backend
17:03:27 sean-k-mooney since the host is only selected after the scheduling
17:03:58 dmitriis sean-k-mooney: yeah, that's the unfortunate thing - I don't know the driver before scheduling happens
17:03:58 sean-k-mooney dmitriis also --binding-profile '{"capabilities": ["switchdev"]}' i dot think was ever actully implemented
17:04:14 dmitriis sean-k-mooney: ah, right we discussed that way back
17:04:24 sean-k-mooney it also would be admin only since binind-profile is admin only
17:04:29 sean-k-mooney yep
17:04:43 dmitriis sean-k-mooney: yes, that would be a shame if it was admin-only
17:04:54 dmitriis sean-k-mooney: ok, I am convinced we need a new VNIC type then
17:05:13 dmitriis sean-k-mooney: I'll propose a change and, I suppose, spec changes too
17:05:26 dmitriis hopefully I'll get quick turnaround from the Neutron team
17:05:31 dmitriis fnordahl ^ FYI
17:05:42 sean-k-mooney yep im happy to review the spec change and approve once neutron agree
17:22:35 dmitriis sean-k-mooney: proposed a spec change for Neutron: https://review.opendev.org/c/openstack/neutron-specs/+/828173. Will upload a neutron-lib change and also raise a Nova spec update.
17:26:52 sean-k-mooney dmitriis: ack.
17:38:32 chateaulav sean-k-mooney: i think i found it, essentially have to tell nova to set `cpu = None` so that no topology is identified. https://usercontent.irccloud-cdn.com/file/EAK0qvLu/riscv
17:39:21 sean-k-mooney topology
17:39:26 sean-k-mooney oh pci topology
17:39:43 sean-k-mooney hum i dont know if None is a supported value or if that is stable
17:39:45 chateaulav https://usercontent.irccloud-cdn.com/file/5U9wBUXt/image.png
17:39:59 sean-k-mooney e.g. does that just mean "qemu you decide"
17:40:42 sean-k-mooney is that new code your addign to nova
17:40:49 sean-k-mooney settign the cpu element to None
17:40:58 sean-k-mooney for risxv64
17:41:15 sean-k-mooney ]hum that might break other features
17:41:29 sean-k-mooney it will disable the topology but also numa affinity
17:41:55 sean-k-mooney have you tried settign the sockets and thread to 1
17:42:11 chateaulav gonna do more testing, but that is locally currently.
17:42:40 sean-k-mooney ack. try hw:cpu_sockets=1 hw:cpu_threads=1
17:42:41 chateaulav yeah i did no 'cpu.model = None' first and it did not like that
17:43:04 sean-k-mooney i think your current code is just not generatign the cpu element entirly
17:43:22 sean-k-mooney but if its just a toplocy issue perhaps you can generate a toplogy it will like
17:43:39 chateaulav yeah, i believe so.
17:44:04 sean-k-mooney when you dont set the falvor extra specs technially the toplogy is driver defined
17:44:18 sean-k-mooney so you can modify it to work based on the requirments
17:45:22 sean-k-mooney for legacy reason the libvirt driver genearte a toplogy of 1 socket per vcpu requested
17:45:30 sean-k-mooney which is bad for a number of reasons
17:45:56 sean-k-mooney it used to improve perfromance but i stongly doubt it does and it cause issue for windows for example
17:46:30 sean-k-mooney chateaulav: so if you need to change the default toplogy for the emulation usecase i think that is ok

Earlier   Later