| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-30 | |||
| 14:11:05 | fungi | fungi99 was more flexible, even if you wound up with a lot more templating and boilerplate | |
| 14:11:41 | fungi | though my heart still belongs to classic k&r fungi | |
| 14:47:53 | ralonsoh | stephenfin, hi again | |
| 14:47:54 | ralonsoh | https://review.opendev.org/#/c/739961/ | |
| 14:48:03 | ralonsoh | if you have a bit of time | |
| 14:52:09 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: DNM: testing tempest copy-image test https://review.opendev.org/742547 | |
| 14:55:41 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: DNM: testing tempest copy-image test https://review.opendev.org/742547 | |
| 14:59:11 | stephenfin | ralonsoh: How do you plan to use these traits? That's still not entirely clear to me | |
| 14:59:41 | ralonsoh | stephenfin, if needed, that will be enforced in the port info | |
| 14:59:41 | sean-k-mooney | stephenfin: we will need nova changes next cycle to report the traits | |
| 14:59:48 | stephenfin | what resource providers will they be reported on, and who sets them? | |
| 14:59:54 | ralonsoh | let me check where exactly | |
| 15:00:30 | sean-k-mooney | stephenfin: initally they will be reported on the compute node root rp if either sriov is enabled or numa vswitches | |
| 15:00:56 | ralonsoh | stephenfin, https://review.opendev.org/#/c/740011/4/specs/victoria/port-numa-affinity-policy.rst@70 | |
| 15:01:03 | sean-k-mooney | stephenfin: if we start tracking pci device in plamcnet then it will be on the pci device RP | |
| 15:01:03 | ralonsoh | is this what you are asking? | |
| 15:01:08 | stephenfin | ah, there's a spec. That's what I was looking for | |
| 15:02:21 | ralonsoh | but yes, we are not pointing to any specific RP | |
| 15:04:12 | stephenfin | this doesn't feel like a trait or something you'd want to report | |
| 15:04:42 | ralonsoh | stephenfin, ? | |
| 15:04:52 | stephenfin | so with this feature, we're saying you'll be able to state whether NUMA affinity is important or not on a port-by-port basis | |
| 15:05:05 | sean-k-mooney | yes | |
| 15:05:06 | ralonsoh | yes | |
| 15:05:10 | stephenfin | cool, so what do we need the trait for | |
| 15:05:21 | ralonsoh | to specify the policy | |
| 15:05:22 | sean-k-mooney | to select host that supprot numa affinity | |
| 15:06:02 | stephenfin | that's a single trait, not three | |
| 15:06:05 | sean-k-mooney | stephenfin: its mostly required for numa_vswitchs | |
| 15:06:14 | stephenfin | SUPPORTS_CONFIGURABLE_NUMA_AFFINITY | |
| 15:06:20 | stephenfin | or something like that | |
| 15:06:38 | sean-k-mooney | not really numa_vsiwtch only support stict affinity today | |
| 15:06:48 | sean-k-mooney | not all 3 | |
| 15:07:20 | sean-k-mooney | although we woudl be changing that as part of the nova half of the feature | |
| 15:07:33 | stephenfin | so it wouldn't report SUPPORTS_CONFIGURABLE_NUMA_AFFINITY | |
| 15:07:42 | stephenfin | (again, just a placeholder trait name) | |
| 15:08:05 | sean-k-mooney | stephenfin: perhaps but i think knowing which policy is support is imporant | |
| 15:08:26 | sean-k-mooney | im also expecting this to be resued for pmem and vgpus | |
| 15:08:31 | stephenfin | are there scenarios where we'd report each one of the traits by themselves? | |
| 15:08:51 | ralonsoh | only one per port | |
| 15:08:53 | sean-k-mooney | yes | |
| 15:09:03 | sean-k-mooney | maybe | |
| 15:09:16 | sean-k-mooney | i need to check if the numa policy is in the alias or pci white list | |
| 15:09:21 | stephenfin | ralonsoh: Sorry, my time's been split multiple different ways so I haven't gotten to sunk my teeth into this before now. Just teasing things out before we merge something we can't undo :) | |
| 15:09:23 | sean-k-mooney | if its in the whitelist then no | |
| 15:09:31 | sean-k-mooney | * alias | |
| 15:09:37 | ralonsoh | stephenfin, sure | |
| 15:10:41 | sean-k-mooney | ok so the numa policy is part of the alias https://github.com/openstack/nova/blob/master/nova/pci/request.py#L93 | |
| 15:11:03 | sean-k-mooney | so we dont need to cater for reporting device as only consumable with a singel policy | |
| 15:11:10 | sean-k-mooney | its not defiend in the whitelist | |
| 15:11:15 | stephenfin | right, though it's only done that way because aliases are as granular as we get | |
| 15:12:13 | stephenfin | we didn't (don't) have a mechanism to say use the PCI device indicated by this alias and don't worry about NUMA affinity | |
| 15:12:26 | sean-k-mooney | yes we do | |
| 15:12:44 | sean-k-mooney | oh you mean per alias | |
| 15:12:46 | stephenfin | the flavor extra spec applies to all PCI devices, not just one | |
| 15:12:48 | sean-k-mooney | right we dont | |
| 15:12:49 | stephenfin | yes | |
| 15:12:56 | stephenfin | right | |
| 15:13:28 | stephenfin | so that's still a TODO after this but we'll be able to do it for SR-IOV ports at least | |
| 15:13:46 | stephenfin | but the ability to do that smells like a simple binary capability | |
| 15:14:12 | sean-k-mooney | stephenfin: we kind of hoped to stop using the alias and move pci deivce to the top level api or use cyborg | |
| 15:14:21 | ralonsoh | but with this information in the port we don't need to limit that to sriov ports only | |
| 15:14:27 | sean-k-mooney | which is why it was not added orginally | |
| 15:14:37 | stephenfin | i.e. I allow you to configure NUMA affinity of individual resources or I don't | |
| 15:14:47 | sean-k-mooney | ralonsoh: right we are not | |
| 15:15:20 | sean-k-mooney | the trait is not technially need we can do the enforcement in the numa toplogy filter without it | |
| 15:15:48 | sean-k-mooney | but with out the trait we do that the proablem that plamcnet may not return hosts with the capablity if the limit is set too low | |
| 15:15:52 | stephenfin | ralonsoh: So I guess you're thinking we could have multiple child RPs, e.g. one for the OVS network, one for the SR-IOV device, one for a generic PCI device etc., and each could support their own type of policy? | |
| 15:16:09 | ralonsoh | that's the goal | |
| 15:16:23 | sean-k-mooney | which iw why we should have a trait but i guess it does not need to be 3 | |
| 15:16:52 | stephenfin | Hmm, okay, that sounds reasonable. I'd been thinking the trait would be reported on the root provider (the compute node) | |
| 15:17:05 | stephenfin | which makes little sense because you provide no granularity that way | |
| 15:17:08 | sean-k-mooney | by the way this is part of https://bugzilla.redhat.com/show_bug.cgi?id=1780721 and https://bugzilla.redhat.com/show_bug.cgi?id=1791834 | |
| 15:17:08 | openstack | bugzilla.redhat.com bug 1780721 in openstack-nova "[RFE] Optional NUMA affinity for neutron ports" [High,New] - Assigned to smooney | |
| 15:17:09 | openstack | bugzilla.redhat.com bug 1791834 in openstack-neutron "[RFE] Optional NUMA affinity for neutron ports" [High,New] - Assigned to amuller | |
| 15:17:33 | ralonsoh | yes | |
| 15:17:49 | stephenfin | sean-k-mooney: oh yeah, I understand and agree with the feature as a whole. It's just this trait bit that's tripping me up | |
| 15:18:11 | sean-k-mooney | stephenfin: right now if we said we want strict affinity we have no way to guarrentee that | |
| 15:19:14 | sean-k-mooney | becasue placment if limit=10 could select 10 allocation candated form host that dont supprot numa affinity | |
| 15:19:33 | sean-k-mooney | the trait is basicaly to make it work in deployment like cerns where they have a really low limit | |
| 15:19:47 | stephenfin | the solution to that is NUMA in placement though, no? | |
| 15:19:52 | sean-k-mooney | no | |
| 15:20:12 | stephenfin | model NUMA nodes as RPs in placement. Say all resources must share the same root RP (NUMA node) | |
| 15:20:14 | sean-k-mooney | numa in placmnet wont advertise which host have numa aware vswitches | |
| 15:20:40 | sean-k-mooney | since we dont model netowrk backend ports as a resouce | |
| 15:21:32 | sean-k-mooney | stephenfin: we would some how have to model the pysnet numa affinity in the rp tree | |
| 15:21:58 | ralonsoh | we have this information in nova | |
| 15:22:09 | sean-k-mooney | yes but we cant model it in plamcnet | |
| 15:22:10 | ralonsoh | more or less: you have the numa nodes of a physnet | |
| 15:22:13 | stephenfin | sean-k-mooney: how does this help? | |
| 15:22:26 | sean-k-mooney | it just allow me to know the host has the feature | |
| 15:22:41 | sean-k-mooney | i guess we could have 3 different tratis. | |
| 15:23:02 | sean-k-mooney | one for numa-vsiwthc, one for sriov and one for numa affinity | |
| 15:23:06 | sean-k-mooney | but that feels messy | |
| 15:23:25 | stephenfin | You mean it has the NUMA-aware vSwitches feature enabled? | |
| 15:23:31 | sean-k-mooney | yes | |
| 15:23:40 | openstackgerrit | Jie Li proposed openstack/nova-specs master: add volume backed server rebuild spec https://review.opendev.org/739349 | |
| 15:23:56 | sean-k-mooney | if it was a non sriov port and it asked for strict affinity | |
| 15:24:04 | sean-k-mooney | it need to land on a host with numa aware vswtichs | |
| 15:24:08 | sean-k-mooney | or we shoudl reject it | |
| 15:24:31 | sean-k-mooney | legacy and prefer not require that | |