| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-08-18 | |||
| 14:13:50 | gibi | ack | |
| 14:13:51 | gibi | thanks | |
| 14:17:43 | gibi | sean-k-mooney: I moved forward with the allocation candidate filtering in hardware.py step. That made me realize two things 1) placement tells us RP uuids in the allocation candidates and the stats hardware.py code needs to map those RP uuids to pools. For that I need extra information in the pool. 2) the prefilters are run in the scheduler so creating RequestGroups there are good for scheduling but | |
| 14:17:49 | gibi | not good for using those RequestGroups (especiall the provider mapping in them) later to drive the PCI claim as those RequestGroups are not visible outside of the scheduler process. For the QoS port we create the groups in the compute.api so those are visible to the conductor and the compute | |
| 14:19:05 | sean-k-mooney | i thinik for cpu pinning we do it else where too | |
| 14:19:43 | gibi | as prefilters are acting on the RequestSpec I'm tempted to move the prefilter run to the conductor | |
| 14:20:05 | gibi | I can try and see what falls out | |
| 14:20:12 | gibi | how do you feel about it? | |
| 14:20:21 | sean-k-mooney | moving part of the schduleing out of the schduler | |
| 14:20:29 | sean-k-mooney | that feel like more then we need to adress this | |
| 14:20:48 | gibi | alternatively I can try to return the modified request spec from the scheduler to the conductor | |
| 14:20:55 | gibi | but that is an RPC change | |
| 14:20:59 | gibi | afaik | |
| 14:21:29 | sean-k-mooney | im looking to see what we do for PMEM and cpu pinnign i rememebr we did it differntly for one of them | |
| 14:21:36 | sean-k-mooney | before the prefileters | |
| 14:22:34 | sean-k-mooney | im fine with moving the request group creation to before the call to the schduler | |
| 14:22:56 | sean-k-mooney | moving running the prefilters to the conductor however i think it probaly mre then we want to do | |
| 14:23:06 | gibi | ack, that would be another alternative, to do the request group generation not in a prefilter | |
| 14:23:23 | gibi | that would make this similar to how QoS works | |
| 14:28:15 | sean-k-mooney | this is where we do it for cpu pinning https://github.com/openstack/nova/blob/e6aa6373d98103348a8ee3c59814350ea1556049/nova/scheduler/utils.py#L80 | |
| 14:29:15 | gibi | I think that also runs in the scheduler | |
| 14:30:20 | sean-k-mooney | the implmenation i think i sin the request spec object | |
| 14:30:42 | sean-k-mooney | oh its not | |
| 14:30:45 | sean-k-mooney | https://github.com/openstack/nova/blob/e6aa6373d98103348a8ee3c59814350ea1556049/nova/scheduler/utils.py#L305-L321 | |
| 14:31:14 | sean-k-mooney | but those are free standing fucntions that budil up the resouce class request | |
| 14:31:28 | gibi | so that works as you never need to know from where the VPMEM resource was fulfilled | |
| 14:32:01 | sean-k-mooney | ya | |
| 14:32:25 | gibi | I will figure out something along the line of cyborg and qos requests | |
| 14:32:50 | gibi | both needs the request groups after the scheduling to drive the claim on the compute | |
| 14:33:50 | sean-k-mooney | ya | |
| 14:34:09 | sean-k-mooney | we do have some pci affintiy code there by the way added by https://github.com/openstack/nova/commit/db7517d5a8aaa5a24be12d9c3453dcd98d9a887e | |
| 14:34:59 | gibi | yeah that also only extends the unsuffixed request group | |
| 14:35:17 | sean-k-mooney | yep its just finding a host that supports it | |
| 14:35:35 | sean-k-mooney | but you would potentally have to do that per pci device now | |
| 14:35:44 | sean-k-mooney | or at least eventually | |
| 14:35:59 | sean-k-mooney | since the policy is setable per alias | |
| 14:36:17 | sean-k-mooney | we can proably pretend i did not mention this for now :) | |
| 14:36:38 | sean-k-mooney | we said in the spec that we would leave numa to the numa toplogy filter | |
| 14:37:11 | sean-k-mooney | so going back to your orginal issue | |
| 14:37:28 | sean-k-mooney | you need to be able to coralate the pci pools to the placement candiates | |
| 14:37:38 | sean-k-mooney | and provider summeries | |
| 14:37:40 | gibi | hm is that settable per alias? I see a flavor extra spec hw:pci_numa_affinity_policy': 'socket' | |
| 14:37:55 | sean-k-mooney | ya it is one sec | |
| 14:38:22 | opendevreview | Dan Smith proposed openstack/nova master: Unify placement client singleton implementations https://review.opendev.org/c/openstack/nova/+/852900 | |
| 14:38:22 | opendevreview | Dan Smith proposed openstack/nova master: Avoid n-cond startup abort for keystone failures https://review.opendev.org/c/openstack/nova/+/852901 | |
| 14:38:23 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/pci/request.py#L104-L107 | |
| 14:38:26 | dansmith | gibi: ^ | |
| 14:38:35 | dansmith | (no rush) | |
| 14:38:48 | gibi | ahh we have "numa_policy": "required" in aloas | |
| 14:38:50 | gibi | alias | |
| 14:38:55 | gibi | dansmith: ack, I will look today | |
| 14:39:28 | sean-k-mooney | gibi: you can set any of the policies via the alias but they only take affect if not override by the flavor or image. | |
| 14:39:38 | sean-k-mooney | or port | |
| 14:39:45 | sean-k-mooney | but port is out of scope for now | |
| 14:40:14 | gibi | sean-k-mooney: I assume now (with a big bunch of ignorance) that it is handled by the NumaTopologyFilter properly regardless of the placement allocation :D | |
| 14:40:38 | gibi | but we have functional tests to see if this assumption holds | |
| 14:41:20 | sean-k-mooney | the numa toplogy fileter shoudl rejec tthe host if the host cant supprot it today | |
| 14:41:30 | sean-k-mooney | but its not currenlty allcoatio candiate aware | |
| 14:41:44 | sean-k-mooney | so its considring all posible devices | |
| 14:41:56 | gibi | the numa topology filter will be after my change as it calls support_request which will be a_c aware | |
| 14:42:00 | sean-k-mooney | not the ones in any one set of allcoaiton candiates | |
| 14:42:07 | sean-k-mooney | yep | |
| 14:42:13 | gibi | so it has a chance to work :) | |
| 14:42:19 | sean-k-mooney | yep | |
| 14:42:35 | sean-k-mooney | as i said we can ignor that wrinkel for now | |
| 14:42:55 | gibi | as of pool correlation with allocation candidate, I will try to add a list of RP uuids to each pool showing that where the pool gets its devices | |
| 14:43:14 | sean-k-mooney | ack. currently it shoudl be 1:1 | |
| 14:43:25 | sean-k-mooney | each pool shoudl be mapped to a singel RP correct | |
| 14:43:28 | gibi | that would be awesome if pool:RP is 1:! | |
| 14:43:29 | gibi | 1:1 | |
| 14:43:49 | gibi | I thought that VFs from two PFs might be and up in the same pool | |
| 14:43:56 | gibi | end | |
| 14:44:03 | sean-k-mooney | i think they will be two pools | |
| 14:44:07 | gibi | I will check | |
| 14:44:13 | gibi | but this sounds good at least | |
| 14:44:32 | sean-k-mooney | pools are not 1:1 to device_spec entires | |
| 14:44:33 | gibi | then I can driver the pools consumption logic based on the RP uuids in the allocation candidate | |
| 14:44:44 | sean-k-mooney | but i each PF gets its own pool | |
| 14:44:57 | sean-k-mooney | and VFs form differnt PFs are seperate | |
| 14:45:36 | sean-k-mooney | by the way if that is not thet case today i don tsee any reason we cant change it to make it 1:1 | |
| 14:45:59 | gibi | yeah that would have been my next proposal :) | |
| 14:46:52 | sean-k-mooney | the pools are stored in teh pci_stats object in the compute node recored | |
| 14:47:11 | gibi | hehe, I already have a note where to create RequestGroups from flavor https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L534-L540 | |
| 14:48:17 | sean-k-mooney | hehe ya i was expecting it to be in the request_spec object | |
| 14:48:25 | sean-k-mooney | for cpu ectra | |
| 14:48:46 | sean-k-mooney | not the scudler utils | |
| 14:50:21 | gibi | ohh, and I even tried to resolve that todo at some point https://review.opendev.org/c/openstack/nova/+/647396 | |
| 14:51:00 | sean-k-mooney | hehe you have too many commits :P | |
| 14:53:32 | gibi | OK at least the commit message confirms my current view | |
| 14:54:03 | gibi | and adding just the pci alias related groups from the flavor does not seem problematic at that point | |
| 14:55:31 | sean-k-mooney | ya we already require that the alias defienition is the same on the api and the compute nodes | |
| 14:57:50 | sean-k-mooney | so you should be able to use the alsis form the current config safely | |
| 14:57:50 | sean-k-mooney | actully that does not matter you are not changing the requirement | |
| 14:58:07 | gibi | I don't think I depend on the alias on the compute but good to know | |
| 14:58:38 | sean-k-mooney | resize required the alisa to be the same to create the correct pci requests i belive | |
| 14:59:13 | gibi | hm, interesting | |
| 15:00:06 | sean-k-mooney | i have to join a call but we have docs about it | |
| 15:01:05 | sean-k-mooney | https://docs.openstack.org/nova/latest/admin/pci-passthrough.html#configure-nova-api i think we droped the reason form the doc | |
| 15:08:19 | JayF | melwitt: ack; thank you. Assuming you are OK if I want to run with trying to get those landed? | |
| 15:09:20 | melwitt | JayF: yes, please feel free | |