| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-08-18 | |||
| 14:06:54 | gibi | dansmith: I'm not sure about the scope of the multi root functionality | |
| 14:07:12 | dansmith | and I'll try to write something more concise than my verbose sentence above, but more useful than the one I put in there that caused the confusion | |
| 14:07:49 | dansmith | sean-k-mooney: they're already together, and can't be separated without losing some of the behavior that gibi wanted to keep | |
| 14:07:50 | gibi | I think in short term keep a separate client per ComputeManager instance and note why we are doing it. | |
| 14:08:05 | dansmith | gibi: ack, sounds good | |
| 14:08:40 | gibi | I dont like the global but I checked and in most cases it is safe based on how we use it | |
| 14:08:52 | gibi | the compute manager is an exception | |
| 14:09:14 | gibi | at least due to the func test, but also might be due to ironic multiple node per compute | |
| 14:09:52 | dansmith | the global results in fewer hits to keystone and also fewer places we could fail if a call to keystone fails, and mirrors our other client behaviors | |
| 14:09:59 | dansmith | (the internal state does not, of course, but...) | |
| 14:10:16 | gibi | yeah I accept the compromise | |
| 14:10:24 | gibi | the global has pros and cons | |
| 14:10:35 | dansmith | if you really hate the global I can switch it to per-use lazy load | |
| 14:10:48 | dansmith | but aside from the state thing, I don't know why we would perfer that | |
| 14:11:02 | gibi | yeah, only the share state that makes it complicated. | |
| 14:11:05 | dansmith | (or prefer even) | |
| 14:11:45 | dansmith | ack, so we could also make each call to get the singleton generate a new local state object so that that part is not shared everywhere | |
| 14:11:57 | dansmith | I could leave a node in there with the idea in case it becomes problematic in the future | |
| 14:12:05 | gibi | if I had time I would also trim the report client to have only those methods there that depend on the shared state and move the independent functions somehere else to make it clear what is problematic | |
| 14:12:30 | dansmith | ack, there are several things we *could* do to make this cleaner for sure | |
| 14:12:49 | gibi | the extra note in the singletone works for me | |
| 14:13:31 | dansmith | ack will do that, no singleton for the compute manager and add that test | |
| 14:13:50 | gibi | ack | |
| 14:13:51 | gibi | thanks | |
| 14:17:43 | gibi | sean-k-mooney: I moved forward with the allocation candidate filtering in hardware.py step. That made me realize two things 1) placement tells us RP uuids in the allocation candidates and the stats hardware.py code needs to map those RP uuids to pools. For that I need extra information in the pool. 2) the prefilters are run in the scheduler so creating RequestGroups there are good for scheduling but | |
| 14:17:49 | gibi | not good for using those RequestGroups (especiall the provider mapping in them) later to drive the PCI claim as those RequestGroups are not visible outside of the scheduler process. For the QoS port we create the groups in the compute.api so those are visible to the conductor and the compute | |
| 14:19:05 | sean-k-mooney | i thinik for cpu pinning we do it else where too | |
| 14:19:43 | gibi | as prefilters are acting on the RequestSpec I'm tempted to move the prefilter run to the conductor | |
| 14:20:05 | gibi | I can try and see what falls out | |
| 14:20:12 | gibi | how do you feel about it? | |
| 14:20:21 | sean-k-mooney | moving part of the schduleing out of the schduler | |
| 14:20:29 | sean-k-mooney | that feel like more then we need to adress this | |
| 14:20:48 | gibi | alternatively I can try to return the modified request spec from the scheduler to the conductor | |
| 14:20:55 | gibi | but that is an RPC change | |
| 14:20:59 | gibi | afaik | |
| 14:21:29 | sean-k-mooney | im looking to see what we do for PMEM and cpu pinnign i rememebr we did it differntly for one of them | |
| 14:21:36 | sean-k-mooney | before the prefileters | |
| 14:22:34 | sean-k-mooney | im fine with moving the request group creation to before the call to the schduler | |
| 14:22:56 | sean-k-mooney | moving running the prefilters to the conductor however i think it probaly mre then we want to do | |
| 14:23:06 | gibi | ack, that would be another alternative, to do the request group generation not in a prefilter | |
| 14:23:23 | gibi | that would make this similar to how QoS works | |
| 14:28:15 | sean-k-mooney | this is where we do it for cpu pinning https://github.com/openstack/nova/blob/e6aa6373d98103348a8ee3c59814350ea1556049/nova/scheduler/utils.py#L80 | |
| 14:29:15 | gibi | I think that also runs in the scheduler | |
| 14:30:20 | sean-k-mooney | the implmenation i think i sin the request spec object | |
| 14:30:42 | sean-k-mooney | oh its not | |
| 14:30:45 | sean-k-mooney | https://github.com/openstack/nova/blob/e6aa6373d98103348a8ee3c59814350ea1556049/nova/scheduler/utils.py#L305-L321 | |
| 14:31:14 | sean-k-mooney | but those are free standing fucntions that budil up the resouce class request | |
| 14:31:28 | gibi | so that works as you never need to know from where the VPMEM resource was fulfilled | |
| 14:32:01 | sean-k-mooney | ya | |
| 14:32:25 | gibi | I will figure out something along the line of cyborg and qos requests | |
| 14:32:50 | gibi | both needs the request groups after the scheduling to drive the claim on the compute | |
| 14:33:50 | sean-k-mooney | ya | |
| 14:34:09 | sean-k-mooney | we do have some pci affintiy code there by the way added by https://github.com/openstack/nova/commit/db7517d5a8aaa5a24be12d9c3453dcd98d9a887e | |
| 14:34:59 | gibi | yeah that also only extends the unsuffixed request group | |
| 14:35:17 | sean-k-mooney | yep its just finding a host that supports it | |
| 14:35:35 | sean-k-mooney | but you would potentally have to do that per pci device now | |
| 14:35:44 | sean-k-mooney | or at least eventually | |
| 14:35:59 | sean-k-mooney | since the policy is setable per alias | |
| 14:36:17 | sean-k-mooney | we can proably pretend i did not mention this for now :) | |
| 14:36:38 | sean-k-mooney | we said in the spec that we would leave numa to the numa toplogy filter | |
| 14:37:11 | sean-k-mooney | so going back to your orginal issue | |
| 14:37:28 | sean-k-mooney | you need to be able to coralate the pci pools to the placement candiates | |
| 14:37:38 | sean-k-mooney | and provider summeries | |
| 14:37:40 | gibi | hm is that settable per alias? I see a flavor extra spec hw:pci_numa_affinity_policy': 'socket' | |
| 14:37:55 | sean-k-mooney | ya it is one sec | |
| 14:38:22 | opendevreview | Dan Smith proposed openstack/nova master: Unify placement client singleton implementations https://review.opendev.org/c/openstack/nova/+/852900 | |
| 14:38:22 | opendevreview | Dan Smith proposed openstack/nova master: Avoid n-cond startup abort for keystone failures https://review.opendev.org/c/openstack/nova/+/852901 | |
| 14:38:23 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/pci/request.py#L104-L107 | |
| 14:38:26 | dansmith | gibi: ^ | |
| 14:38:35 | dansmith | (no rush) | |
| 14:38:48 | gibi | ahh we have "numa_policy": "required" in aloas | |
| 14:38:50 | gibi | alias | |
| 14:38:55 | gibi | dansmith: ack, I will look today | |
| 14:39:28 | sean-k-mooney | gibi: you can set any of the policies via the alias but they only take affect if not override by the flavor or image. | |
| 14:39:38 | sean-k-mooney | or port | |
| 14:39:45 | sean-k-mooney | but port is out of scope for now | |
| 14:40:14 | gibi | sean-k-mooney: I assume now (with a big bunch of ignorance) that it is handled by the NumaTopologyFilter properly regardless of the placement allocation :D | |
| 14:40:38 | gibi | but we have functional tests to see if this assumption holds | |
| 14:41:20 | sean-k-mooney | the numa toplogy fileter shoudl rejec tthe host if the host cant supprot it today | |
| 14:41:30 | sean-k-mooney | but its not currenlty allcoatio candiate aware | |
| 14:41:44 | sean-k-mooney | so its considring all posible devices | |
| 14:41:56 | gibi | the numa topology filter will be after my change as it calls support_request which will be a_c aware | |
| 14:42:00 | sean-k-mooney | not the ones in any one set of allcoaiton candiates | |
| 14:42:07 | sean-k-mooney | yep | |
| 14:42:13 | gibi | so it has a chance to work :) | |
| 14:42:19 | sean-k-mooney | yep | |
| 14:42:35 | sean-k-mooney | as i said we can ignor that wrinkel for now | |
| 14:42:55 | gibi | as of pool correlation with allocation candidate, I will try to add a list of RP uuids to each pool showing that where the pool gets its devices | |
| 14:43:14 | sean-k-mooney | ack. currently it shoudl be 1:1 | |
| 14:43:25 | sean-k-mooney | each pool shoudl be mapped to a singel RP correct | |
| 14:43:28 | gibi | that would be awesome if pool:RP is 1:! | |
| 14:43:29 | gibi | 1:1 | |
| 14:43:49 | gibi | I thought that VFs from two PFs might be and up in the same pool | |
| 14:43:56 | gibi | end | |
| 14:44:03 | sean-k-mooney | i think they will be two pools | |
| 14:44:07 | gibi | I will check | |
| 14:44:13 | gibi | but this sounds good at least | |
| 14:44:32 | sean-k-mooney | pools are not 1:1 to device_spec entires | |
| 14:44:33 | gibi | then I can driver the pools consumption logic based on the RP uuids in the allocation candidate | |
| 14:44:44 | sean-k-mooney | but i each PF gets its own pool | |