| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-09 | |||
| 12:37:03 | sean-k-mooney | right so i think what stephenfin had in mind was if you whitelist the PF and it has VFs we would only expose the VFs | |
| 12:37:22 | sean-k-mooney | where as today unless you use the product_id to filter | |
| 12:37:30 | sean-k-mooney | we expos both the VFs and PFs | |
| 12:37:55 | sean-k-mooney | if we maintian the current behavior we obvilly need ot dynamically adjst the reserved value | |
| 12:38:04 | gibi | yes, that is the complexity | |
| 12:38:07 | sean-k-mooney | to emulated the unclaimable state | |
| 12:38:08 | gibi | but it is solveable | |
| 12:38:31 | gibi | I think I will keep this open for bauzas or other reviews to chime in | |
| 12:38:59 | sean-k-mooney | sure | |
| 12:39:13 | sean-k-mooney | we decided to reduce flexiblity for cpu pinning | |
| 12:39:15 | sean-k-mooney | with isolate | |
| 12:39:25 | sean-k-mooney | and we know that not everyone was happy with that | |
| 12:40:01 | sean-k-mooney | we can elect to do the same here but we need to be deliberiate about it and comunicate it well if we want to force this change | |
| 12:40:09 | ralonsoh | sean-k-mooney, sorry, I was having lunch | |
| 12:40:13 | ralonsoh | what do you need? | |
| 12:40:22 | sean-k-mooney | if we can live with the complexity then we proably shoudl keep it | |
| 12:40:28 | sean-k-mooney | ralonsoh: i found it | |
| 12:40:36 | gibi | OK, I will plan with the complexity | |
| 12:40:37 | ralonsoh | ah perfect | |
| 12:40:39 | sean-k-mooney | ralonsoh: https://review.opendev.org/q/topic:bp%252Fenable-sriov-nic-features | |
| 12:40:47 | sean-k-mooney | ralonsoh: your code for tracking pci device in placment | |
| 12:41:12 | sean-k-mooney | ralonsoh: we are just discussing the spec to enable it. gibi will be taking on that feature | |
| 12:41:20 | ralonsoh | perfect | |
| 12:41:39 | gibi | ralonsoh: https://review.opendev.org/c/openstack/nova-specs/+/791047/4/specs/zed/approved/pci-device-tracking-in-placement.rst this is the spec if you are interested :) | |
| 12:41:47 | ralonsoh | sure | |
| 12:42:01 | gibi | sean-k-mooney: so I have one more open question | |
| 12:42:08 | sean-k-mooney | gibi: go for it | |
| 12:42:09 | gibi | upgrade | |
| 12:42:19 | gibi | obviously rolling upgrade is a pain | |
| 12:42:35 | sean-k-mooney | ah well you burn down the datacenter and build a new one in the ashes | |
| 12:42:45 | sean-k-mooney | obvisouly the least painful approch | |
| 12:42:46 | gibi | in the PCPU case we did a fallback query | |
| 12:43:01 | gibi | to allow schedling to not-yet upgraded computes | |
| 12:43:05 | sean-k-mooney | yep | |
| 12:43:13 | sean-k-mooney | we could make the prefilter configurable | |
| 12:43:14 | gibi | I'm not sure how we did the allocation in that case | |
| 12:43:19 | gibi | but | |
| 12:43:51 | gibi | in the PCI case if we select a host based on the fallback query then on that host the scheduler will not allocate PCI devices in placement | |
| 12:44:12 | sean-k-mooney | right so i would not use a fallback | |
| 12:44:21 | sean-k-mooney | by default we shoudl reprot the inventores to placemnt | |
| 12:44:28 | sean-k-mooney | and ahve a prefileter | |
| 12:44:43 | sean-k-mooney | the prefilter would add the pci device request to the query | |
| 12:44:52 | sean-k-mooney | and we disable it by default in zed | |
| 12:44:59 | sean-k-mooney | then enable it by default in AA | |
| 12:45:18 | sean-k-mooney | so you would rolling upgrade to Zed | |
| 12:45:29 | sean-k-mooney | then enable the prefilter once all host are upgraded | |
| 12:45:59 | sean-k-mooney | you likely would have to then do a heal-allocation like command to update the allcotiosn of existign instances | |
| 12:46:07 | gibi | ahh i see | |
| 12:46:34 | sean-k-mooney | we could also have a nova-status check | |
| 12:46:36 | gibi | so we report devices but we don't allocate yet | |
| 12:46:44 | sean-k-mooney | yep | |
| 12:46:52 | gibi | then when every compute is ready we do a heal and then start allocating | |
| 12:47:06 | sean-k-mooney | yes using the claim in the pci_devices table | |
| 12:47:26 | gibi | yepp we keep using the claim and the pci_device table | |
| 12:47:29 | gibi | anyhow | |
| 12:47:36 | gibi | as that tracks exact VF PCI addresses | |
| 12:47:41 | gibi | Placement wont | |
| 12:47:48 | sean-k-mooney | yep so form the contolers | |
| 12:48:00 | sean-k-mooney | we will have all the info in the pci_devices table to heal the allocations | |
| 12:48:17 | sean-k-mooney | since we also have the parent adresses | |
| 12:48:24 | sean-k-mooney | we can constuct the RP names | |
| 12:49:07 | gibi | yepp | |
| 12:49:39 | sean-k-mooney | we could consider | |
| 12:49:51 | sean-k-mooney | if we can activate teh filter based on min compute service version | |
| 12:50:21 | sean-k-mooney | if we were to do that we would likely need the compute agent to heal the allcoations automaticaly | |
| 12:50:38 | sean-k-mooney | perhaps on starup or in the upsadate_avaiable_resouces periodic task | |
| 12:51:29 | sean-k-mooney | im not sure if we want that level of compleixty but we already do reshapes in init_host | |
| 12:52:11 | sean-k-mooney | do you think that is too much "magic"/complexity | |
| 12:52:38 | sean-k-mooney | it would make the operator experince much nicer as it woudl just start working once everythin was upgraded | |
| 12:52:42 | gibi | this reshap will be just RP creation, we won't move things | |
| 12:52:59 | gibi | so calling that code from periodic feels OK | |
| 12:53:14 | gibi | if we have that then it is safe to enable the prefilter automatically | |
| 12:53:27 | gibi | by the compute min version | |
| 12:53:31 | sean-k-mooney | right so when the compaute-agent start with the new code. the first time it create the inventroeis it would also update the allcoations | |
| 12:53:55 | sean-k-mooney | and then the prefilter woudl activate once all computes are upgraded | |
| 12:54:01 | sean-k-mooney | based on min verion check | |
| 12:54:25 | sean-k-mooney | the proably i see with this would be move operations before the prefilter is enabled | |
| 12:54:38 | sean-k-mooney | unless we have it continue to heal | |
| 12:54:55 | sean-k-mooney | untill the min version reaces the required version | |
| 12:55:31 | sean-k-mooney | there would be some inconsitency for a time but the pci_tracker would enforce the corret behaivor with regards to not over subscibing | |
| 12:55:56 | gibi | yeah we have the pci tracker and pci claim as a fallback | |
| 12:56:03 | gibi | so we can move even if the prefilter is disabled | |
| 12:56:28 | gibi | just have to have a way to heal the placement allocation | |
| 12:56:33 | gibi | eventually | |
| 12:56:36 | sean-k-mooney | yep | |
| 12:56:48 | gibi | OK, I think I got my answers | |
| 12:56:55 | gibi | thank you for your time | |
| 12:56:59 | gibi | I really appreciate it | |
| 12:57:00 | sean-k-mooney | which again can use current and min service version to disable the healing when its not needed | |
| 12:57:04 | sean-k-mooney | no worries | |
| 12:57:12 | sean-k-mooney | im excited to see this moving forward | |
| 12:57:20 | sean-k-mooney | will you summerise this in the spec | |
| 12:57:26 | sean-k-mooney | perhaps like to the irc logs | |
| 12:57:33 | sean-k-mooney | *link | |
| 12:57:36 | gibi | I will do the summary | |
| 12:57:41 | gibi | and linking to the log | |
| 12:57:47 | gibi | then I will respin the spec | |
| 12:58:02 | gibi | and trim the questions | |
| 12:58:34 | gibi | I'm excited to stat coding up some of these in nova and watch them fail in the func env :) | |
| 12:58:39 | gibi | it will be fun | |