| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-09 | |||
| 12:43:19 | gibi | but | |
| 12:43:51 | gibi | in the PCI case if we select a host based on the fallback query then on that host the scheduler will not allocate PCI devices in placement | |
| 12:44:12 | sean-k-mooney | right so i would not use a fallback | |
| 12:44:21 | sean-k-mooney | by default we shoudl reprot the inventores to placemnt | |
| 12:44:28 | sean-k-mooney | and ahve a prefileter | |
| 12:44:43 | sean-k-mooney | the prefilter would add the pci device request to the query | |
| 12:44:52 | sean-k-mooney | and we disable it by default in zed | |
| 12:44:59 | sean-k-mooney | then enable it by default in AA | |
| 12:45:18 | sean-k-mooney | so you would rolling upgrade to Zed | |
| 12:45:29 | sean-k-mooney | then enable the prefilter once all host are upgraded | |
| 12:45:59 | sean-k-mooney | you likely would have to then do a heal-allocation like command to update the allcotiosn of existign instances | |
| 12:46:07 | gibi | ahh i see | |
| 12:46:34 | sean-k-mooney | we could also have a nova-status check | |
| 12:46:36 | gibi | so we report devices but we don't allocate yet | |
| 12:46:44 | sean-k-mooney | yep | |
| 12:46:52 | gibi | then when every compute is ready we do a heal and then start allocating | |
| 12:47:06 | sean-k-mooney | yes using the claim in the pci_devices table | |
| 12:47:26 | gibi | yepp we keep using the claim and the pci_device table | |
| 12:47:29 | gibi | anyhow | |
| 12:47:36 | gibi | as that tracks exact VF PCI addresses | |
| 12:47:41 | gibi | Placement wont | |
| 12:47:48 | sean-k-mooney | yep so form the contolers | |
| 12:48:00 | sean-k-mooney | we will have all the info in the pci_devices table to heal the allocations | |
| 12:48:17 | sean-k-mooney | since we also have the parent adresses | |
| 12:48:24 | sean-k-mooney | we can constuct the RP names | |
| 12:49:07 | gibi | yepp | |
| 12:49:39 | sean-k-mooney | we could consider | |
| 12:49:51 | sean-k-mooney | if we can activate teh filter based on min compute service version | |
| 12:50:21 | sean-k-mooney | if we were to do that we would likely need the compute agent to heal the allcoations automaticaly | |
| 12:50:38 | sean-k-mooney | perhaps on starup or in the upsadate_avaiable_resouces periodic task | |
| 12:51:29 | sean-k-mooney | im not sure if we want that level of compleixty but we already do reshapes in init_host | |
| 12:52:11 | sean-k-mooney | do you think that is too much "magic"/complexity | |
| 12:52:38 | sean-k-mooney | it would make the operator experince much nicer as it woudl just start working once everythin was upgraded | |
| 12:52:42 | gibi | this reshap will be just RP creation, we won't move things | |
| 12:52:59 | gibi | so calling that code from periodic feels OK | |
| 12:53:14 | gibi | if we have that then it is safe to enable the prefilter automatically | |
| 12:53:27 | gibi | by the compute min version | |
| 12:53:31 | sean-k-mooney | right so when the compaute-agent start with the new code. the first time it create the inventroeis it would also update the allcoations | |
| 12:53:55 | sean-k-mooney | and then the prefilter woudl activate once all computes are upgraded | |
| 12:54:01 | sean-k-mooney | based on min verion check | |
| 12:54:25 | sean-k-mooney | the proably i see with this would be move operations before the prefilter is enabled | |
| 12:54:38 | sean-k-mooney | unless we have it continue to heal | |
| 12:54:55 | sean-k-mooney | untill the min version reaces the required version | |
| 12:55:31 | sean-k-mooney | there would be some inconsitency for a time but the pci_tracker would enforce the corret behaivor with regards to not over subscibing | |
| 12:55:56 | gibi | yeah we have the pci tracker and pci claim as a fallback | |
| 12:56:03 | gibi | so we can move even if the prefilter is disabled | |
| 12:56:28 | gibi | just have to have a way to heal the placement allocation | |
| 12:56:33 | gibi | eventually | |
| 12:56:36 | sean-k-mooney | yep | |
| 12:56:48 | gibi | OK, I think I got my answers | |
| 12:56:55 | gibi | thank you for your time | |
| 12:56:59 | gibi | I really appreciate it | |
| 12:57:00 | sean-k-mooney | which again can use current and min service version to disable the healing when its not needed | |
| 12:57:04 | sean-k-mooney | no worries | |
| 12:57:12 | sean-k-mooney | im excited to see this moving forward | |
| 12:57:20 | sean-k-mooney | will you summerise this in the spec | |
| 12:57:26 | sean-k-mooney | perhaps like to the irc logs | |
| 12:57:33 | sean-k-mooney | *link | |
| 12:57:36 | gibi | I will do the summary | |
| 12:57:41 | gibi | and linking to the log | |
| 12:57:47 | gibi | then I will respin the spec | |
| 12:58:02 | gibi | and trim the questions | |
| 12:58:34 | gibi | I'm excited to stat coding up some of these in nova and watch them fail in the func env :) | |
| 12:58:39 | gibi | it will be fun | |
| 12:59:02 | sean-k-mooney | gibi: on a related not you reviewed Uggla spec. there was kind of an open question regarding updating hte AZ when you specify a host did you weigh in on that. | |
| 12:59:33 | gibi | I saw it and I think it was settled, I had no objection. But then I will doulecheck | |
| 12:59:38 | gibi | doublecheck | |
| 12:59:57 | sean-k-mooney | gibi: ack ill try and review it again shortly so | |
| 13:01:11 | sean-k-mooney | gibi: on a more selfish note i could also use your input on something else but its not super urgent https://review.opendev.org/c/openstack/nova/+/841017/1/nova/virt/libvirt/driver.py | |
| 13:02:02 | sean-k-mooney | i dont think that is 100% correct but i works for vdpa i need to test it with VFs and other vnic-types | |
| 13:02:48 | sean-k-mooney | basically we are curently unpluging neutron interface using _detach_pci_dev for suspend | |
| 13:03:07 | sean-k-mooney | that does not work for vdpa and im pretty sure it does not work in general | |
| 13:03:41 | sean-k-mooney | so i need to verify that and file a bug | |
| 13:07:15 | gibi | I never tried suspend with PCI / neutron SRIOV. So I neither confirm now deny that it works | |
| 13:07:38 | sean-k-mooney | it used to but its been a very long time since i checked it. | |
| 13:08:01 | sean-k-mooney | so ya i need to test it with differnt backends | |
| 13:08:15 | gibi | but your comments seems valid that if something is an interface then that cannot be detached as a hostdev | |
| 13:09:29 | sean-k-mooney | i have the ablity to test hardware offloaded ovs and sriov at home and i still have the servers i was usign for vdpa although ill be giving those abck today | |
| 13:10:08 | sean-k-mooney | so i can see if i can test the differnt combinations | |
| 13:10:34 | sean-k-mooney | i think self.detach_interface(context, instance, vif) shoudl work however in all cases | |
| 13:10:53 | sean-k-mooney | i dont really know why we have sepcial handelign for the host dev elements | |
| 13:11:03 | sean-k-mooney | detach_interface | |
| 13:11:26 | sean-k-mooney | is ment to be the abstraction here and its what is called when we call detach form the api | |
| 13:12:19 | gibi | yepp detach inteface dynamically use hostdev or interface config object | |
| 13:12:53 | sean-k-mooney | so i think i can just factor out the common code form https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L9811-L9829= | |
| 13:13:09 | sean-k-mooney | making the migrate_data optionall effectivly | |
| 13:13:29 | gibi | yepp | |
| 13:13:33 | gibi | that seems doable | |
| 13:13:52 | sean-k-mooney | basically i woudl jsut take in a list of vifs | |
| 13:14:57 | sean-k-mooney | so what im wonderign is it better to adapt the curent fucion as i did in the wip patch | |
| 13:15:06 | sean-k-mooney | or jsut do the refactor | |
| 13:15:24 | sean-k-mooney | and call detach_interface | |
| 13:15:49 | sean-k-mooney | via _detach_direct_passthrough_vifs | |
| 13:15:53 | gibi | I would do the refactor and call detach_inteface but I'm biased with the detach_interface code :D | |
| 13:16:12 | sean-k-mooney | well see i trust the detach_inteface code more | |
| 13:16:19 | sean-k-mooney | its better tested | |
| 13:17:06 | sean-k-mooney | ok thanks ill try and confirm my sepculation that suspend was broken and file a bug | |
| 13:17:11 | gibi | cool | |
| 13:18:10 | sean-k-mooney | one thing i need to bring up in the team meeting tomorrow is how to track the vdpa work | |
| 13:18:26 | sean-k-mooney | https://review.opendev.org/q/topic:bug%252F1970467 the non WIP patch is the bug fix | |