| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-23 | |||
| 10:38:18 | gibi | the expected tree would be computeRP <- neutron agent RP <- deviceRP | |
| 10:38:35 | gibi | but after a bugfix in Ussuri it was changed to | |
| 10:38:38 | bauzas | gibi: related to the ML thread I saw ? | |
| 10:39:03 | gibi | computeRP <- neutron agent RP | |
| 10:39:06 | gibi | computeRP <- device RP | |
| 10:39:22 | bauzas | I see | |
| 10:39:31 | gibi | bauzas: which thread? We just figured this problem out yesterday | |
| 10:39:35 | bauzas | sec | |
| 10:39:53 | gibi | it does not cuase any scheduling issue at the momemnt, this is why it was hidden so long | |
| 10:40:06 | bauzas | title is '[ops] Bandwidth problem on computes' | |
| 10:40:20 | gibi | bauzas: nope, that is some physical bandwidht issue | |
| 10:40:28 | bauzas | ah yeah, just looked | |
| 10:40:54 | gibi | so the above bug does not case any issue toady as the agent RP is basically unused | |
| 10:41:20 | gibi | but I have a request to start tracking OVS packet processing capacity as a resource | |
| 10:41:45 | gibi | and as today the ovs agent has a 1-1 relationship with the ovs agent, the ovs agent RP would be a good place for that new resource | |
| 10:42:01 | bauzas | okay I see the problem | |
| 10:42:05 | lucasagomes | hi, can someone please take a look at https://review.opendev.org/c/openstack/nova/+/776419, https://review.opendev.org/c/openstack/nova/+/776944 and https://review.opendev.org/c/openstack/nova/+/776934 ? | |
| 10:42:06 | gibi | I mean the ovs agent has 1-1 relationship with OVS itself | |
| 10:42:22 | gibi | bauzas: so first I would like to fix the bug | |
| 10:42:25 | lucasagomes | These are small patches that will prevent the nova gate from breaking when we change the default network backend in DevStack to OVN next cycle | |
| 10:42:29 | gibi | and then introduce new resources | |
| 10:43:04 | gibi | lucasagomes: I don't expect that these patches will be merged before the RC1 and the stable/wallaby is cut. | |
| 10:43:44 | lucasagomes | gibi, ah fair enough ok | |
| 10:44:02 | gibi | lucasagomes: but kick me after the cut and I will find somebody to review them | |
| 10:44:17 | gibi | bauzas: so one way to fix it is to re-parent the device RP | |
| 10:44:20 | lucasagomes | gibi, that sounds good, thanks much! | |
| 10:44:30 | gibi | bauzas: under the agent RP as it was before Ussuri | |
| 10:44:58 | gibi | bauzas: anyhow I will look into the reshape way now | |
| 10:45:58 | gibi | that is a good tip | |
| 10:46:50 | gibi | s/tip/suggestion/ | |
| 10:49:34 | gibi | bauzas: hm, so /reshape moves inventory and allocation between RPs but it does not move RPs between parents | |
| 10:50:44 | gibi | so I could create a new device RP under the agent and move the inventory and the allocations from the old device RP to the new device RP and then delete the old device RP | |
| 10:51:02 | gibi | e | |
| 10:51:02 | gibi | this could work from placement perspectiv | |
| 10:52:44 | gibi | neutron port does store the uuid of the RP the port is allocation from. So that would need to change too | |
| 10:53:48 | bauzas | gibi: you provide a new tree with the reshape API | |
| 10:54:42 | gibi | bauzas: really? I don't see RPs provided in the API | |
| 10:54:50 | gibi | I see inventories and allocations | |
| 11:03:17 | gibi | bauzas: the vgpu reshape also create the pgpu RPs _before_ the placement reshape call https://github.com/openstack/nova/blob/3de7fb7c327db348d04d15d4cd3c4f811a336126/nova/virt/libvirt/driver.py#L8414 | |
| 11:05:07 | bauzas | gibi: sure, but we provided back the new provider tree by this method https://github.com/openstack/nova/blob/3de7fb7c327db348d04d15d4cd3c4f811a336126/nova/virt/libvirt/driver.py#L8380 | |
| 11:05:33 | bauzas | gibi: so, we provide a new tree and then we look at the allocations | |
| 11:05:43 | bauzas | if there are some of them, then we call the reshape API | |
| 11:06:06 | bauzas | https://github.com/openstack/nova/blob/3de7fb7c327db348d04d15d4cd3c4f811a336126/nova/virt/libvirt/driver.py#L8102 | |
| 11:06:29 | bauzas | s/allocs/inventories | |
| 11:08:23 | bauzas | https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/reshape-provider-tree.html#proposed-change | |
| 11:08:58 | bauzas | gibi: every 60 secs (by default), the virt driver passes back the provider tree | |
| 11:09:09 | bauzas | so it can change it | |
| 11:09:57 | bauzas | but if the virt driver wants to modify the tree by punting new inventories and allocations from a RP to another one, then it needs to tell placement to move them | |
| 11:10:43 | bauzas | gibi: I can understand your concern, you don't want to move inventories or allocations, but just moving the parent | |
| 11:12:25 | bauzas | but then I guess that the resources would be modified | |
| 11:12:51 | bauzas | gibi: say, for example, with https://docs.openstack.org/placement/latest/user/provider-tree.html#filtering-by-tree | |
| 11:13:11 | bauzas | you would then get different candidates | |
| 11:13:44 | bauzas | that's why we need to be sure that we won't have different resources | |
| 11:14:50 | gibi | sure if I change the structure of the tree then the in_tree a_c queries will change | |
| 11:15:27 | gibi | regardless if I change the tree with reshape or just going in the db and change the parent_uuid of an RP | |
| 11:16:27 | gibi | as far as I see if I base the fix to reshape then the process is create new dev RP, rehaspe inv and alloc from old dev RP, change neutron ports to point to the new dev RP, deled old dev RP | |
| 11:16:46 | gibi | I think this would work | |
| 11:17:03 | gibi | but it creates a new RP instead of move the existing RP | |
| 11:17:15 | gibi | so it is more complicated that what I need | |
| 11:24:13 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add a real-time guide https://review.opendev.org/c/openstack/nova/+/781434 | |
| 11:24:13 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add a resource limits guide https://review.opendev.org/c/openstack/nova/+/781433 | |
| 11:24:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Remove duplicated PCI passthrough extra spec info https://review.opendev.org/c/openstack/nova/+/781436 | |
| 11:24:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Remove duplicate TPM extra spec info https://review.opendev.org/c/openstack/nova/+/781435 | |
| 11:24:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add CPU models guide https://review.opendev.org/c/openstack/nova/+/781438 | |
| 11:24:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add SEV guide https://review.opendev.org/c/openstack/nova/+/781437 | |
| 11:24:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add libvirt misc doc https://review.opendev.org/c/openstack/nova/+/781440 | |
| 11:24:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Change formatting of hypervisor config guides https://review.opendev.org/c/openstack/nova/+/781439 | |
| 11:24:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Fold in MDS security flaw doc https://review.opendev.org/c/openstack/nova/+/782411 | |
| 11:24:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Clarify host-model, host-passthrough differences https://review.opendev.org/c/openstack/nova/+/782410 | |
| 11:24:18 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Improve extra spec validator help texts https://review.opendev.org/c/openstack/nova/+/782412 | |
| 11:41:08 | gibi | bauzas: another issue. I need to keep the name of the device RP as nova depends on it | |
| 11:41:28 | gibi | and RP name needs to b unique | |
| 11:41:34 | bauzas | yup | |
| 11:41:35 | gibi | in placement | |
| 11:41:46 | bauzas | in general, we use (parent)_(name) | |
| 11:41:48 | gibi | so the create a new device RP way does not work | |
| 11:42:14 | sean-k-mooney | is this in relation to something speciric? | |
| 11:42:15 | bauzas | I wonder | |
| 11:42:58 | bauzas | sean-k-mooney: see above | |
| 11:43:31 | sean-k-mooney | im trying to read back but what i have got sofar is moving bandwith inventories via a reshape? | |
| 11:43:36 | bauzas | gibi: maybe you should just create a new reshape method | |
| 11:43:41 | bauzas | gibi: and then modify the names | |
| 11:43:47 | bauzas | in there | |
| 11:44:10 | bauzas | like : does the RP have the wrong parent UUID ? yes => ReshapeNeeded | |
| 11:44:11 | gibi | bauzas: ohh the RP name is settable, so we can use a temporary name | |
| 11:44:56 | bauzas | then, ReshapeNeeded will be provided so you could say 'ah, ReshapeNeeded, so create a new RP, modify the existing name, and then use this name for my new RP' | |
| 11:45:07 | sean-k-mooney | gibi: its setable only one i think | |
| 11:45:22 | sean-k-mooney | gibi: eg i dont think you can modify it after you create teh RP | |
| 11:45:24 | gibi | sean-k-mooney: parent (null-> nonnull) and name is settable | |
| 11:45:37 | gibi | as far as I see | |
| 11:45:45 | bauzas | gibi: this way, you wouldn't change the RPs after the compute service is restarted | |
| 11:45:58 | sean-k-mooney | there used to be a limiation around this but im tryign to recall what it was | |
| 11:46:07 | gibi | bauzas: these RPs are not handled by nova, so this reshape will happen in neutron :) | |
| 11:46:15 | bauzas | ah, shit, right | |
| 11:46:16 | gibi | bauzas: but besides that yes | |
| 11:47:28 | sean-k-mooney | allowing RPs to be reparented would be a much cleaner approch | |
| 11:47:36 | sean-k-mooney | i have asked for this in the past | |
| 11:47:49 | sean-k-mooney | to avoid the need to reshap | |
| 11:48:06 | bauzas | no | |
| 11:48:20 | bauzas | because you need to modify them just after restarting the service | |