Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-03-23
10:28:45 gibi I can see that moving an RP to one tree to the other could break things in the allocations
10:34:54 bauzas gibi: because it would need some reshape, maybe ?
10:35:05 bauzas :)
10:35:18 gibi bauzas: yeah if there are allocations then moving an RP could invalidate them
10:35:24 gibi in the generic case
10:35:27 bauzas gibi: afair, we only accept to modify the RPs by some reshape method
10:35:39 bauzas like we did for vGPUs and others
10:36:01 bauzas if we modify the parent, then the resources would also be modified
10:36:04 gibi does reshape allows reparenting?
10:36:10 bauzas gibi: yup
10:36:22 bauzas you provide new inventories
10:36:25 gibi hm, then reshape it is
10:36:43 bauzas gibi: context ?
10:37:43 gibi bauzas: we have a bug in neutron that creates a slightly wrong RP tree for qos
10:38:16 bauzas gibi: ahah
10:38:18 gibi the expected tree would be computeRP <- neutron agent RP <- deviceRP
10:38:35 gibi but after a bugfix in Ussuri it was changed to
10:38:38 bauzas gibi: related to the ML thread I saw ?
10:39:03 gibi computeRP <- neutron agent RP
10:39:06 gibi computeRP <- device RP
10:39:22 bauzas I see
10:39:31 gibi bauzas: which thread? We just figured this problem out yesterday
10:39:35 bauzas sec
10:39:53 gibi it does not cuase any scheduling issue at the momemnt, this is why it was hidden so long
10:40:06 bauzas title is '[ops] Bandwidth problem on computes'
10:40:20 gibi bauzas: nope, that is some physical bandwidht issue
10:40:28 bauzas ah yeah, just looked
10:40:54 gibi so the above bug does not case any issue toady as the agent RP is basically unused
10:41:20 gibi but I have a request to start tracking OVS packet processing capacity as a resource
10:41:45 gibi and as today the ovs agent has a 1-1 relationship with the ovs agent, the ovs agent RP would be a good place for that new resource
10:42:01 bauzas okay I see the problem
10:42:05 lucasagomes hi, can someone please take a look at https://review.opendev.org/c/openstack/nova/+/776419, https://review.opendev.org/c/openstack/nova/+/776944 and https://review.opendev.org/c/openstack/nova/+/776934 ?
10:42:06 gibi I mean the ovs agent has 1-1 relationship with OVS itself
10:42:22 gibi bauzas: so first I would like to fix the bug
10:42:25 lucasagomes These are small patches that will prevent the nova gate from breaking when we change the default network backend in DevStack to OVN next cycle
10:42:29 gibi and then introduce new resources
10:43:04 gibi lucasagomes: I don't expect that these patches will be merged before the RC1 and the stable/wallaby is cut.
10:43:44 lucasagomes gibi, ah fair enough ok
10:44:02 gibi lucasagomes: but kick me after the cut and I will find somebody to review them
10:44:17 gibi bauzas: so one way to fix it is to re-parent the device RP
10:44:20 lucasagomes gibi, that sounds good, thanks much!
10:44:30 gibi bauzas: under the agent RP as it was before Ussuri
10:44:58 gibi bauzas: anyhow I will look into the reshape way now
10:45:58 gibi that is a good tip
10:46:50 gibi s/tip/suggestion/
10:49:34 gibi bauzas: hm, so /reshape moves inventory and allocation between RPs but it does not move RPs between parents
10:50:44 gibi so I could create a new device RP under the agent and move the inventory and the allocations from the old device RP to the new device RP and then delete the old device RP
10:51:02 gibi this could work from placement perspectiv
10:51:02 gibi e
10:52:44 gibi neutron port does store the uuid of the RP the port is allocation from. So that would need to change too
10:53:48 bauzas gibi: you provide a new tree with the reshape API
10:54:42 gibi bauzas: really? I don't see RPs provided in the API
10:54:50 gibi I see inventories and allocations
11:03:17 gibi bauzas: the vgpu reshape also create the pgpu RPs _before_ the placement reshape call https://github.com/openstack/nova/blob/3de7fb7c327db348d04d15d4cd3c4f811a336126/nova/virt/libvirt/driver.py#L8414
11:05:07 bauzas gibi: sure, but we provided back the new provider tree by this method https://github.com/openstack/nova/blob/3de7fb7c327db348d04d15d4cd3c4f811a336126/nova/virt/libvirt/driver.py#L8380
11:05:33 bauzas gibi: so, we provide a new tree and then we look at the allocations
11:05:43 bauzas if there are some of them, then we call the reshape API
11:06:06 bauzas https://github.com/openstack/nova/blob/3de7fb7c327db348d04d15d4cd3c4f811a336126/nova/virt/libvirt/driver.py#L8102
11:06:29 bauzas s/allocs/inventories
11:08:23 bauzas https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/reshape-provider-tree.html#proposed-change
11:08:58 bauzas gibi: every 60 secs (by default), the virt driver passes back the provider tree
11:09:09 bauzas so it can change it
11:09:57 bauzas but if the virt driver wants to modify the tree by punting new inventories and allocations from a RP to another one, then it needs to tell placement to move them
11:10:43 bauzas gibi: I can understand your concern, you don't want to move inventories or allocations, but just moving the parent
11:12:25 bauzas but then I guess that the resources would be modified
11:12:51 bauzas gibi: say, for example, with https://docs.openstack.org/placement/latest/user/provider-tree.html#filtering-by-tree
11:13:11 bauzas you would then get different candidates
11:13:44 bauzas that's why we need to be sure that we won't have different resources
11:14:50 gibi sure if I change the structure of the tree then the in_tree a_c queries will change
11:15:27 gibi regardless if I change the tree with reshape or just going in the db and change the parent_uuid of an RP
11:16:27 gibi as far as I see if I base the fix to reshape then the process is create new dev RP, rehaspe inv and alloc from old dev RP, change neutron ports to point to the new dev RP, deled old dev RP
11:16:46 gibi I think this would work
11:17:03 gibi but it creates a new RP instead of move the existing RP
11:17:15 gibi so it is more complicated that what I need
11:24:13 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add a resource limits guide https://review.opendev.org/c/openstack/nova/+/781433
11:24:13 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add a real-time guide https://review.opendev.org/c/openstack/nova/+/781434
11:24:14 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Remove duplicate TPM extra spec info https://review.opendev.org/c/openstack/nova/+/781435
11:24:14 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Remove duplicated PCI passthrough extra spec info https://review.opendev.org/c/openstack/nova/+/781436
11:24:15 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add SEV guide https://review.opendev.org/c/openstack/nova/+/781437
11:24:15 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add CPU models guide https://review.opendev.org/c/openstack/nova/+/781438
11:24:16 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Change formatting of hypervisor config guides https://review.opendev.org/c/openstack/nova/+/781439
11:24:16 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add libvirt misc doc https://review.opendev.org/c/openstack/nova/+/781440
11:24:17 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Clarify host-model, host-passthrough differences https://review.opendev.org/c/openstack/nova/+/782410
11:24:17 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Fold in MDS security flaw doc https://review.opendev.org/c/openstack/nova/+/782411
11:24:18 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Improve extra spec validator help texts https://review.opendev.org/c/openstack/nova/+/782412
11:41:08 gibi bauzas: another issue. I need to keep the name of the device RP as nova depends on it
11:41:28 gibi and RP name needs to b unique
11:41:34 bauzas yup
11:41:35 gibi in placement
11:41:46 bauzas in general, we use (parent)_(name)
11:41:48 gibi so the create a new device RP way does not work
11:42:14 sean-k-mooney is this in relation to something speciric?
11:42:15 bauzas I wonder
11:42:58 bauzas sean-k-mooney: see above
11:43:31 sean-k-mooney im trying to read back but what i have got sofar is moving bandwith inventories via a reshape?
11:43:36 bauzas gibi: maybe you should just create a new reshape method
11:43:41 bauzas gibi: and then modify the names
11:43:47 bauzas in there
11:44:10 bauzas like : does the RP have the wrong parent UUID ? yes => ReshapeNeeded
11:44:11 gibi bauzas: ohh the RP name is settable, so we can use a temporary name

Earlier   Later