| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-12 | |||
| 14:27:28 | sean-k-mooney | is the contol plan. e.g. your contolers all RDO or is it also mixed | |
| 14:27:42 | NewBruce | control plane is upgrade to Rocky/OSA | |
| 14:28:21 | NewBruce | we have multiple sites, with the same config which dont exhibit the problem | |
| 14:28:27 | sean-k-mooney | ok so you have Rocky/OSA contol plane + OSA computes + some rocky/RDO computes | |
| 14:28:36 | NewBruce | correct | |
| 14:28:54 | NewBruce | looking in the setup today we have some RDO net nodes as well | |
| 14:29:20 | sean-k-mooney | and on the neutron side is it also a mix or all OSA for you neturon controler services | |
| 14:29:30 | NewBruce | so they could do with being upgraded.... but in the debugging ive done today, it seems to bail out in the call to _setup_migration_port_profile | |
| 14:29:46 | dansmith | mriedem_snow: okay I was thinking we should maybe try real hard to get that merged today | |
| 14:29:59 | NewBruce | in a successful migration, the host_id passed to _setup_migration_port_profile is the target/dest compute; whereas in a failure the host_id passed to _setup_migration_port_profile is the source | |
| 14:30:02 | sean-k-mooney | how the multiple portbinging code is ment to work is that the nova condocutor is first ment to check the neutron api to determin if neutron support it | |
| 14:31:10 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: libvirt: Add workaround to cleanup instance dir when using rbd https://review.openstack.org/628726 | |
| 14:31:10 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Fail to live migration if instance has a NUMA topology https://review.openstack.org/629597 | |
| 14:31:47 | sean-k-mooney | the condoctor then check the version of both the souce and dest compute service and if they both support the multiple port binding workflow it is enabled | |
| 14:31:58 | NewBruce | will be used as described below. | |
| 14:31:58 | NewBruce | If both compute nodes are queens or newer. In this case the new workflow | |
| 14:32:08 | lyarwood | bauzas / stephenfin / mriedem_snow ^ reworked these into a series now on top of https://review.openstack.org/#/c/620111/, hopefully that's cleaner for everyone. | |
| 14:32:47 | NewBruce | so that check is based on the nova versions, yeah? which is fair enough, as when i monitor the ml2_port_bindings. i see the duplicate port entries in both successful and fail cases; | |
| 14:33:18 | NewBruce | in the services table both source and dest are v35 | |
| 14:33:35 | NewBruce | for nova-compute | |
| 14:33:46 | sean-k-mooney | NewBruce: thew new workflow should only take effect if the source and dest compute nodes are rocky | |
| 14:33:51 | NewBruce | which they are | |
| 14:34:30 | NewBruce | sorry sean-k-mooney - gotta run for 10-15 will be back online | |
| 14:34:38 | sean-k-mooney | sure no worries | |
| 14:35:54 | zigo | Hi there! | |
| 14:36:09 | zigo | I'm getting a weirdo bug inside Horizon which is related to Novaclient. | |
| 14:36:18 | openstackgerrit | Vladyslav Drok proposed openstack/nova master: libvirt: Do not reraise DiskNotFound exceptions during resize https://review.openstack.org/571410 | |
| 14:36:25 | zigo | When listing instances, Horizon first try to get the nova microversion. | |
| 14:36:36 | kaisers | stephenfin: Hi! would you care to revisit https://review.openstack.org/#/c/522245/ ? Old driver bugfix change that has been sitting for some time and is not overly complex. | |
| 14:36:57 | zigo | So it tries to get https://api/compute/v2.1 | |
| 14:37:17 | zigo | But nova sends as reply a redirect to https://api/compute/v2.1/ (note the lasting slash ...) | |
| 14:37:18 | stephenfin | kaisers: Sure. I'd totally forgotten about that :) I'll take a look now | |
| 14:37:31 | kaisers | stephenfin: greate, thnx | |
| 14:37:34 | zigo | It's probably a miss-configuration of haproxy, apache, or something else... | |
| 14:37:52 | zigo | Though shouldn't nova-api reply with the microversion even without the lasting / ? | |
| 14:42:12 | zigo | bauzas: ^ | |
| 14:42:21 | bauzas | efried: gibi: I'm torn, given we only pass allocations down to the virt driver, how can the driver know whether the related RP is a root RP or a child ? | |
| 14:42:58 | bauzas | it should then look at the ProviderTree, right? | |
| 14:43:22 | efried | bauzas: You can't, until we implement https://review.openstack.org/#/c/597601/ | |
| 14:43:38 | efried | does spawn have access to the provider tree? | |
| 14:43:39 | gibi | bauzas: I think from the allocation alone you cannot figure out | |
| 14:44:07 | bauzas | because then we have a problem with the reshape | |
| 14:44:31 | efried | The reshape comes from update_provider_tree, which *does* have access to the provider tree. | |
| 14:44:34 | gibi | bauzas: do you really need that information? can you simply look at the allocation, select the RP that has VGPU resource allocated from | |
| 14:44:53 | bauzas | gibi: how can I know this question ? | |
| 14:46:08 | bauzas | gibi: efried: I mean, for the moment, we use a mdev when creating an instance by https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6273 | |
| 14:46:09 | gibi | bauzas: you want to know which PGPU you are allocated the mdev from. You have the allocations dict. That dict has RPs with resources in it. The RP you need is the one that has VGPU resource allocated from | |
| 14:46:45 | bauzas | gibi: efried so I'm changing the above to check with RP was allocated | |
| 14:47:09 | bauzas | and then only either creating a new mdev or use an existing mdev from the pci device related to the RP | |
| 14:47:18 | stephenfin | kaisers: Done. Feel free to ping me whenever I do forget stuff. I'll simply ignore you if I'm too busy :) | |
| 14:47:42 | bauzas | gibi: I'm getting allocations, right? | |
| 14:47:58 | stephenfin | lyarwood: Thanks for tackling that (y) It was on my TODO list | |
| 14:48:07 | bauzas | gibi: so I guess I'm getting https://developer.openstack.org/api-ref/placement/?expanded=list-allocations-detail#response-example-1-28 | |
| 14:48:29 | gibi | yes | |
| 14:48:50 | bauzas | so how can I get the RP ? | |
| 14:49:01 | bauzas | I just have the RP UUID | |
| 14:49:02 | efried | the rp is in there, but only its uuid, not its name. | |
| 14:49:10 | gibi | ok it is just the UUID | |
| 14:49:22 | gibi | what else you would need? then name ? | |
| 14:49:25 | efried | I thought we made a convention for setting the RP UUID based on some aspect of the VGPU | |
| 14:49:31 | gibi | what connects an RP and an mdev? | |
| 14:49:33 | efried | gibi: he needs to be able to associate the RP with the PGPU | |
| 14:49:46 | efried | which he can do from the name for sure, because we used some kind of naming convention. | |
| 14:49:53 | gibi | efried: I see | |
| 14:50:01 | efried | but with just the uuid, unless we named the rp somehow after the uuid of the pgpu, we're fooked. | |
| 14:50:06 | gibi | efried: that is a problem as the provider summaries are not send down | |
| 14:50:16 | efried | Is the name in the provider summary? | |
| 14:50:25 | gibi | hm | |
| 14:50:27 | gibi | looking | |
| 14:50:57 | efried | doesn't look like it. | |
| 14:51:38 | TheJulia | jroll: That looks awesome. Thanks! | |
| 14:52:34 | efried | bauzas: So unless we did in fact make some special way of assigning the RP UUID based on the identity of the pgpu, you're going to need to store that mapping locally somehow, which is ew. | |
| 14:52:50 | gibi | efried: you are right | |
| 14:53:46 | bauzas | efried: yeah it's a problem | |
| 14:53:54 | efried | you can store an instance variable in the virt driver from update_provider_tree, with a big TODO on it to figure out a better way. | |
| 14:54:00 | bauzas | we only have a convention for the name, not the uuid | |
| 14:54:02 | efried | And I'm wrong, it has nothing to do with gibi's spec | |
| 14:54:03 | jroll | TheJulia: cool, I'll un-wip. thanks | |
| 14:54:12 | efried | jroll: I'll bump my vote | |
| 14:54:29 | jroll | efried: thanks | |
| 14:54:57 | bauzas | efried: okay, I guess we have a big problem honestly | |
| 14:55:06 | bauzas | because I don't want to do this like that | |
| 14:55:23 | bauzas | if we want to select a node, then we need to know which one | |
| 14:55:25 | efried | um, maybe I'm wrong about being wrong. But it gives us something extra to think about in gibi's spec, which is that we have to have the information in the allocation itself, OR figure out some other way to get that info into spawn. | |
| 14:55:42 | gibi | efried: yeah, it is not the spec you linked, but I had similar issue to map pci device to RP, there it is done by the RP name, and there is an extra call in the compute/manager to get the RP name | |
| 14:55:43 | efried | but I think the libvirt problem is an extra one. | |
| 14:56:07 | efried | Perhaps we should consider amending the allocation structure to include the rp name. | |
| 14:56:16 | bauzas | so, again, I'll say we're blocked | |
| 14:56:40 | bauzas | unless the virt driver calls placement to know the RP tree | |
| 14:56:41 | efried | jaypipes: backscroll to :42:21 | |
| 14:56:53 | cdent | i'd be pretty opposed to include the name of the rp in the allocation unless we really can't come up with something | |
| 14:56:54 | efried | bauzas: Yes, that would be another "solution" that would be ew. | |
| 14:56:59 | cdent | why _not_ pass the summaries? | |
| 14:57:16 | cdent | or why not let the virt driver talk to placement? | |
| 14:57:18 | NewBruce | sean-k-mooney : back | |
| 14:57:26 | cdent | or any of several other options | |
| 14:57:34 | bauzas | I'll upload my change anyway | |
| 14:57:35 | efried | cdent: not pass the summaries because a) way too much information, unless we pare it down to just the summaries in the allocation, and b) the name isn't in the summaries anyway :P | |
| 14:58:07 | cdent | b) right, but putting it in the summaries would make some form of sense: it's a provider _summary_ after all | |
| 14:58:09 | cdent | however | |