| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-08-31 | |||
| 13:47:16 | dansmith | I just wanted to find something that looks like a real failure to contrast it to a passing run, and also to make sure it's actually reporting failures | |
| 13:47:46 | gibi | yeah, me neither | |
| 13:48:12 | gibi | I'm trying to figure out if Yingji Sun or jhui@vmware.com is on IRC or not | |
| 13:48:44 | gibi | so we can ask them | |
| 13:53:48 | sean-k-mooney | dansmith: they might be hitting the same keystone error that is breaking the first party ci | |
| 13:54:09 | dansmith | sean-k-mooney: and reporting empty logs? seems weird, but okay | |
| 13:55:09 | sean-k-mooney | well no that is likely another issue | |
| 13:55:23 | sean-k-mooney | i just noticed they started failing when the upstream jobs also started failng | |
| 14:00:05 | sean-k-mooney | hum the hyperv ci also seams to be broken | |
| 14:00:20 | sean-k-mooney | its been red for at least the last 7 days | |
| 14:33:24 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Add a routed networks scheduler pre-filter https://review.opendev.org/749068 | |
| 14:33:37 | bauzas | gibi: sean-k-mooney: there it is ^ | |
| 14:36:02 | bauzas | gibi: sean-k-mooney: I now use a pre-filter that looks at the existing physnets that were provided by https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L2082 | |
| 14:39:16 | bauzas | I mean the network metadata that isn't persisted in the spec object but reused by every move operation | |
| 15:08:49 | gibi | bauzas: then I think you have to resolve TODO in the codepath that gather the physnets https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/network/neutron.py#L2020 | |
| 15:09:06 | gibi | as today it only get one physnet per network, even if there are multiple | |
| 15:09:32 | bauzas | holy shit. | |
| 15:14:13 | bauzas | gibi: in a meeting | |
| 15:25:31 | bauzas | gibi: actually, looking at vladik's comment, I'm not expert, but do we really support multiple networks per vlans? | |
| 15:25:49 | bauzas | multiple physnets* | |
| 15:27:34 | gibi | as far as I know in neutron nothing prevents you to create two vlan segments with different physnets in the same network | |
| 15:47:28 | bauzas | gibi: okay, then I don't feel enough expert to fix this TODO | |
| 15:48:05 | bauzas | gibi: the other way would be to find a way to pass requested networks down in the request spedc | |
| 15:48:07 | bauzas | spec* | |
| 15:48:19 | bauzas | which is something we don't do | |
| 15:49:39 | gibi | the way the pervious patches did it was to pass the aggregates via request_spec.request_level_params.member_of | |
| 15:50:04 | gibi | and do the network segment -> host aggregate translation in an upper layer (conductor) | |
| 15:50:20 | bauzas | gibi: I know | |
| 15:50:33 | bauzas | but a pre-filter is better, right? | |
| 15:50:55 | bauzas | and we don't need to pass the aggregates by a new field | |
| 15:51:08 | bauzas | the destination object already contains what we need | |
| 15:51:26 | gibi | this is still a pre-filter as the aggregate filterin happens in the a_c query | |
| 15:51:53 | gibi | also I don't think request_spec.request_level_params.member_of is a new field at all | |
| 15:52:03 | bauzas | gibi: matt added it | |
| 15:52:11 | gibi | the member_of parts? | |
| 15:52:16 | gibi | that could be | |
| 15:52:30 | gibi | but the request_level_params object should be on master I think | |
| 15:52:30 | bauzas | gibi: we already have pre-filters that add specific aggregates to verify | |
| 15:52:45 | bauzas | so, the problem is not how to ask placement | |
| 15:52:59 | bauzas | but rather, how to pass the networks to the pre-filter | |
| 15:53:36 | gibi | you have to pass in new information so you have to extend an passed object or pass a new object | |
| 15:53:37 | bauzas | gibi: see, https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/scheduler/request_filter.py#L121 | |
| 15:54:00 | bauzas | gibi: this way, from the pre-filter, you can pass the list of aggregates to require | |
| 15:54:11 | bauzas | so this is a solved problem | |
| 15:54:34 | bauzas | but since we only have the request spec object that is passed down to the pre-filter, we need to get the networks from it | |
| 15:54:43 | bauzas | my proposal was to use the network metadata info | |
| 15:55:02 | bauzas | but we only provide a list of physnets | |
| 15:55:49 | gibi | so you can extend the NetworkMetadata if you wish, or you can extend the RequestLevelParams of the RequestSpec | |
| 15:56:17 | bauzas | or I could get the info like this https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/objects/request_spec.py#L547 | |
| 15:56:35 | bauzas | (from the vif) | |
| 15:56:48 | gibi | do we have the instance info cache populated before reaching the compute? | |
| 15:57:27 | bauzas | no, that's only for a move op | |
| 15:57:34 | bauzas | AFAIK | |
| 15:57:43 | bauzas | anyway, I need to think about the problem more | |
| 15:57:45 | gibi | yeah, for the move you can reach into the info cache | |
| 15:58:36 | bauzas | gibi: I need to balance all the options | |
| 15:58:56 | bauzas | I don't want a pre-filter that would behave different between operations | |
| 15:59:16 | bauzas | so I literrally need to find a way to provide the required information into the request spec object directly | |
| 15:59:39 | bauzas | which would prevent us backports, which is a bit sad unfortunately for my company :/ | |
| 16:00:36 | gibi | ohh, so the motivation is backportability | |
| 16:03:23 | bauzas | gibi: not really, I'd say the easier be the better | |
| 16:04:00 | bauzas | I was thinking the physnets be a clean way to do this, but given this TODO, I'm stuck | |
| 16:04:21 | bauzas | so, I'll just provide the networks the best way, and good bye backportability | |
| 16:04:29 | gibi | fixing that TODO would need also an object change I guess, so you would not gain much | |
| 16:05:28 | sean-k-mooney | hi so i was not following due to downstream call | |
| 16:05:57 | sean-k-mooney | bauzas: we cant trust the phsynets in the networking info cache if the network has multiple phsynets | |
| 16:06:09 | sean-k-mooney | unless we fix how we currently do the phsynet lookup | |
| 16:06:21 | sean-k-mooney | so that that is segment aware | |
| 16:06:39 | bauzas | sean-k-mooney: that was gibi's point | |
| 16:06:57 | bauzas | (17:08:49) gibi: bauzas: then I think you have to resolve TODO in the codepath that gather the physnets https://github.com/openstack/nova/blob/b5d48043466b53fbdfe7b93c2e4efd449904e593/nova/network/neutron.py#L2020 | |
| 16:07:02 | sean-k-mooney | bauzas: the physnet in the nova vif object are only correct when you have 1 phsynet and transitivly 1 segment on the network | |
| 16:07:25 | bauzas | sean-k-mooney: I was just looking up the network metadata | |
| 16:07:42 | sean-k-mooney | bauzas: right so if we resolve that todo we need also consider the inpact on sriov | |
| 16:08:11 | sean-k-mooney | by the way this is the issue i pointed out before. actully supporting the multi_provider_physnet extension | |
| 16:08:17 | bauzas | sean-k-mooney: https://review.opendev.org/#/c/749068/1/nova/scheduler/request_filter.py@286 | |
| 16:08:37 | bauzas | sean-k-mooney: but I used the physnets because that was the quickiest path | |
| 16:08:44 | bauzas | since we don't pass the list of networks | |
| 16:08:50 | bauzas | in the request spec | |
| 16:08:54 | sean-k-mooney | ya so that set of phsynets is not correct for routed networks if you have more then 1 segment | |
| 16:09:01 | bauzas | sure | |
| 16:09:02 | bauzas | so | |
| 16:09:16 | bauzas | what I want is just a way to get a list of networks et voila | |
| 16:09:44 | bauzas | which means I have to augment the RequestSpec object or one of its nested children | |
| 16:09:50 | sean-k-mooney | the pshnets are in the segments not in the networks so you need to list the network then query neutron for the segment to get the phsynet | |
| 16:10:09 | bauzas | again, I don't care of the physnets | |
| 16:10:16 | bauzas | I want to get the segments | |
| 16:10:33 | sean-k-mooney | yes but neutron uses physnets to map host to segments | |
| 16:10:44 | sean-k-mooney | but yes you want the segment | |
| 16:11:02 | bauzas | sean-k-mooney: we have a neutron API for getting the segments that are related to either a network or a physnet | |
| 16:11:29 | bauzas | the original proposal from matt was to get the networks and ask neutron to give the segments | |
| 16:11:37 | sean-k-mooney | yes | |
| 16:11:38 | bauzas | this was easy since he wrote that in the conductor | |
| 16:11:51 | bauzas | and then we directly have the requested networks (or ports) | |
| 16:12:02 | bauzas | but here, we want to make it more generic in a pre-filter | |
| 16:12:19 | bauzas | and in my case, I only have the requestspec object this is passed as argument | |
| 16:12:29 | bauzas | good bye networks and ports | |
| 16:12:49 | bauzas | sean-k-mooney: see my problem and why I went using the physnets ? | |
| 16:13:12 | sean-k-mooney | bauzas: yep i know but just to be clear this will only work for existing vms | |
| 16:13:21 | sean-k-mooney | you cant use phsynets for booting new vms | |
| 16:13:48 | bauzas | sean-k-mooney: well, I can see us populating the network metadata object when creating the instance | |