| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-28 | |||
| 12:05:00 | sean-k-mooney | how it works is the ovs vswitchd process caluates openflow rules as normal and then use the tcflower protocal to install those flows into the hardware dataplane if it supprot them | |
| 12:05:01 | sean-k-mooney | how it works is the ovs vswitchd process caluates openflow rules as normal and then use the tcflower protocal to install those flows into the hardware dataplane if it supprot them | |
| 12:05:24 | gibi | sean-k-mooney: thanks that helps visualizing the situation for me | |
| 12:05:24 | gibi | sean-k-mooney: thanks that helps visualizing the situation for me | |
| 12:06:19 | sean-k-mooney | basically you can thinkg of the programming model like a cache if the hardware can cache the rule in its hardware swtich it will process them in hardware. the ovs proces just caculates them as normal and they are then copied to the hardware | |
| 12:06:19 | sean-k-mooney | basically you can thinkg of the programming model like a cache if the hardware can cache the rule in its hardware swtich it will process them in hardware. the ovs proces just caculates them as normal and they are then copied to the hardware | |
| 12:06:50 | sean-k-mooney | what that means in partcie is the first packet(s) of each connection is processed in software and then hardware takes over | |
| 12:06:50 | sean-k-mooney | what that means in partcie is the first packet(s) of each connection is processed in software and then hardware takes over | |
| 12:08:07 | gibi | I see | |
| 12:09:18 | gibi | and in this hardwer the incoming and outgoing packet processing is implemented independently so we can run out of one direction while the other direction still has capacity | |
| 12:09:18 | gibi | and in this hardwer the incoming and outgoing packet processing is implemented independently so we can run out of one direction while the other direction still has capacity | |
| 12:09:40 | gibi | and therefore we need separated resource tracking | |
| 12:09:40 | gibi | and therefore we need separated resource tracking | |
| 12:10:11 | sean-k-mooney | i think that depens on the nic but in principal yes the internal capasity of the switch is higher then the phsyical connectors | |
| 12:10:11 | sean-k-mooney | i think that depens on the nic but in principal yes the internal capasity of the switch is higher then the phsyical connectors | |
| 12:10:45 | sean-k-mooney | so it can run out of capasity to transmit to the datacenter network before its ablity to swtich between vms on the same host | |
| 12:10:45 | sean-k-mooney | so it can run out of capasity to transmit to the datacenter network before its ablity to swtich between vms on the same host | |
| 12:11:17 | sean-k-mooney | although intel tend and mellonox until recently had a habbit of oversubscibing ther port bandwith | |
| 12:11:17 | sean-k-mooney | although intel tend and mellonox until recently had a habbit of oversubscibing ther port bandwith | |
| 12:11:59 | sean-k-mooney | i.e. intel 2*40G fortviles only had 56Gb/s of pcie bandwith | |
| 12:12:00 | sean-k-mooney | i.e. intel 2*40G fortviles only had 56Gb/s of pcie bandwith | |
| 12:12:53 | sean-k-mooney | which is fine for the 2*25G card but ya pcie bandwith and phycial port bandwith can often be less the the chip bandwith in the data center space | |
| 12:12:53 | sean-k-mooney | which is fine for the 2*25G card but ya pcie bandwith and phycial port bandwith can often be less the the chip bandwith in the data center space | |
| 12:13:40 | sean-k-mooney | the assumtion often is that you are using 2 port cards for HA not more bandwitr/pps | |
| 12:13:40 | sean-k-mooney | the assumtion often is that you are using 2 port cards for HA not more bandwitr/pps | |
| 12:14:16 | gibi | when you say "run out of capasity to transmit to the datacenter" do you mean it runs out of physical nework bandwidth? | |
| 12:14:16 | gibi | when you say "run out of capasity to transmit to the datacenter" do you mean it runs out of physical nework bandwidth? | |
| 12:14:29 | sean-k-mooney | yep | |
| 12:14:43 | gibi | so while it can still swtich packets, it cannot transmit them | |
| 12:14:43 | gibi | so while it can still swtich packets, it cannot transmit them | |
| 12:15:09 | gibi | but then this is showing just that the packet processing capacity is independent from the bandwidth | |
| 12:15:09 | gibi | but then this is showing just that the packet processing capacity is independent from the bandwidth | |
| 12:15:43 | gibi | it does not show that switching incoming packets are independend from switching outgoing packets | |
| 12:15:43 | gibi | it does not show that switching incoming packets are independend from switching outgoing packets | |
| 12:15:47 | sean-k-mooney | ya the 2*25G intel fortvales actully had 2 40G chips one per uplink port but had 25G phys | |
| 12:15:47 | sean-k-mooney | ya the 2*25G intel fortvales actully had 2 40G chips one per uplink port but had 25G phys | |
| 12:17:27 | sean-k-mooney | gibi: you are right it does not show they are independeint on the recive side if you have an intel cpu with vt-c | |
| 12:17:27 | sean-k-mooney | gibi: you are right it does not show they are independeint on the recive side if you have an intel cpu with vt-c | |
| 12:17:56 | sean-k-mooney | that allows supported nic to place recived packet directly into the cpus l3 cache | |
| 12:17:56 | sean-k-mooney | that allows supported nic to place recived packet directly into the cpus l3 cache | |
| 12:18:40 | sean-k-mooney | on transmit i know the cpu can do a dma transfre to the nic queue or indead to a zerocopy transmit in some cases | |
| 12:18:40 | sean-k-mooney | on transmit i know the cpu can do a dma transfre to the nic queue or indead to a zerocopy transmit in some cases | |
| 12:20:36 | gibi | OK, I rest my case. I will do allow the admin to configure either two direction aware pps resource inventory (in case of offloaded OVS) or a single direction less resource inventory (for the normal OVS) | |
| 12:20:36 | gibi | OK, I rest my case. I will do allow the admin to configure either two direction aware pps resource inventory (in case of offloaded OVS) or a single direction less resource inventory (for the normal OVS) | |
| 12:20:53 | sean-k-mooney | well no | |
| 12:20:53 | sean-k-mooney | well no | |
| 12:21:16 | sean-k-mooney | if you think that scoping this down to just normal ovs for now is the best way forward that would be ok | |
| 12:21:16 | sean-k-mooney | if you think that scoping this down to just normal ovs for now is the best way forward that would be ok | |
| 12:21:58 | sean-k-mooney | it would proably be nice to support both modesl as you said but we could start with the one you want | |
| 12:21:58 | sean-k-mooney | it would proably be nice to support both modesl as you said but we could start with the one you want | |
| 12:22:18 | gibi | I can do both, this is not the hard part (I guess :D)_ | |
| 12:22:18 | gibi | I can do both, this is not the hard part (I guess :D)_ | |
| 12:22:21 | sean-k-mooney | i just would not like to see logic to traslate driction based request into driectionless | |
| 12:22:21 | sean-k-mooney | i just would not like to see logic to traslate driction based request into driectionless | |
| 12:23:00 | sean-k-mooney | the translation is the bit i found slightly unclean about the proposal | |
| 12:23:00 | sean-k-mooney | the translation is the bit i found slightly unclean about the proposal | |
| 12:24:06 | gibi | the QoS rule will be direction aware. So that is good. | |
| 12:24:06 | gibi | Then in case of normal OVS should we still allow configuring direction aware resource inventory? | |
| 12:24:06 | gibi | the QoS rule will be direction aware. So that is good. | |
| 12:24:06 | gibi | Then in case of normal OVS should we still allow configuring direction aware resource inventory? | |
| 12:24:23 | gibi | that seems wrong to me as both direction are handled by the same set of CPUs runnig the OVS software | |
| 12:24:23 | gibi | that seems wrong to me as both direction are handled by the same set of CPUs runnig the OVS software | |
| 12:24:35 | sean-k-mooney | well i kind of thik we shoudl have 3 polices as i said above | |
| 12:24:35 | sean-k-mooney | well i kind of thik we shoudl have 3 polices as i said above | |
| 12:24:54 | gibi | ohh | |
| 12:24:54 | gibi | ohh | |
| 12:24:55 | sean-k-mooney | 2 that are direction aware and 1 that is drectionless | |
| 12:24:55 | sean-k-mooney | 2 that are direction aware and 1 that is drectionless | |
| 12:25:19 | sean-k-mooney | that map directly to the PPS_TOTAL, PPS_INGRESS and PPS_EGRESS resouce classes | |
| 12:25:19 | sean-k-mooney | that map directly to the PPS_TOTAL, PPS_INGRESS and PPS_EGRESS resouce classes | |
| 12:25:20 | gibi | so for vnic_type=normal the user should only use a directionless QoS policy | |
| 12:25:20 | gibi | so for vnic_type=normal the user should only use a directionless QoS policy | |
| 12:25:24 | sean-k-mooney | yep | |
| 12:25:49 | gibi | OK now I got your point | |
| 12:25:49 | gibi | OK now I got your point | |
| 12:26:19 | gibi | 3 resource classes and 3 QoS rule types | |
| 12:26:19 | gibi | 3 resource classes and 3 QoS rule types | |
| 12:26:25 | sean-k-mooney | so ovs would report PPS_TOTAL inventories and the directionless pps rule would consume it | |
| 12:26:26 | sean-k-mooney | so ovs would report PPS_TOTAL inventories and the directionless pps rule would consume it | |
| 12:26:32 | sean-k-mooney | yep | |
| 12:26:44 | sean-k-mooney | and no need for special logic to translate between them | |
| 12:26:44 | sean-k-mooney | and no need for special logic to translate between them | |
| 12:27:25 | gibi | OK so it is a bit more complexity to the user to select the proper QoS rule but in return we never lie about in the QoS policy about the direction awareness | |
| 12:27:25 | gibi | OK so it is a bit more complexity to the user to select the proper QoS rule but in return we never lie about in the QoS policy about the direction awareness | |
| 12:27:30 | sean-k-mooney | sorry i tought i had said that at the PTG but maybe i was not very clear | |
| 12:27:30 | sean-k-mooney | sorry i tought i had said that at the PTG but maybe i was not very clear | |
| 12:27:58 | gibi | sean-k-mooney: no it is my bad, I mixied the QoS policy part with the resource classes | |
| 12:27:58 | gibi | sean-k-mooney: no it is my bad, I mixied the QoS policy part with the resource classes | |
| 12:28:16 | gibi | next step | |
| 12:28:16 | gibi | next step | |
| 12:28:27 | sean-k-mooney | but ya its a little more complet for the tenant but simpler in code | |
| 12:28:27 | sean-k-mooney | but ya its a little more complet for the tenant but simpler in code | |
| 12:28:32 | sean-k-mooney | at least i think it is | |
| 12:28:32 | sean-k-mooney | at least i think it is | |
| 12:28:38 | gibi | yepp | |
| 12:29:54 | gibi | so next step, when we have the directionless QoS policy and later we want to implement data plane enforcement for that, then how we map the direction less min guarantee to direction aware pps rate limit rules to do enforcement? | |
| 12:29:54 | gibi | so next step, when we have the directionless QoS policy and later we want to implement data plane enforcement for that, then how we map the direction less min guarantee to direction aware pps rate limit rules to do enforcement? | |
| 12:31:06 | gibi | this is the packet rate limiter work https://bugs.launchpad.net/neutron/+bug/1912460 from Liu | |
| 12:31:09 | openstack | Launchpad bug 1912460 in neutron "[RFE] [QoS] add qos rule type packet per second (pps)" [Wishlist,In progress] - Assigned to LIU Yulong (dragon889) | |
| 12:31:09 | gibi | this is the packet rate limiter work https://bugs.launchpad.net/neutron/+bug/1912460 from Liu | |