| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-17 | |||
| 17:54:08 | sean-k-mooney | it looks like all the long tails are gone | |
| 17:55:59 | sean-k-mooney | well maybe not from 1600 yes | |
| 17:56:55 | sean-k-mooney | i think there is still an improvement? | |
| 17:57:07 | sean-k-mooney | but from the graph alone its hare to tell | |
| 17:58:07 | mnaser | sean-k-mooney: yeah, it's still early but the results look much better so far | |
| 17:58:08 | sean-k-mooney | mnaser: i think the median and mean times has droped and the make latencies are better ? is that correct | |
| 17:58:29 | mnaser | yes, and also max time dropped (with noe or two odd ones) | |
| 17:58:36 | sean-k-mooney | mnaser: in this config os-vif does not need to use privesep at all i belive if you are also using native | |
| 17:58:55 | mnaser | sean-k-mooney: its using native but it still does because i still see a privsep process up | |
| 17:59:04 | mnaser | but i cant imagine why it's needed | |
| 17:59:08 | sean-k-mooney | that is proably novas | |
| 17:59:34 | mnaser | sean-k-mooney: /openstack/venvs/nova-19.0.8/bin/python2 /openstack/venvs/nova-19.0.8/bin/privsep-helper --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpD6Vj2G/privsep.sock | |
| 17:59:37 | sean-k-mooney | if there is an os vif on i can check quickly | |
| 17:59:43 | mnaser | maybe we should look into taht | |
| 18:00:09 | sean-k-mooney | well my guess that privsep would not be need is just i dont htink we need to do most of the ops that need it in this config | |
| 18:00:24 | sean-k-mooney | it actuly was never a goal | |
| 18:00:58 | mnaser | sean-k-mooney: oh its used for a few funcs that we need for hybrid plug | |
| 18:01:08 | sean-k-mooney | yes | |
| 18:01:15 | mnaser | sean-k-mooney: i suspect because this machine has a mix of both now, it is why privsep-helper went up | |
| 18:01:24 | sean-k-mooney | ah ok that would make sense ya | |
| 18:01:31 | mnaser | if there is no hybrid ports plugged anymore and i restart nova i suspect it would disappear | |
| 18:01:37 | sean-k-mooney | we need it to create he linux bridges | |
| 18:01:52 | mnaser | ya i found those refs here https://github.com/openstack/os-vif/blob/master/vif_plug_ovs/linux_net.py | |
| 18:02:29 | mnaser | if i find a box with only taps, ill try and restart and theoretically it should not go up | |
| 18:02:40 | sean-k-mooney | yep its also need for ovs-vsctl to acess the ovs-db unix sockt but the python lib connect over tcp and does not need it | |
| 18:03:07 | sean-k-mooney | i can check my home deployment | |
| 18:03:36 | mnaser | yeah in my case im using tcp (which is also the default) | |
| 18:04:49 | sean-k-mooney | yep tcp to localhost | |
| 18:04:56 | sean-k-mooney | well 127.0.0.1 | |
| 18:04:58 | sean-k-mooney | same thing | |
| 18:05:22 | sean-k-mooney | root 48 0.0 0.0 3099668 61044 ? Sl Jun20 0:00 /var/lib/kolla/venv/bin/python /var/lib/kolla/venv/bin/privsep-helper --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmp1n9arrof/privsep.sock | |
| 18:05:40 | sean-k-mooney | so i still have it but i suspec its basically unused | |
| 18:09:53 | sean-k-mooney | ah | |
| 18:10:25 | sean-k-mooney | mnaser: https://github.com/openstack/os-vif/blob/989f5a7d92b2459444e6956facb332f19e7c6537/vif_plug_ovs/ovsdb/ovsdb_lib.py#L50 | |
| 18:10:42 | sean-k-mooney | we set the mtu | |
| 18:10:51 | sean-k-mooney | for the bridge | |
| 18:11:26 | mnaser | ah | |
| 18:11:35 | sean-k-mooney | as part of create port https://github.com/openstack/os-vif/blob/989f5a7d92b2459444e6956facb332f19e7c6537/vif_plug_ovs/ovsdb/ovsdb_lib.py#L107 | |
| 18:11:56 | sean-k-mooney | so ya that is the only privadge call which is a significat improvment | |
| 18:15:09 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add helpers for suspend, resume and reboot of server https://review.opendev.org/741285 | |
| 18:15:10 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Pass context, instance to '_create_domain' https://review.opendev.org/741286 | |
| 18:15:10 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Reject non-spawn operations for vTPM https://review.opendev.org/741500 | |
| 18:15:11 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Add emulated TPM support to Nova https://review.opendev.org/631363 | |
| 18:15:11 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add docs for vTPM support https://review.opendev.org/739213 | |
| 18:15:12 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934 | |
| 18:16:06 | stephenfin | dansmith: I won't be doing this tonight, but you might be interested in https://review.opendev.org/#/c/639934/40//COMMIT_MSG@37 | |
| 18:16:48 | dansmith | omg | |
| 18:16:50 | dansmith | scary | |
| 18:16:55 | dansmith | I | |
| 18:17:06 | stephenfin | tl;dr: I need access at the virt driver layer to both the old and new flavor when confirming/reverting a resize, and I don't have that at the moment because we're zeroing out those attributes before said calls to virt driver | |
| 18:17:20 | dansmith | am covered up in other stuff (some of which I broke) but .. yeah maybe I can look next week | |
| 18:17:32 | dansmith | yeah | |
| 18:17:42 | stephenfin | I'm planning to just delay that zeroing out 'til after, but I don't know if there'll be knock on effects or not | |
| 18:17:53 | dansmith | yeah | |
| 18:18:18 | stephenfin | seeing as your name is beside all the zeroing out, I figure you're the person to ask :) | |
| 18:18:22 | stephenfin | next week is good though | |
| 18:18:56 | sean-k-mooney | stephenfin: do you want me to take a look at that too or should i start from the begining of the seriese | |
| 18:19:34 | stephenfin | sean-k-mooney: If you can follow the flow of that code, sure :) | |
| 18:20:03 | stephenfin | You can do earlier but this is the only bit I personally consider somewhat tricky now. The rest of the series is mostly wiring up stuff | |
| 18:20:11 | sean-k-mooney | well i have not looke at the code in about 2 months so that is why i was hesitent to jump into the end of the series | |
| 18:20:27 | sean-k-mooney | i think i need to start form the start to understand all the dettails | |
| 18:21:31 | stephenfin | Sure, whatever works. It's big. You can ignore the functional test cleanup patches, obv | |
| 18:25:09 | dansmith | stephenfin: just skimming it looks like a lot is missing from that patch you linked | |
| 18:25:18 | dansmith | like, it doesn't seem to do much of what the tin describes | |
| 18:25:38 | stephenfin | \o/ | |
| 18:25:39 | dansmith | like, it removes some exceptions and adds some privsep things that don't otherwise get used | |
| 18:25:50 | dansmith | just wondering if you forgot to git add or something | |
| 18:26:03 | stephenfin | maybe I did | |
| 18:26:07 | stephenfin | checking | |
| 18:26:19 | dansmith | ohh, wait, sorry | |
| 18:26:33 | dansmith | the sorted-last v-for-virt stuff was getting hidden under tests | |
| 18:26:34 | dansmith | sorry | |
| 18:26:50 | stephenfin | ah, yes | |
| 18:27:05 | stephenfin | this is all libvirt-specific so nothing happens higher than that | |
| 18:28:19 | sean-k-mooney | stephenfin: oh am ill respin the backport after ci completes or monday | |
| 18:28:34 | sean-k-mooney | but ya i can just move the not to the commit and list the updated file | |
| 18:28:52 | stephenfin | yup, that ought to do the trick | |
| 18:29:30 | mnaser | sean-k-mooney: could we not use libvirt to set mtu and avoid that privsep call? | |
| 18:30:05 | mnaser | or maybe we already use it? https://www.irccloud.com/pastebin/ho53ZFfZ/ | |
| 18:30:17 | mnaser | so that mtu set is.. probably not super useful when we're not doing a hybrid plug | |
| 18:31:37 | sean-k-mooney | we do but its needed when using hybrid plug | |
| 18:31:59 | sean-k-mooney | i dont think we need to do it for non hybrid plug | |
| 18:32:18 | sean-k-mooney | mnaser: that said i really want to stop libvirt doing anything related to networking | |
| 18:32:38 | mnaser | ok so architecturally we want os_vif to do the work | |
| 18:32:39 | sean-k-mooney | mnaser: it casues an unavoidble race | |
| 18:32:44 | sean-k-mooney | yes | |
| 18:32:51 | mnaser | that's a fair point, so mtu stays then i guess | |
| 18:33:03 | sean-k-mooney | https://review.opendev.org/#/c/602432/ | |
| 18:33:32 | sean-k-mooney | your familar with https://bugs.launchpad.net/neutron/+bug/1734320 and https://bugs.launchpad.net/neutron/+bug/1815989 | |
| 18:33:32 | openstack | Launchpad bug 1734320 in OpenStack Compute (nova) "Eavesdropping private traffic" [Undecided,In progress] - Assigned to sean mooney (sean-k-mooney) | |
| 18:33:33 | openstack | Launchpad bug 1815989 in OpenStack Compute (nova) "OVS drops RARP packets by QEMU upon live-migration causes up to 40s ping pause in Rocky" [Medium,In progress] | |
| 18:33:38 | mnaser | ah yes | |
| 18:33:54 | mnaser | so our lack of control when tap goes up | |
| 18:34:32 | sean-k-mooney | what happens today is os-vif set up the port and the l2 agent start to wire it up | |
| 18:34:40 | sean-k-mooney | then libvirt deletes an recreates it | |
| 18:35:10 | sean-k-mooney | that is only a problem when using contrack firewall | |
| 18:35:12 | mnaser | oh yikes, i thought it was os-vif that was wiring up what libvirt creates (but then that's just a race in another direction too) | |
| 18:35:38 | sean-k-mooney | noop | |
| 18:35:49 | sean-k-mooney | libvirt basicaly does if exist delete and recreate | |
| 18:36:12 | sean-k-mooney | undoing all our careful external event stuff | |