| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-16 | |||
| 13:51:02 | hkominos | Will do | |
| 13:58:56 | openstackgerrit | Elod Illes proposed openstack/nova stable/pike: [stable-only] gate: Pin CEPH_RELEASE to nautilus in LM hook https://review.opendev.org/c/openstack/nova/+/780852 | |
| 14:10:14 | stephenfin | gibi: I'm reworking Sean's "block unsupported ops with vDPA interface" patch to return HTTP 409 (Conflict) instead of HTTP (Forbidden) since that seems more sensible. It's not a permissions or access issue, IMO. Do you agree? | |
| 14:31:57 | Luzi | hey, is someone around who knows the qemu native tls config options? I think i might have found a bug in the guide: https://docs.openstack.org/nova/latest/admin/secure-live-migration-with-qemu-native-tls.html | |
| 14:34:53 | kashyap | Luzi: Hi, I think I wrote that doc :) | |
| 14:35:05 | kashyap | Luzi: What's the bug? Feel free to amend / send a pull-req :) | |
| 14:35:20 | kashyap | s/pull-req/patch/ | |
| 14:36:20 | kashyap | At the bottom right, there is also "found an error? report a bug" | |
| 14:36:34 | Luzi | i followed that guide but got to a point when openstack did not use the tls path, but kept using the tcp path | |
| 14:36:56 | Luzi | i onlny found out because i used tcpdump on both ports | |
| 14:37:55 | kashyap | Luzi: Hmm, if you followed that guide to the word, then your setup should definitely be using the QEMU-native TLS | |
| 14:38:49 | Luzi | i was looking through code and found out, that the uri used for live migration is different from the connection uri and uses a config option not mentioned in the guide | |
| 14:39:21 | kashyap | Luzi: Hmm, possible the guide got slightly outdated ... as migration-related code got reworked | |
| 14:39:22 | Luzi | only after i set live_migration_scheme = tls | |
| 14:39:32 | Luzi | the traffic was encrypted | |
| 14:39:44 | Luzi | and using the correct port :) | |
| 14:41:06 | kashyap | Luzi: I see. That's correct _scheme is favoured | |
| 14:41:19 | kashyap | And _uri parameter is deprecated in favor of the above | |
| 14:41:41 | Luzi | but when you don't set it, as i had not done it, the code uses the hardcoded tcp parameter | |
| 14:41:44 | fungi | looks like that config option was added by https://review.openstack.org/410817 which merged in early 2017 | |
| 14:42:08 | kashyap | fungi: Heya; yep. That timeframe is right ... /me clicks | |
| 14:42:35 | kashyap | Yep; patch is also correct. | |
| 14:42:43 | Luzi | live_migration_scheme = tls | |
| 14:42:45 | fungi | first appeared in nova 15.0.0, so ocata and later | |
| 14:42:48 | kashyap | Luzi: Also you don't tell what version of OpenStack you were using? | |
| 14:43:13 | kashyap | Yeah, what fungi says. | |
| 14:43:19 | Luzi | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1224 | |
| 14:43:23 | openstackgerrit | Stephen Finucane proposed openstack/nova master: pci: Add vDPA vnic to PCI request mapping and filtering https://review.opendev.org/c/openstack/nova/+/778350 | |
| 14:43:24 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add functional test for vDPA device https://review.opendev.org/c/openstack/nova/+/780112 | |
| 14:43:24 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Block unsupported actions with vDPA https://review.opendev.org/c/openstack/nova/+/780333 | |
| 14:43:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add release note for vDPA https://review.opendev.org/c/openstack/nova/+/780866 | |
| 14:43:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: tests: Make mdev stubs work like vDPA https://review.opendev.org/c/openstack/nova/+/780234 | |
| 14:43:31 | stephenfin | gibi: lyarwood: ^ | |
| 14:43:36 | Luzi | i am working on train and looking to upgrade soon | |
| 14:44:12 | Luzi | i posted the line of code which seems to be still on master | |
| 14:45:06 | fungi | same change deprecated live_migration_uri | |
| 14:45:36 | kashyap | (Yep) | |
| 14:46:22 | Luzi | in the guide the only config option which is mentioned is live_migration_with_native_tls = true | |
| 14:46:36 | Luzi | which is definitely not enough to enable native tls | |
| 14:46:40 | kashyap | Luzi: So, indeed - it checks if the config option is set, then uses the _scheme, if not defaults to TLS | |
| 14:47:20 | fungi | so alternatively, the document could warn that setting the _scheme will prevent a natural fallback to tls | |
| 14:47:21 | Luzi | well i thought it worked, until i started tcpdump | |
| 14:47:37 | Luzi | tcp | |
| 14:47:51 | Luzi | the default in code is tcp | |
| 14:48:10 | kashyap | Luzi: Err, defaults to TCP, I mean, I'm sorry. | |
| 14:48:22 | Luzi | the guid should at least say it is needed to set the _scheme to tls | |
| 14:48:38 | fungi | ayup, i agree it defaults to 'tcp' there | |
| 14:49:10 | kashyap | Luzi: Right; I'll check the installer code to double-confirm; and then can update the doc | |
| 14:49:20 | fungi | okay, so the doc just needs updating to mention setting live_migration_scheme="tls"? | |
| 14:49:37 | Luzi | i'm just worried, that this may also concern some users which did not check their deployment with a tcpdump :D | |
| 14:49:51 | fungi | it got mentioned in the release notes (under features) but yeah that's fairly hidden | |
| 14:50:17 | Luzi | it should be enough, to update the guide, i think | |
| 14:50:37 | fungi | and even the release note doesn't come out and say it's needed for turning on tls | |
| 14:50:58 | Luzi | fungi, yeah thats the point | |
| 14:51:06 | Luzi | i needed a whole day to find out :D | |
| 14:51:22 | kashyap | fungi: Yep - on the doc needs updating. Luzi: Good catch! | |
| 14:51:46 | fungi | Luzi: if you're worried that we need to do some outreach, the typical process for that is an openstack security note: https://wiki.openstack.org/wiki/Security/Security_Note_Process | |
| 14:52:00 | Luzi | i need to go off now, thank you for replying so quickly kashyap and fungi :) | |
| 14:52:03 | kashyap | Luzi: So you need _both_: | |
| 14:52:06 | kashyap | - live_migration_scheme | |
| 14:52:14 | fungi | once published, we can send a copy to the openstack-announce and openstack-discuss mailing lists | |
| 14:52:16 | kashyap | - live_migration_with_native_tls | |
| 14:52:44 | Luzi | fungi, i will do that when i'm at home :) | |
| 14:52:47 | Luzi | kashyap, yes | |
| 14:54:57 | kashyap | (Ah, Luzi is gone before I wanted to mention a way to actually verify how to check native TLS is in effect for disks during migration w/ low-level logs.) | |
| 15:03:54 | lyarwood | stephenfin: https://review.opendev.org/c/openstack/nova/+/780333 - quick commit message nit in there but otherwise LGTM | |
| 15:07:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add functional test for vDPA device https://review.opendev.org/c/openstack/nova/+/780112 | |
| 15:07:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Block unsupported actions with vDPA https://review.opendev.org/c/openstack/nova/+/780333 | |
| 15:07:20 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: tests: Make mdev stubs work like vDPA https://review.opendev.org/c/openstack/nova/+/780234 | |
| 15:07:20 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add release note for vDPA https://review.opendev.org/c/openstack/nova/+/780866 | |
| 15:08:10 | lyarwood | ta | |
| 15:17:37 | lyarwood | stephenfin: https://review.opendev.org/c/openstack/nova/+/780866 - same in the releasenote btw | |
| 15:17:49 | stephenfin | gdi :) | |
| 15:17:59 | stephenfin | will fix | |
| 15:18:36 | lyarwood | np my review ratio thanks you ;) | |
| 15:18:56 | lyarwood | joys of being a stable core | |
| 15:21:31 | gibi | stephenfin: ack, I have no problem with 409, and I will review the series soon | |
| 15:52:32 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Deprecate `live_migration_tunnelled` https://review.opendev.org/c/openstack/nova/+/780908 | |
| 15:54:27 | kashyap | gibi: stephenfin: Would be nice to get it going in Wallaby --^. Will reduce some future "live migration config fatigue" | |
| 15:55:23 | gibi | kashyap: ack, I will check after the vdpa series | |
| 15:55:38 | kashyap | Sure; no prob. It's just a deprecation meta-work; that's all. | |
| 15:55:53 | kashyap | I'll add a code-reference in the change comment | |
| 15:58:41 | lyarwood | kashyap: random question, do we have coverage of live_migration_with_native_tls in the gate somewhere? | |
| 15:59:15 | lyarwood | kashyap: I can't find anything but that smells like something we should enable in nova-next at least | |
| 15:59:29 | kashyap | lyarwood: Good question, I do not know; and near as I know, no. It requires a full TLS env to be setup -- which is what TripleO does | |
| 15:59:44 | kashyap | See my prerequisite here: https://docs.openstack.org/nova/latest/admin/secure-live-migration-with-qemu-native-tls.html#prerequisites | |
| 16:00:33 | kashyap | lyarwood: I swear there is some automated test of this upstream (perhaps the upstream whitebox from OSP QE folks), can't find a link yet | |
| 16:01:14 | lyarwood | kashyap: yeah it's likely on the TripleO side and sorry I thought the virt tools would automate the CA creation etc, if not ignore me as it's going to be too much work to enable in devstack for nova-next etc | |
| 16:01:54 | kashyap | lyarwood: Yeah, it requires good old PKI setup. It _can_ be automated, using GnuTLS | |
| 16:02:10 | kashyap | lyarwood: If you don't mind holding your nose, something like ... https://kashyapc.fedorapeople.org/Auto-Setup-GnuTLS/Setup-GnuTLS-CA-and-ServerCert.bash | |
| 16:02:55 | lyarwood | that said we do have some novnc tls stuff in devstack so... | |
| 16:03:12 | kashyap | I see | |
| 16:03:36 | kashyap | But pretty sure I recall chatting w/ TripleO folks to have an automated test. Once I find a URL, I'll link it in the change or post here | |
| 16:30:13 | bauzas | gibi: others, folks, i'm taking an urgent PTO tomorrow, my wife is turning 40 as well | |
| 16:32:31 | gibi | bauzas: ack, have a nice celebration! | |
| 16:33:57 | gibi | bauzas: will you push a new rev from the rpc bump patch before you leave? | |
| 16:34:36 | bauzas | gibi: not sure :( | |
| 16:34:53 | bauzas | still fixing to not support 5.0 | |
| 16:36:48 | gibi | bauzas: no worries just preparing my review queue | |
| 16:39:34 | Luzi | kashyap, are you around? | |