| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-02 | |||
| 15:35:20 | stephenfin | sean-k-mooney: Nope, I'm closing gaps to allow us to remove it in the future | |
| 15:36:02 | sean-k-mooney | ... i guess i normally only change the user name so i could chagne to live_migration_username instead | |
| 15:36:08 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ignore forcing of live migration for nested instance https://review.openstack.org/605785 | |
| 15:36:08 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Consider nested allocations during allocation cleanup https://review.openstack.org/606050 | |
| 15:36:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ignore forcing of evacuation for nested instance https://review.openstack.org/606111 | |
| 15:36:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Run negative server moving tests with nested RPs https://review.openstack.org/604125 | |
| 15:36:37 | stephenfin | sean-k-mooney: yeah, that's the idea | |
| 15:37:29 | sean-k-mooney | live_migration_username=root woudl be the same as live_migration_uri = qemu+ssh://root@%s/system | |
| 15:37:50 | mdbooth | efried: Incidentally, I ran that test 250 times locally without failure. | |
| 15:38:14 | efried | mdbooth: And you had been able to reproduce it some percentage of the time prior to the fix? | |
| 15:38:29 | mdbooth | Yeah, before it failed 50%, as expected. | |
| 15:38:37 | mdbooth | That's with *my* bug | |
| 15:38:41 | owalsh | stephenfin: live_migration_tunnelled isn't related, is it? | |
| 15:38:46 | mdbooth | With the original bug it failed about 1 in 10 locally | |
| 15:39:13 | efried | mdbooth: Okay, then 250 passes is good enough for me. Thanks for your work on this. That bug has been a thorn in our collective sides for a long time. | |
| 15:39:20 | openstackgerrit | Merged openstack/os-vif master: add nested DPDK VIF classes for kuryr-kubernetes https://review.openstack.org/604992 | |
| 15:39:50 | stephenfin | owalsh: If tunneling is enabled, none of that configuration is used, right? | |
| 15:40:00 | mdbooth | efried: Hasn't passed check queue once yet, though :) | |
| 15:40:00 | owalsh | stephenfin: commit message says the opposite | |
| 15:40:07 | stephenfin | whoops | |
| 15:40:11 | mdbooth | Latest revision, that is | |
| 15:40:16 | efried | mdbooth: But it's still on the first run? | |
| 15:40:22 | mdbooth | Yeah | |
| 15:40:25 | stephenfin | That should read "disabled" | |
| 15:40:25 | efried | :) | |
| 15:40:48 | mdbooth | efried: If it fails for any related reason I promise I'll shout about it :) | |
| 15:41:53 | owalsh | stephenfin: ack, makes a lot more sense then :-) | |
| 15:42:48 | sean-k-mooney | stephenfin: why are you defaulting to qemu+tcp and not qemu+ssh if the schema is not set? | |
| 15:44:03 | sean-k-mooney | libvirt is typiclaly not exosed over tcp by default | |
| 15:45:02 | stephenfin | sean-k-mooney: That was the case previously, right? (i.e. I haven't change anything) | |
| 15:45:37 | sean-k-mooney | ya just looking at the old code. it was the case yes however the code comment is inccort in the old code too | |
| 15:46:28 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Gather 'live_migration_scheme', 'live_migration_inbound_addr' https://review.openstack.org/456572 | |
| 15:46:29 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Convert 'live_migration_inbound_addr' to HostAddressOpt https://review.openstack.org/456573 | |
| 15:46:29 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Add three new '[libvirt] live_migration_*' options https://review.openstack.org/456571 | |
| 15:47:00 | stephenfin | johnthetubaguy: Sorry, fancy hitting the first of those again. Typo fix pointed out by owalsh | |
| 15:48:49 | sean-k-mooney | stephenfin: perhapes as a follow up we should have a seperate patch to change the default. or at least advise againts using tcp as that will copy the state unencrypted | |
| 15:51:12 | sean-k-mooney | stephenfin: ssh or libssh/libssh2 would be much better defaults imho | |
| 15:52:08 | stephenfin | sean-k-mooney: Hmm, while I agree, I'm nervous to go changing defaults willy-nilly. I'm assuming ssh would require extra configuration for anyone relying on TCP? | |
| 15:52:13 | stephenfin | *currently using TCP | |
| 15:52:31 | stephenfin | And I imagine TripleO, Kolla etc. already tweak those knobs | |
| 15:53:41 | sean-k-mooney | stephenfin: yes but to use TCP that person would have had to 1 expose libvirt unencyrted and unauthentated via tcp as it is disabled by default in ubunut/debian and rhel/fedora distros | |
| 15:53:54 | sean-k-mooney | stephenfin: see https://libvirt.org/remote.html#Remote_transports | |
| 15:55:00 | sean-k-mooney | tcp is expressed not recommended for production use which is why im uneasey about that being our defualt | |
| 15:55:24 | stephenfin | sean-k-mooney: I can do that so but it'll warrant an Upgrade reno. ssh or tls as the default? | |
| 15:55:27 | sean-k-mooney | stephenfin: kolla uses ssh not sure about tripplo | |
| 15:55:48 | stephenfin | I think TripleO uses TLS if TLS everywhere is toggled but owalsh would need to confirm | |
| 15:55:52 | artom | gryf, around? You wanted to ask about NUMA live migration? | |
| 15:56:16 | sean-k-mooney | stephenfin: i would go with ssh but it likely somthing we want to discuss perhaps in the nova team meeting | |
| 15:56:21 | owalsh | stephenfin, sean-k-mooney: yea, TLS if it's enabled, otherwise ssh tunnelling | |
| 15:56:24 | sean-k-mooney | or as a bug/bluepint | |
| 15:59:10 | sean-k-mooney | stephenfin: if we dont change the default it would be nice to atleast issue a wanrning on compute agent startup so that operators are very ware that they should not be using tcp by defualt | |
| 16:00:18 | owalsh | sean-k-mooney: I wouldn't be surprised if it got #ifdef-ed out of libvirt by default too | |
| 16:00:37 | efried | mdbooth: Oh, I meant to mention this earlier... | |
| 16:01:18 | efried | we don't need context.nested, because that's implemented natively in py2 (I think as of 2.6?) and py3. | |
| 16:01:28 | sean-k-mooney | owalsh: its compiled into the centos bin but its disabled in the libvirt deamon config so you have to expcily opt in to it | |
| 16:01:49 | efried | mdbooth: s/context.nested/nested_contexts/ | |
| 16:02:16 | mdbooth | efried: It's in py3? Where? | |
| 16:02:27 | efried | mdbooth: natively | |
| 16:02:31 | owalsh | sean-k-mooney: yes, but all of the random blogs on setting up migration enabled it | |
| 16:02:33 | mdbooth | Called what? | |
| 16:02:43 | efried | mdbooth: Called 'with' :) | |
| 16:02:48 | mdbooth | efried: Oh, you mean the 'with-syntax' | |
| 16:02:54 | efried | mdbooth: maybe I'm not understanding what you're trying to do with it. | |
| 16:02:56 | sean-k-mooney | efried: really when did that get backported | |
| 16:02:57 | mdbooth | Right, but that's *not* implemented in py2 | |
| 16:03:02 | mdbooth | The optional bit, that is | |
| 16:03:18 | sean-k-mooney | efried: we still have at least 1 place where we stub out compatiably in nova | |
| 16:03:20 | mdbooth | You can't say: | |
| 16:03:31 | mdbooth | with X, Y (unless Y is None) | |
| 16:03:32 | mdbooth | : | |
| 16:04:16 | mdbooth | Basically the choices for optional composition are contextlib.nested, and ExitStack, and neither is available in both py2 and py3 | |
| 16:04:31 | efried | mdbooth: oh, I was missing the filter | |
| 16:04:37 | stephenfin | mdbooth: https://pythonclock.org/ | |
| 16:05:12 | mdbooth | stephenfin: Python 3 is the culprit here, because Python 2 was already here. | |
| 16:05:43 | mdbooth | Deprecating a whole programming language was a monumentally stupid idea. | |
| 16:06:02 | openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: [Stable Only] Add amd-ssbd and amd-no-ssb CPU flags https://review.openstack.org/607296 | |
| 16:09:12 | mdbooth | efried: Incidentally, you can make Y a 'null context', i.e. a context manager which does nothing other than execute whatever it's given. | |
| 16:09:25 | mdbooth | And py3 includes a contextlib.nullcontext | |
| 16:09:29 | mdbooth | But py2 doesn't | |
| 16:09:44 | mdbooth | So I could have written one, but it would still be duplicating code in py3 | |
| 16:10:32 | sean-k-mooney | mdbooth: it could jsut delegate to the py3 verions on py3 and implement the 10 lines it takes on py2. | |
| 16:10:42 | efried | or after the import: | |
| 16:10:43 | efried | if not hasattr(contextlib, 'nullcontext'): | |
| 16:10:43 | efried | contextlib.nullcontext = lambda x: x | |
| 16:10:43 | efried | ? | |
| 16:10:57 | sean-k-mooney | we cant drop py2 support for 2 more releases so i dont see an issue with it | |
| 16:10:58 | efried | anyway, what you've done is foyne. | |
| 16:11:10 | mdbooth | efried: That's not quite a nullcontext. You probably need at least 4 lines for it | |
| 16:11:16 | efried | yeah yeah | |
| 16:11:49 | mdbooth | efried: The point is there were a bunch of ways to do it, but none of them are as good as what you could do if we were only targetting a single programming language. | |
| 16:12:09 | efried | yup, I get it. You had to choose among several ugly options. What you've done is foyne. | |
| 16:15:07 | artom | Weird - I'm trying to reproduce a bug where if an instance reschedules it gets two vifs - I'm trying to force a reschedule by just raising in build_and_run_instance one one of the two computes. Except *no instance ever lands on that compute* | |
| 16:15:54 | artom | That host definitely doesn't get filtered out, it's enabled, etc | |
| 16:15:56 | openstackgerrit | Matthew Booth proposed openstack/nova master: DNM: Add some debug logging https://review.openstack.org/607301 | |
| 16:18:26 | sean-k-mooney | artom: try raising right at the point the libvirt driver calls libvir to spawn the instance | |
| 16:18:52 | gibi | mriedem: hi, do you have something for the notification subteam meeting? I don't have any | |
| 16:19:14 | artom | sean-k-mooney, it's not even that. I've booted 10 instances with --min-count, and they've *all* gone to the "wrong" host | |
| 16:21:02 | sean-k-mooney | artom: you didnt hit the auto host down on faiure codepath? | |
| 16:21:33 | artom | sean-k-mooney, well, how would I see that? The hypervisor shows as up, same for the service | |
| 16:23:39 | dansmith | artom: there's a weigher now | |