Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-02
15:14:12 openstackgerrit Balazs Gibizer proposed openstack/nova master: Enable nested allocation candidates in scheduler https://review.openstack.org/585672
15:14:31 openstackgerrit Balazs Gibizer proposed openstack/nova master: Use provider tree in virt FakeDriver https://review.openstack.org/604083
15:15:10 openstackgerrit Balazs Gibizer proposed openstack/nova master: Refactor allocation checking in functional tests https://review.openstack.org/607287
15:15:27 openstackgerrit Balazs Gibizer proposed openstack/nova master: Run ServerMovingTests with nested resources https://review.openstack.org/604084
15:15:44 sean-k-mooney gibi: so while macvtap sriov livemigration i was able to cause the same neutron db confilct and fush error so it looks liek tehre is definetly a neutron portbining bug at least on master
15:19:12 gibi sean-k-mooney: good info
15:20:17 sean-k-mooney i need to do more diging to see if this is just an issue for sriov or if i get the same behavior with just ovs so ill test that later today.
15:20:52 gibi sean-k-mooney: good info
15:20:58 gibi sean-k-mooney: oops
15:21:04 gibi sean-k-mooney: I mean, thank you
15:22:02 sean-k-mooney no worries this will be a blocker for me to enable sriov livemigration anyway so i need to figure out how to fix it one way or another
15:22:07 johnthetubaguy stephenfin: what about uri being last in the list, leaving the mostly commonly changed ones first? Not sure I am actually that bothered by the order actually, kinda happy with whatever gets the later fixes in quickly, which is probably what you have now.
15:22:25 jaypipes sean-k-mooney: ack, will do. thx.
15:22:55 stephenfin johnthetubaguy: Sure, I can do that. I'm writing an expanded message to better explain the reasoning for why this is necessary
15:29:39 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Convert 'live_migration_inbound_addr' to HostAddressOpt https://review.openstack.org/456573
15:29:39 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Gather 'live_migration_scheme', 'live_migration_inbound_addr' https://review.openstack.org/456572
15:29:40 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Add three new '[libvirt] live_migration_*' options https://review.openstack.org/456571
15:29:54 stephenfin johnthetubaguy: aaand done. Hopefully that's clearer
15:30:21 johnthetubaguy stephenfin: cool, thanks
15:30:45 stephenfin owalsh: You probably want to look at those (again) too ^
15:30:59 owalsh stephenfin: ack
15:31:49 johnthetubaguy stephenfin: although now you have scheme first which basically says "please don't touch this config"
15:32:22 stephenfin johnthetubaguy: I was just about to comment on https://review.openstack.org/#/c/456572/8/nova/conf/libvirt.py@278 about that
15:32:28 stephenfin They're in URI order :)
15:32:54 stephenfin I wonder if they should all be advanced?
15:33:05 stephenfin *marked as advanced
15:33:31 stephenfin Seeing as in many (most?) cases the driver can configure this automatically for us
15:33:54 johnthetubaguy stephenfin: it should be marked advanced, yeah
15:34:13 johnthetubaguy stephenfin: I can live with URI order then :)
15:34:57 sean-k-mooney stephenfin: wait you are removing live_migration_uri
15:35:20 stephenfin sean-k-mooney: Nope, I'm closing gaps to allow us to remove it in the future
15:36:02 sean-k-mooney ... i guess i normally only change the user name so i could chagne to live_migration_username instead
15:36:08 openstackgerrit Balazs Gibizer proposed openstack/nova master: Consider nested allocations during allocation cleanup https://review.openstack.org/606050
15:36:08 openstackgerrit Balazs Gibizer proposed openstack/nova master: Ignore forcing of live migration for nested instance https://review.openstack.org/605785
15:36:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Run negative server moving tests with nested RPs https://review.openstack.org/604125
15:36:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Ignore forcing of evacuation for nested instance https://review.openstack.org/606111
15:36:37 stephenfin sean-k-mooney: yeah, that's the idea
15:37:29 sean-k-mooney live_migration_username=root woudl be the same as live_migration_uri = qemu+ssh://root@%s/system
15:37:50 mdbooth efried: Incidentally, I ran that test 250 times locally without failure.
15:38:14 efried mdbooth: And you had been able to reproduce it some percentage of the time prior to the fix?
15:38:29 mdbooth Yeah, before it failed 50%, as expected.
15:38:37 mdbooth That's with *my* bug
15:38:41 owalsh stephenfin: live_migration_tunnelled isn't related, is it?
15:38:46 mdbooth With the original bug it failed about 1 in 10 locally
15:39:13 efried mdbooth: Okay, then 250 passes is good enough for me. Thanks for your work on this. That bug has been a thorn in our collective sides for a long time.
15:39:20 openstackgerrit Merged openstack/os-vif master: add nested DPDK VIF classes for kuryr-kubernetes https://review.openstack.org/604992
15:39:50 stephenfin owalsh: If tunneling is enabled, none of that configuration is used, right?
15:40:00 owalsh stephenfin: commit message says the opposite
15:40:00 mdbooth efried: Hasn't passed check queue once yet, though :)
15:40:07 stephenfin whoops
15:40:11 mdbooth Latest revision, that is
15:40:16 efried mdbooth: But it's still on the first run?
15:40:22 mdbooth Yeah
15:40:25 efried :)
15:40:25 stephenfin That should read "disabled"
15:40:48 mdbooth efried: If it fails for any related reason I promise I'll shout about it :)
15:41:53 owalsh stephenfin: ack, makes a lot more sense then :-)
15:42:48 sean-k-mooney stephenfin: why are you defaulting to qemu+tcp and not qemu+ssh if the schema is not set?
15:44:03 sean-k-mooney libvirt is typiclaly not exosed over tcp by default
15:45:02 stephenfin sean-k-mooney: That was the case previously, right? (i.e. I haven't change anything)
15:45:37 sean-k-mooney ya just looking at the old code. it was the case yes however the code comment is inccort in the old code too
15:46:28 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Gather 'live_migration_scheme', 'live_migration_inbound_addr' https://review.openstack.org/456572
15:46:29 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Add three new '[libvirt] live_migration_*' options https://review.openstack.org/456571
15:46:29 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Convert 'live_migration_inbound_addr' to HostAddressOpt https://review.openstack.org/456573
15:47:00 stephenfin johnthetubaguy: Sorry, fancy hitting the first of those again. Typo fix pointed out by owalsh
15:48:49 sean-k-mooney stephenfin: perhapes as a follow up we should have a seperate patch to change the default. or at least advise againts using tcp as that will copy the state unencrypted
15:51:12 sean-k-mooney stephenfin: ssh or libssh/libssh2 would be much better defaults imho
15:52:08 stephenfin sean-k-mooney: Hmm, while I agree, I'm nervous to go changing defaults willy-nilly. I'm assuming ssh would require extra configuration for anyone relying on TCP?
15:52:13 stephenfin *currently using TCP
15:52:31 stephenfin And I imagine TripleO, Kolla etc. already tweak those knobs
15:53:41 sean-k-mooney stephenfin: yes but to use TCP that person would have had to 1 expose libvirt unencyrted and unauthentated via tcp as it is disabled by default in ubunut/debian and rhel/fedora distros
15:53:54 sean-k-mooney stephenfin: see https://libvirt.org/remote.html#Remote_transports
15:55:00 sean-k-mooney tcp is expressed not recommended for production use which is why im uneasey about that being our defualt
15:55:24 stephenfin sean-k-mooney: I can do that so but it'll warrant an Upgrade reno. ssh or tls as the default?
15:55:27 sean-k-mooney stephenfin: kolla uses ssh not sure about tripplo
15:55:48 stephenfin I think TripleO uses TLS if TLS everywhere is toggled but owalsh would need to confirm
15:55:52 artom gryf, around? You wanted to ask about NUMA live migration?
15:56:16 sean-k-mooney stephenfin: i would go with ssh but it likely somthing we want to discuss perhaps in the nova team meeting
15:56:21 owalsh stephenfin, sean-k-mooney: yea, TLS if it's enabled, otherwise ssh tunnelling
15:56:24 sean-k-mooney or as a bug/bluepint
15:59:10 sean-k-mooney stephenfin: if we dont change the default it would be nice to atleast issue a wanrning on compute agent startup so that operators are very ware that they should not be using tcp by defualt
16:00:18 owalsh sean-k-mooney: I wouldn't be surprised if it got #ifdef-ed out of libvirt by default too
16:00:37 efried mdbooth: Oh, I meant to mention this earlier...
16:01:18 efried we don't need context.nested, because that's implemented natively in py2 (I think as of 2.6?) and py3.
16:01:28 sean-k-mooney owalsh: its compiled into the centos bin but its disabled in the libvirt deamon config so you have to expcily opt in to it
16:01:49 efried mdbooth: s/context.nested/nested_contexts/
16:02:16 mdbooth efried: It's in py3? Where?
16:02:27 efried mdbooth: natively
16:02:31 owalsh sean-k-mooney: yes, but all of the random blogs on setting up migration enabled it
16:02:33 mdbooth Called what?
16:02:43 efried mdbooth: Called 'with' :)
16:02:48 mdbooth efried: Oh, you mean the 'with-syntax'
16:02:54 efried mdbooth: maybe I'm not understanding what you're trying to do with it.
16:02:56 sean-k-mooney efried: really when did that get backported
16:02:57 mdbooth Right, but that's *not* implemented in py2
16:03:02 mdbooth The optional bit, that is
16:03:18 sean-k-mooney efried: we still have at least 1 place where we stub out compatiably in nova
16:03:20 mdbooth You can't say:
16:03:31 mdbooth with X, Y (unless Y is None)
16:03:32 mdbooth :

Earlier   Later