Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-06
15:58:14 sean-k-mooney stephenfin: sorry in advance https://review.openstack.org/#/c/541290/1 comments in line
15:58:23 stephenfin sean-k-mooney: :)
15:59:33 sean-k-mooney stephenfin: by the way where did the 50% performanc number come from? the numa effect on ovs-dpdk should be in the singel figure if you turn on the numa support in dpdk
16:00:14 stephenfin sean-k-mooney: I was wondering if anyone was going to call me on that. It's been quoted numerous times to me by fbaudin, but I think it's based on internal analysis
16:00:43 stephenfin I can root out the details and add them in some form to the spec
16:00:50 stephenfin ...which clearly needs more work :)
16:01:20 sean-k-mooney stephenfin: ya but i have mention to him in the past that we have messured much lower numbers in the past if you turn on the numa support in dpdk/ovs which is off by default
16:03:22 stephenfin sean-k-mooney: Thanks, I wasn't aware of that. There's some great info in there
16:03:33 stephenfin sean-k-mooney: You aren't, by any chance, going to be in Dublin, are you?
16:04:25 sean-k-mooney i proably will be. im not sure if i will be there on behalf of intel though as my travel has not been approved so i might not be the there the whole week if i go up myself
16:14:39 sean-k-mooney stephenfin: just looking at the at the dpdk-devel packages for centos and fedora and they appear to have the the numa support turned on. and the ovs packages depend on libnuma so they "should" have numa support but if the testing was not done with at least 1 pmd per numa node then ovs will not be able to compensate internally if the vm and nic are not on the same numa node
16:15:26 stephenfin sean-k-mooney: Could you stick that in the review too? This is all good data to use while working on this
16:15:48 stephenfin sean-k-mooney: Planning on setting up an AIO deployment this evening to start hacking on this and see what I can get working
16:16:04 efried sean-k-mooney: Are you from Dublin originally?
16:16:41 sean-k-mooney stephenfin: yep will do. if you use networking-ovs-dpdk/devstack just be aware we dont acally compile in the ovs support you will have to add it in but patches are welcome.
16:16:44 sahid mriedem can you ack this https://review.openstack.org/#/c/515008/, it's a bugfix
16:17:18 sean-k-mooney efried: no i lived there for a year im on the other cost in shannon but its only about a 3 hour drive or so
16:17:44 efried I thought pronouncing 'th' as 't' was a Dublin thing.
16:18:07 stephenfin sean-k-mooney: You mean you don't compile in DPDK support...right?
16:18:29 stephenfin If not, what about compiling in?
16:18:36 stephenfin efried: For example?
16:18:49 efried stephenfin: "taught" for "thought"
16:19:26 sean-k-mooney efried: they pronounce 'th' more like a 'd' but 'th' as a 't' is more of a tipperary thing that i generally avoid doing in speach
16:20:06 stephenfin Ah, no. My father, who's also from the west coast, does that too
16:20:07 stephenfin One, two, tree...
16:21:25 sean-k-mooney ya losing the th is a munster thing mainly
16:22:49 sean-k-mooney efried: also i spell phonetically not correctly so the acent i write with is not the one i speak with. at least not entirely
16:22:50 stephenfin Pronouncing 's' as 'sh' is also a thing. Fun
16:25:54 sean-k-mooney stephenfin: speaking of fun did you see this yet https://bugs.launchpad.net/nova/+bug/1747496 all those mtu patch last year and this is still a thing ...
16:25:55 openstack Launchpad bug 1747496 in OpenStack Compute (nova) "MTUs are not set for VIFs if using kernel ovs + hybrid plug = false" [Undecided,New] - Assigned to sean mooney (sean-k-mooney)
16:26:15 stephenfin Oh, I saw the ping for that last night and forgot
16:26:29 stephenfin Fun indeed
16:27:02 sean-k-mooney isint nice when libvirt tries to be helpful and plug vif for us to not over burden os-vif with all the work
16:27:04 openstackgerrit James E. Blair proposed openstack/python-novaclient stable/pike: Zuul: Remove project name https://review.openstack.org/541370
16:28:34 sean-k-mooney stephenfin: i found it while i was deploying or new internal devcloud with jumbo frames and no security group dirver
16:29:03 stephenfin sean-k-mooney: Oh, that crap
16:29:12 stephenfin I want to remove that _entire_ code path
16:29:36 stephenfin In fact, I think I started on doing it about a year ago and got distracted/confused
16:30:08 sean-k-mooney stephenfin: ya so do i but as a backportable solution first im going to set the mtu in the xml for libvirt >3.2 and then stop libvirt form helping and let os-vif do it as a followup patch
16:30:28 stephenfin sean-k-mooney: Yup, I can get onboard with that approach
16:39:17 cfriesen sean-k-mooney: just curious, why is numa support in dpdk off by default? seems like that'd be a good thing.
16:41:11 mriedem melwitt: comments in https://review.openstack.org/#/c/515008/ since sahid asked for review and left
16:42:08 openstackgerrit Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
16:46:28 stephenfin cfriesen: I'm not sean-k-mooney, but I'm guessing it's the need for an extra dependency?
16:49:23 bauzas mriedem: I have a noob question, how can you know that vol.volume_id is set in https://review.openstack.org/#/c/539739/4/nova/virt/libvirt/driver.py@7430 .
16:49:24 bauzas ?
16:51:39 mriedem bauzas: because when we attach a volume to an instance, we save the volume_id on the bdm record,
16:51:50 mriedem see nova.virt.block_device.DriverVolumeBlockDevice.attach
16:52:05 mriedem DriverVolumeBlockDevice wraps an proxies a BlockDeviceMapping object,
16:52:22 mriedem and the @update_db decorator on attach() updates the wrapped bdm object
16:53:34 mriedem i should say that is the case for bfv with source type blank/image/snapshot where nova creates the volume and then attaches it,
16:53:53 mriedem for normal volume attach to an existing instance, the bdm is created in compute here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L5220
16:53:54 bauzas ok, looking at the code
16:53:56 mriedem with the volume_id
16:56:11 hrw zuul forgot about me ;( 7h since patch upload and no results
16:56:33 bauzas hrw: you probably haven't see the gate status
16:56:46 bauzas they had problems in the morning with very slow processing
16:56:51 hrw bauzas: probably
16:57:35 kashyap dansmith: Okay, after talking to libvirt and QEMU migration maintainers, you're right in pointing out here: https://review.openstack.org/#/c/497457/18/nova/virt/libvirt/driver.py
16:57:39 kashyap That...
16:57:40 hrw ok I see
16:57:45 kashyap dansmith: It is an absolutely horrible hack
16:58:10 kashyap dansmith: ...to slow down migration via by limiting the speed to 1 MiB/s
16:58:11 dansmith kashyap: Well, I know it's a hack, but wasn't sure if there was another option :)
16:58:25 hrw bauzas: will see. in worst case 'recheck' in the morning tomorrow
16:58:56 bauzas hrw: you know the status page or not ?
16:59:00 kashyap dansmith: Discussing the design aspects w/ them. I have a half-comment in draft, will complete it in a bit :-)
16:59:07 dansmith kashyap: excellent, thanks!
16:59:08 kashyap dansmith: Just wanted to confirm your instinct :P
16:59:14 hrw bauzas: https://wiki.openstack.org/wiki/Infrastructure_Status one?
16:59:14 dansmith kashyap: awesome :P
17:00:14 stephenfin (I looked at the underlying bug and also assumed there wasn't an alternative)
17:00:41 bauzas hrw: http://zuul.openstack.org/ and http://status.openstack.org/openstack-health/#/
17:01:07 hrw thx
17:01:28 openstackgerrit Dan Smith proposed openstack/nova master: Bump compute RPC API to version 5.0 https://review.openstack.org/541005
17:01:28 openstackgerrit Dan Smith proposed openstack/nova master: Compute RPC client bump to 5.0 https://review.openstack.org/541035
17:01:48 kashyap dansmith: At least, QEMU migratoin upstream said: "If Nova can define a need, we can add a tunable that libvirt can use"
17:02:08 dansmith ++
17:03:08 stephenfin dansmith: <3 I've been waiting for RPC version bumps for aaaaages
17:03:41 stephenfin Mostly thanks to lots of comments from ndipanov saying we can cruft in virt/hardware when that happens
17:03:44 dansmith stephenfin: don't review it yet.. still working out the bugs. I think this iteration is close, but need tempest tests
17:04:04 dansmith stephenfin: well, then you need to look at those notes and make sure I've got the bits that need to be removed
17:04:16 stephenfin I can do that
17:04:23 dansmith stephenfin: I've deprecated the things I know about, but if there are others, now's the time...
17:06:32 gibi bauzas: could you look at https://review.openstack.org/#/c/525242 again? (and the one below)
17:06:45 gibi bauzas: mriedem is already +2 on both
17:06:48 bauzas fortunately I have https://docs.openstack.org/nova/latest/user/block-device-mapping.html for help
17:07:00 bauzas gibi: only if that's super quick
17:07:22 gibi bauzas: you already reviewed it :)
17:07:32 gibi bauzas: so I hope it is a quick one
17:07:33 bauzas gibi: I'm reviewing mriedem's change https://review.openstack.org/#/c/539739/4 that requires some BDM knowledge
17:07:49 bauzas ^ RC1 blocker, your call, core
17:08:05 mriedem did you lay down a guantlet just now?
17:08:18 mriedem *gauntlet even
17:08:23 gibi bauzas: sure, the LUKS patch has priority
17:09:12 bauzas hah, no
17:09:53 bauzas for the first time of my poor life, I have the opportunity to understand the BDM mess^H^H implementation
17:10:52 dansmith only 341 things in check.. I look forward to my CI results later in the week
17:12:17 bauzas dansmith: yeah like I said to hrw, that GMT morning was awful
17:21:14 kashyap stephenfin: Quickly, for your point on that live migration issue -- yeah, for the immediate term, we may have to go with the ugly hack of slowing the n/w bandwidth. Until we figure out a way to stop migration much earlier in Nova.

Earlier   Later