Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-08-09
19:39:34 jaypipes patches are cheap.
19:39:46 mriedem http://logs.openstack.org/10/488510/30/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/5a36c66/logs/screen-n-cpu.txt.gz#_Aug_09_16_27_46_100320
19:40:37 mriedem yeah so on resize to same host, it's the _instance_to_allocations_dict that overwrites the doubled allocation
19:40:46 mriedem dansmith: which is what you were saying
19:40:54 mriedem Sending allocation for instance {'VCPU': 1, 'MEMORY_MB': 64} {{(pid=19567) _allocate_for_instance /opt/stack/new/nova/nova/scheduler/client/report.py:924}}
19:40:59 dansmith yeah
19:41:01 jaypipes mriedem: right, and that's the thing that doens't get "corrected" until the later patch
19:41:14 mriedem that's at Aug 09 16:27:46.100320
19:41:29 mriedem the doubled up allocation in the scheduler was at Aug 09 16:27:37.633725
19:41:58 mriedem ok so maybe not a turd furgeson after all
19:42:20 mriedem although, an ocata compute will trample that and then the last patch will double the allocations again?
19:42:40 dansmith mriedem: while we're healing,
19:42:47 dansmith we'll fix it eventually on the destination node
19:43:26 dansmith once we're all pike, we don't trample and thus we're good
19:44:09 mriedem but we'll continue to log ERRORs?
19:44:41 dansmith I dunno I didn't look at the actual site where we log that, but probably so
19:45:09 mriedem ok, and we probably don't have a job that would test it either,
19:45:23 mriedem would require a grenade multinode job that runs a migration
19:45:34 mriedem we do have a multinode grenade job that runs live migration...
19:45:45 mriedem back and forth too so that actually might show it
19:45:48 dansmith mriedem: you can borrow my pistol when I'm done with it
19:46:35 mriedem i'll be classy and do Seppuku
19:46:51 dansmith heh, less mess, how midwestern of you
19:47:07 mriedem disemboweling is still a mess
19:47:26 mriedem i'll do it in the soaker tub
19:48:15 mriedem this error log case wouldn't be the end of the world, we could release with that and backport a fix later,
19:48:26 mriedem adding some logic to determine we're about to push a 0 allocation and just not do that
19:49:03 dansmith well, normally that would be a legit error case,
19:49:07 dansmith and worth the log
19:49:26 dansmith so I dunno.. maybe if not has_ocata and ==0, then log error or something
19:51:08 mriedem my point is, it doesn't cause things to fail, so not a regression, so we could cleanup the error log later and backport if necessary, yeah?
19:51:26 dansmith yes
19:51:51 openstackgerrit Merged openstack/nova-specs master: doc: Remove crud from Sphinx conf.py https://review.openstack.org/456129
19:52:10 mriedem jaypipes: so you're putting a patch at the bottom of the series to remove that bogus log in the filter scheduler, and then fixing the pep8 in https://review.openstack.org/#/c/491850/ on top of that?
19:52:40 mriedem and then i think we can limp https://review.openstack.org/#/c/488510/ in and fix my other problems in a follow up
19:53:11 mriedem and then once we get a good run on https://review.openstack.org/#/c/491012/ we call it a ball for rc1
19:53:19 mriedem oh except this of course https://review.openstack.org/#/c/491491/
19:53:42 dansmith jay needs to look at that one I think
19:53:53 dansmith it makes sense to me, but I figure he should look
19:53:54 mriedem gibi's?
19:53:57 mriedem yes agree
19:53:59 mriedem he wrote it
19:53:59 dansmith yeah
19:54:03 jaypipes I'm still addressing mriedem's review comments on the confirm/reverrt resize patch.
19:54:06 mriedem i mean, the code that's changing
19:54:07 jaypipes should be done shortly.
19:54:11 dansmith mriedem: right
19:54:29 mriedem although it does say, "this should really raise NoValidHost but tests...."
19:58:51 mriedem what i don't understand in his test https://review.openstack.org/#/c/490814/ is how the resize doesn't fail on the compute during the claim
19:58:55 mriedem the claim should fail
20:00:02 dansmith allocation ratio?
20:00:23 dansmith he's not setting it so I would expect it's defaulting to 16 or whatever
20:00:24 dansmith for vcpu
20:00:34 mriedem default=0.0,
20:00:40 dansmith which means 16
20:00:48 mriedem yeah
20:00:48 mriedem and defaulted to 16.0.
20:00:48 mriedem set on the scheduler node(s) or compute node(s) will be used
20:00:48 mriedem if set to 0.0, the value
20:01:15 mriedem haha
20:01:47 mriedem i'll pull his test down, set that to 1.0 and make sure it fails the resize, and then if that's the case i'll be cool with the test
20:02:41 dansmith so, um
20:02:49 dansmith we really fall back to legacy scheduling in that case?
20:02:54 dansmith I don't think I knew that was the plan
20:03:33 openstackgerrit Jay Pipes proposed openstack/nova master: Remove provider allocs in confirm/revert resize https://review.openstack.org/488510
20:03:33 openstackgerrit Jay Pipes proposed openstack/nova master: placement: refactor healing of allocations in RT https://review.openstack.org/491850
20:03:34 openstackgerrit Jay Pipes proposed openstack/nova master: Resource tracker compatibility with Ocata and Pike https://review.openstack.org/491012
20:03:35 openstackgerrit Jay Pipes proposed openstack/nova master: remove log message with potential stale info https://review.openstack.org/492242
20:03:46 jaypipes mriedem, dansmith: ^.
20:03:51 mriedem well we wouldn't with gibi's patch
20:04:04 dansmith jaypipes: I'm not reviewing any more things today unless mriedem +2s them first
20:04:08 dansmith my stats and ego just can't take it
20:04:59 mriedem if it makes you feel better, maya told laura that she didn't want to go to camp this morning b/c she thought i'd forget her again
20:06:32 dansmith hah that's pretty awesome
20:06:57 jaypipes lol Dad of the Yea.
20:06:59 jaypipes Year
20:07:35 dansmith when camp called, I'd have said "oh uh yeah I "forgot" and stuff"
20:07:53 mriedem i can't remember if i swore when the front desk lady called me
20:07:58 mriedem "oh shit!"
20:08:12 mriedem "lemme put the bong down and i'll be right over"
20:08:37 mriedem hmm, setting cpu_allocation_ratio=1.0 in gibi's test doesn't make it fail
20:08:55 dims mriedem : ouch!
20:09:04 dansmith mriedem: nice
20:09:16 dansmith mriedem: set it before anything gets started?
20:09:24 mriedem yeah, in setUp
20:09:27 mriedem before setting the virt driver
20:09:51 dansmith mriedem: before compute or scheduler starts?
20:10:07 mriedem yes before everything starts
20:10:19 dansmith but wait,
20:10:23 dansmith before _everything_ starts? like all thethings?
20:10:26 mriedem http://paste.openstack.org/show/617983/
20:11:03 mriedem so you're just <1 from 4 day weekend and no longer caring, i see
20:11:04 dansmith no mriedem I mean BEFORE EVERYTHING
20:11:07 dansmith haha
20:11:13 mriedem *1 hour
20:11:42 dansmith don't worry, this four day weekend will be a special kind of hell where I'll be begging to work on unit tests by mid-day tomorrow
20:15:13 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Use ksa adapter for placement conf & requests https://review.openstack.org/492247
20:16:27 mriedem dansmith: i'm +2 on this https://review.openstack.org/#/c/492242/
20:19:59 mriedem ok bottom 2 are approved
20:26:32 mriedem jaypipes: easy one https://review.openstack.org/#/c/488510/

Earlier   Later