Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-30
14:21:34 lbragstad jaypipes sounds good
14:21:43 lbragstad i'll be around
14:32:04 fghaas Hi kashyap, assuming you're back and recovered from Edinburgh, could I bother you to take another peek at https://review.openstack.org/#/c/609788/ ?
14:32:19 kashyap fghaas: Hi, I'm also recovering from "other things", will look. :-)
14:32:34 gibi mriedem: thanks for the review.
14:32:47 fghaas Perfect. No rush, just wanted to make sure it didn't fall off the table. :)
14:33:39 kashyap fghaas: Ah, you've answered the questions. And have added live migration sections
14:35:04 johnthetubaguy jaypipes: are you planning work on placement quotas, just saw your tidy up patches
14:35:50 johnthetubaguy so I should have read the scrollback
14:36:10 johnthetubaguy I was planning on looking at this stuff ASAP: https://review.openstack.org/#/c/602201
14:39:12 johnthetubaguy jaypipes: I see your thread now, quota classes I think come from some out of tree rackspace idea that never made it downstream.
14:40:01 johnthetubaguy jaypipes: the idea was a project was assigned a class, if not assigned you fallback to the default. i.e. you could upgrade someone to the I_HAVE_BIG_CREDIT_LIMIT quota class, etc. A bit like flavors.
14:40:25 gaudenz Hi
14:40:51 mriedem gibi: a few nits and questions in https://review.openstack.org/#/c/567267/
14:41:37 sean-k-mooney johnthetubaguy: is that related to the idea of qoutas for flavors and generic placement resouce class quotas
14:41:55 johnthetubaguy sadly no, that's different
14:42:17 johnthetubaguy this is more a groups of limits, rather than limits on flavors
14:42:18 sean-k-mooney oh ok i like that usecase but never herard of quota classes until now
14:42:33 johnthetubaguy its half lurking in the code, and was never finished
14:42:39 johnthetubaguy best ignored
14:42:47 openstackgerrit Jack Ding proposed openstack/nova-specs master: High Precision Event Timer (HPET) on x86 guests https://review.openstack.org/607989
14:43:00 sean-k-mooney ok ill continue to do so
14:43:00 jaypipes johnthetubaguy: or you could just have quota templates internally and assign a new project the values of that quota template... quota classes never needed to be in the API.
14:43:33 johnthetubaguy jaypipes: yeah, totally, that is the current plan for unified limits. its crazy pants stuff
14:43:40 jaypipes in a similar way that "registered quotas" in the new keystone limits API doesn't need to be exposed to users at all.
14:43:54 jaypipes johnthetubaguy: ++
14:46:06 johnthetubaguy jaypipes: did you get chance to look at the unified limits spec I am looking at working on with alex_xu?
14:48:01 mriedem bauzas: do you remember this? https://review.openstack.org/#/c/582417/1/nova/conductor/tasks/migrate.py@a287
14:48:31 mriedem because i'm having a hell of a time figuring out why i can't pass a RequestSpec object to the compute prep_resize RPC API but we can pass request spec to other things like select_destinations and build_and_run_instance
14:49:28 jaypipes johnthetubaguy: in my queue.
14:49:57 johnthetubaguy jaypipes: cool, just making sure to avoid duplicate work
14:55:30 kashyap fghaas: One latest remark in the change; regardless, I think it can be merged now. And can fine-tune in further patches.
14:59:43 bauzas mriedem: looking
15:00:01 bauzas currently on an internal priority bug hence my absence today
15:01:37 bauzas mriedem: oh I remember the tz issue, but I don't remember how I fixed it
15:01:54 bauzas mriedem: that's only a test issue, right?
15:03:12 mriedem it's not fixed
15:03:25 mriedem there was an oslo.messaging bug for it against the fake driver, but was invalidated
15:03:52 mriedem i'm just not sure why the prep_resize cast is a problem, but passing request spec to other rpc methods isn't a problem
15:06:12 melwitt efried: sure
15:06:19 efried thanks!
15:07:01 bauzas mriedem: I think I fixed this using the call_for_cast for my tests
15:07:06 bauzas IIRC
15:09:14 mriedem fghaas: a couple of questions in https://review.openstack.org/#/c/609788/2
15:10:01 wangy Is there anyone can help me？I used rally to boot server with concurrency 50，
15:10:02 wangy I got error {u'message': u'No valid host was found. There are not enough hosts available.', u'code': 500,} My platform version is stable/pike, and nova-placement-api.log showed “DB error: (pymysql.err.InternalError) (1213, u'Deadlock found when trying to get lock;”
15:12:26 mriedem wangy: you might need https://github.com/openstack/nova/commit/c66ae65775bb9d885fac059847063fee70617bc5#diff-a040eef16b56cc7862fb2a334c757dea
15:12:38 mriedem since you didn't mention where the deadlock is occurring, it's hard to say
15:12:46 mriedem there was also https://github.com/openstack/nova/commit/e001b0538a9d413f864d39584c2aea0b9e200e4b#diff-a040eef16b56cc7862fb2a334c757dea but that was fixed in pike
15:13:27 wangy The db deadlock was caused by "INSERT INTO allocations "
15:14:28 fghaas mriedem, kashyap: all good points, thanks — will push update momentarily
15:15:01 wangy mriedem: thanks,I had added retry decorator, it did not help
15:15:52 mriedem wangy: seems that would be covered with https://github.com/openstack/nova/commit/e001b0538a9d413f864d39584c2aea0b9e200e4b#diff-a040eef16b56cc7862fb2a334c757dea
15:16:42 mriedem do you see "Performing DB retry for function" in the placement api logs?
15:16:50 mriedem if so, that means it's retrying on deadlock
15:17:03 mriedem if you see "DB exceeded retry limit" that means it hit the retry limit
15:17:24 mriedem and probably means you need more resources / more placement workers in your env
15:18:10 wangy mriedem: yes, I see the logs, thanks for your opinion, I will try it.
15:19:33 openstackgerrit Theodoros Tsioutsias proposed openstack/nova-specs master: Enable rebuild for instances in cell0 https://review.openstack.org/554218
15:21:40 Shilpa mriedem: HI.
15:21:56 Shilpa mriedem:Can you please help me to understand in case of nova evacuate, why we are restoing allocations at source?
15:22:34 Shilpa is it for rollback?
15:23:04 Shilpa or in case of evacuate, in case of success, how allocaions at source are cleared off? Is any process set for it?
15:23:48 mriedem i'm not sure what you mean by "restoring allocations at source"
15:24:16 mriedem if the evacuated source nova-compute service starts up again and instances have been successfully migrated from it, the allocations are cleaned up for the source node during that startup
15:24:46 mriedem here https://github.com/openstack/nova/blob/d1e38f06bb5282dd9bf81d979926a0ebbfd93caa/nova/compute/manager.py#L712
15:25:43 mriedem if the evacuation fails, the source node allocations should still exist in placement
15:26:19 Shilpa mriedem: yes in case of failure, observe source node allocations are still there in placement
15:26:38 mriedem if the evacuation fails, the destination node allocations, which are created by the scheduler, should be cleaned up here https://github.com/openstack/nova/blob/d1e38f06bb5282dd9bf81d979926a0ebbfd93caa/nova/compute/manager.py#L3044 or here https://github.com/openstack/nova/blob/d1e38f06bb5282dd9bf81d979926a0ebbfd93caa/nova/compute/manager.py#L3058
15:26:55 Shilpa mriedem: will check for evacuated source nova-compute service starts up again, and what happens there
15:26:59 gibi mriedem: thanks for the review on https://review.openstack.org/#/c/567267/ I have to get back to these patches next week. Now I'm in a heavy preparation for our bandwidth demo for the summit
15:27:30 openstackgerrit sean mooney proposed openstack/nova master: harden placement init under wsgi https://review.openstack.org/610034
15:28:04 mriedem gibi: ok, the changes don't seem very difficult for either change, would be ok with me just addressing my comments if you don't get to them first? so we can keep moving that series?
15:28:16 mriedem *would you be ok
15:29:23 mriedem tssurya: i'm not sure how caught up your are on the changes dansmith made to the handling a down cell series, but i've got a slew of comments in the majority of the bottom changes - are you going to address those this week? if not, i might start working on that as well since i think dansmith is busy
15:30:30 openstackgerrit sean mooney proposed openstack/nova master: harden placement init under wsgi https://review.openstack.org/610034
15:30:43 Shilpa mriedem: checked that if evacuated source nova-compute service starts up again and instances have been successfully migrated from it, the allocations are cleaned up for the source node during that startup
15:31:03 Shilpa mriedem: thanks for this explaination
15:32:53 mriedem Shilpa: yw
15:33:04 bauzas mriedem: oops sorry I meant cast_as_calls
15:33:22 bauzas this ^ fixed my tz issue for the functional tests
15:33:26 sean-k-mooney Shilpa: note that technicall the instanaces are not migrated. they are recreaed during evacuate
15:33:36 mriedem bauzas: you mean the CastAsCallFixture right?
15:33:42 bauzas yup
15:33:42 openstackgerrit Matt Riedemann proposed openstack/nova master: Drop request spec migration code https://review.openstack.org/614265
15:33:46 mriedem either way, i don't want to use that fixture if it can be helped
15:33:47 sean-k-mooney Shilpa: but if the source comes back up it will fix the allcoations
15:33:51 mriedem i'm also not sure why that would make a difference
15:34:08 mriedem i suppose because it stubs out the prepare method in the rpc client so we don't hit the validation in the fake rpc driver
15:34:21 bauzas mriedem: I didn't really had an issue when testing live, so I didn't went further down
15:34:43 dansmith mriedem: just saw your comment on the cells fixture thing
15:34:43 bauzas mriedem: yeah what you say rings a bell to me
15:34:45 bauzas but that's one
15:34:49 bauzas old*
15:34:52 Shilpa sean-k-mooney: thanks, understood
15:34:52 mriedem i know we have functional tests that don't use the CastAsCallFixture which still pass a request spec to select_destinations and build_and_run_instance
15:34:54 mriedem so i'm still confused
15:35:00 dansmith mriedem: if you hit a problem with that, there should be a runtimeerror raised
15:35:31 bauzas mriedem: FWIW, I'm disecting some internal bug for a customer that sounds pretty similar to https://bugs.launchpad.net/nova/+bug/1600251
15:35:32 openstack Launchpad bug 1600251 in OpenStack Compute (nova) "live migration does not honor server group policy" [High,Fix released]

Earlier   Later