Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-02-07
10:23:25 rpodolyaka i.e. a sha1 hash
10:23:42 mdbooth rpodolyaka: Ah... so that's logging an sha1 of what went over the wire?
10:23:45 openstack Launchpad bug 1656276 in OpenStack Compute (nova) "Error running nova-manage cell_v2 simple_cell_setup when configuring nova with puppet-nova" [Undecided,In progress] - Assigned to Sylvain Bauza (sylvain-bauza)
10:23:45 bauzas markus_z: see the bug https://bugs.launchpad.net/nova/+bug/1656276
10:23:53 rpodolyaka mdbooth: yep
10:24:02 bauzas markus_z: that's the consensus now https://git.openstack.org/cgit/openstack/puppet-nova/commit/?id=dc2f3a358663c0fb97795666b16dec63cfcc3872
10:24:03 mdbooth Are you sure about that? Seems kinda weird we wouldn't just elide it.
10:24:42 mdbooth In other places we replace passwords with ***
10:25:08 rpodolyaka mdbooth: strace of `nova list` http://paste.openstack.org/raw/597914/
10:25:19 rpodolyaka well, the relevant part of it :)
10:25:49 mdbooth rpodolyaka: Cool, that doesn't look like an sha1 :)
10:25:53 mdbooth rpodolyaka: Thanks
10:25:56 rpodolyaka np!
10:27:14 bauzas markus_z: also http://docs.openstack.org/developer/nova/cells.html#first-time-setup now differs from http://docs.openstack.org/developer/nova/cells.html#upgrade-minimal
10:27:21 markus_z bauzas: We see that issue when we setup devstack (with devstack-gate). I'm not sure how that change applies to that situation.
10:28:24 openstackgerrit Bob Ball proposed openstack/nova master: XenAPI: fix memory usage miscalculation https://review.openstack.org/394857
10:28:40 bauzas markus_z: tbc, unless you're upgrading a newton cloud, you now need to stop using single_cell_setup and rather use map_cell0, then discover_hosts()
10:31:11 markus_z bauzas: Do I need to configure that in Devstacks local.conf? As said, it's for our CI.
10:34:46 bauzas markus_z: what do you mean by local.conf ?
10:35:03 bauzas markus_z: you don't really need to modify your workflow
10:35:20 bauzas if you're using devstack, it starts n-cpu because running simple_cell_setup AFAIK
10:35:31 bauzas which is then all good
10:35:49 bauzas the problem here is when you start n-cpu after running simple_cell_setup
10:36:09 bauzas then that call considers finding no hosts is an error
10:38:48 markus_z bauzas: To make sure I understand it: This call is the correct one https://github.com/openstack-dev/devstack/blob/bd14f7f23e01968aba70f0025b85dc15f110abc1/lib/nova#L952
10:38:51 markus_z ?
10:40:00 markus_z And then nova-manage "discover_hosts" is called? https://github.com/openstack/nova/blob/9eb7bcea05ce2447496f6561caa1ae46b800f85d/nova/cmd/manage.py#L1379-L1379
10:41:17 bauzas discover_hosts is when you add a compute *after* doing the cellsv2 setup
10:41:24 bauzas markus_z: ^
10:42:02 bauzas for the moment, there is no auto-registration of hosts in the cellsv2 world, but we need their mappings to order to place instances onto them
10:44:52 markus_z bauzas: I still don't get it :/ At the end of January there has to be a change which introduced something in that area which our CI isn't prepared to deal with.
10:46:51 markus_z bauzas: Do I understand it correctly that "n-cpu" has to run *before* simple_cell_setup is called?
10:47:42 bauzas markus_z: leave me 10 mins and we could hangout if you wish, okay
10:47:43 bauzas ?
10:49:11 markus_z bauzas: Let me dig more into the issue on my side before I'm consuming you time, but thanks a lot for the offer.
10:56:11 johnthetubaguy markus_z: did you see the new docs that just merged by the way
10:56:20 johnthetubaguy markus_z: they explain things a lot better
10:57:02 johnthetubaguy markus_z: the thing that confused me is that "db sync" created cell0, only after you first did the create_cell0 thing, it could be a different thing you hit
10:57:28 mdbooth lyarwood: Hey, so detach_volume races with delete()
10:57:42 tdurakov johnthetubaguy: hi, which docs?
10:58:21 mdbooth i.e. if you detach_volume(), delete() really quickly, the delete will do the detach which is already in progress
10:58:28 johnthetubaguy tdurakov: https://github.com/openstack/nova/commit/da591bc543c84fb601e5d832ba6a28e38c036901
10:58:36 mdbooth lyarwood: Can you think of any bad things this might cause on the cinder side?
10:58:40 tdurakov previous time I've deployed multinode devstack, I need to run nova-manage cell_v2 simple_cell_setup after second compute is deployed
10:58:43 mdbooth johnthetubaguy: Same question ^^^^ ?
10:59:02 tdurakov today I've deployed new one, and it doesn't recognise second compute is added
10:59:11 tdurakov thanks for the link!
10:59:30 lyarwood mdbooth: about to jump on a call
10:59:34 johnthetubaguy tdurakov: markus_z: there is an ordering issue I suspect. You need to have started n-cpu to be able to discover it
10:59:37 mdbooth lyarwood: k
10:59:44 lyarwood mdbooth: I can't think of anything bad on the cinder cide
10:59:58 lyarwood mdbooth: but that is going to screw up host state and leave things hanging around with iSCSI/FC etc
11:00:18 lyarwood mdbooth: ah no wait we have locks around disconnect_volume
11:00:19 mdbooth Well it's going do to do 2 detaches
11:00:20 tdurakov johnthetubaguy: hm... I'm not sure how this should work
11:00:26 mdbooth So it's that going to extra cleanup?
11:00:27 johnthetubaguy mdbooth: lyarwood: unsure on cinder, I suspect its OK
11:00:49 lyarwood mdbooth: double terminate_connection calls should be fine as long as something has disconnected the volume from the host
11:00:49 tdurakov local.conf on compute only node already has endpoints of controller node
11:01:03 tdurakov so how I should reorder them properly
11:01:13 mdbooth lyarwood: Thanks
11:01:13 johnthetubaguy tdurakov: like I say, the docs do try cover that now, including the nova-manage man pages: https://github.com/openstack/nova/blob/5dad7eaef7f8562425cce6b233aed610ca2d3148/doc/source/man/nova-manage.rst
11:01:28 johnthetubaguy tdurakov: I would talk you through it, but I would like to test the docs :)
11:01:46 tdurakov johnthetubaguy: that's ok
11:01:50 tdurakov thank you
11:02:08 johnthetubaguy tdurakov: now worries, this is the rendered docs: http://docs.openstack.org/developer/nova/cells.html#step-by-step-for-common-use-cases
11:02:23 johnthetubaguy markus_z: http://docs.openstack.org/developer/nova/cells.html#step-by-step-for-common-use-cases and https://github.com/openstack/nova/blob/5dad7eaef7f8562425cce6b233aed610ca2d3148/doc/source/man/nova-manage.rst should be your friend
11:03:44 mdbooth Ah... attach/delete would be worse, which is probably why that takes an instance lock
11:04:47 johnthetubaguy mdbooth: I am wondering about the end API state vs what we have on the host, its not clear what we end up with
11:42:33 openstackgerrit Matthew Booth proposed openstack/nova master: Fix various races with detach_volume https://review.openstack.org/430213
11:42:56 openstackgerrit Sergey Nikitin proposed openstack/nova master: TrivialFix: replace list comprehension with 'for' https://review.openstack.org/430214
11:51:53 markus_z johnthetubaguy: thanks, I haven't yet read the new docs, I'll do that today.
11:52:38 johnthetubaguy markus_z: no worries, hopefully it should help
11:52:48 johnthetubaguy markus_z: if not we should fix the docs :)
11:58:39 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Remove stuck live-migration detection https://review.openstack.org/430218
12:00:26 johnthetubaguy pkoniszewski: kick started the discussion around that bug in the above patch^
12:01:22 mdbooth johnthetubaguy: Haven't looked at what it does yet, but +87, -163 looks good :)
12:01:33 johnthetubaguy mdbooth: heh :)
12:26:22 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Remove stuck live-migration detection https://review.openstack.org/430218
12:28:57 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Remove stuck live-migration detection https://review.openstack.org/430218
12:40:56 BobBall_AWOL johnthetubaguy: Could you re-approve https://review.openstack.org/#/c/424428/ pls? :)
12:43:33 openstackgerrit Vladik Romanovsky proposed openstack/nova master: neutron: retrieve physical netwrok name from a multi-provider network https://review.openstack.org/430026
12:44:36 mdbooth vladikr: nit: you have a typo in the title
12:46:34 vladikr mdbooth, ah, thanks! :)
12:47:06 mdbooth ;)
12:48:42 openstackgerrit Vladik Romanovsky proposed openstack/nova master: neutron: retrieve physical network name from a multi-provider network https://review.openstack.org/430026
12:53:53 gcb sfinucan, can you remove your -2 now ? https://review.openstack.org/#/c/323313/
13:02:28 sfinucan gcb: Sure
13:03:11 sfinucan gcb: Actually, per johnthetubaguy's comments, perhaps we should wait until next week to merge that?
13:03:32 gcb sfinucan, agree
13:04:14 sfinucan gcb: So the reason changes but the -2 remains until then :)
13:04:31 gcb sfinucan, okay
13:04:34 gcb :)
13:05:16 sfinucan but ping me next week and I'll re-review
13:28:32 openstackgerrit Sivasathurappan Radhakrishnan proposed openstack/nova master: Initiate network changes from source during live migration https://review.openstack.org/404607
13:35:19 openstackgerrit Matthew Booth proposed openstack/nova master: Fix various races with detach_volume https://review.openstack.org/430213
13:36:01 pumaranikar johnthetubaguy, cdent can you please check https://review.openstack.org/#/c/395194/ . Placement json formatter change
13:39:11 mdbooth lyarwood: A festival of obscure syntax: https://github.com/mdbooth/openstack-dev-hacks/commit/1e13942134c298dd27af674fdcaf7a5ac72b1e82 :)
13:39:17 johnthetubaguy pumaranikar: whats the plan for all the other handlers?
13:40:00 mdbooth To top it off, here's how I just called it: ci_log_fetch http://logs.openstack.org/09/395709/6/check/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/f829785/logs/ {c-{api,vol},n-{api,cond,cpu}}

Earlier   Later