Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-02-07
10:44:52 markus_z bauzas: I still don't get it :/ At the end of January there has to be a change which introduced something in that area which our CI isn't prepared to deal with.
10:46:51 markus_z bauzas: Do I understand it correctly that "n-cpu" has to run *before* simple_cell_setup is called?
10:47:42 bauzas markus_z: leave me 10 mins and we could hangout if you wish, okay
10:47:43 bauzas ?
10:49:11 markus_z bauzas: Let me dig more into the issue on my side before I'm consuming you time, but thanks a lot for the offer.
10:56:11 johnthetubaguy markus_z: did you see the new docs that just merged by the way
10:56:20 johnthetubaguy markus_z: they explain things a lot better
10:57:02 johnthetubaguy markus_z: the thing that confused me is that "db sync" created cell0, only after you first did the create_cell0 thing, it could be a different thing you hit
10:57:28 mdbooth lyarwood: Hey, so detach_volume races with delete()
10:57:42 tdurakov johnthetubaguy: hi, which docs?
10:58:21 mdbooth i.e. if you detach_volume(), delete() really quickly, the delete will do the detach which is already in progress
10:58:28 johnthetubaguy tdurakov: https://github.com/openstack/nova/commit/da591bc543c84fb601e5d832ba6a28e38c036901
10:58:36 mdbooth lyarwood: Can you think of any bad things this might cause on the cinder side?
10:58:40 tdurakov previous time I've deployed multinode devstack, I need to run nova-manage cell_v2 simple_cell_setup after second compute is deployed
10:58:43 mdbooth johnthetubaguy: Same question ^^^^ ?
10:59:02 tdurakov today I've deployed new one, and it doesn't recognise second compute is added
10:59:11 tdurakov thanks for the link!
10:59:30 lyarwood mdbooth: about to jump on a call
10:59:34 johnthetubaguy tdurakov: markus_z: there is an ordering issue I suspect. You need to have started n-cpu to be able to discover it
10:59:37 mdbooth lyarwood: k
10:59:44 lyarwood mdbooth: I can't think of anything bad on the cinder cide
10:59:58 lyarwood mdbooth: but that is going to screw up host state and leave things hanging around with iSCSI/FC etc
11:00:18 lyarwood mdbooth: ah no wait we have locks around disconnect_volume
11:00:19 mdbooth Well it's going do to do 2 detaches
11:00:20 tdurakov johnthetubaguy: hm... I'm not sure how this should work
11:00:26 mdbooth So it's that going to extra cleanup?
11:00:27 johnthetubaguy mdbooth: lyarwood: unsure on cinder, I suspect its OK
11:00:49 tdurakov local.conf on compute only node already has endpoints of controller node
11:00:49 lyarwood mdbooth: double terminate_connection calls should be fine as long as something has disconnected the volume from the host
11:01:03 tdurakov so how I should reorder them properly
11:01:13 johnthetubaguy tdurakov: like I say, the docs do try cover that now, including the nova-manage man pages: https://github.com/openstack/nova/blob/5dad7eaef7f8562425cce6b233aed610ca2d3148/doc/source/man/nova-manage.rst
11:01:13 mdbooth lyarwood: Thanks
11:01:28 johnthetubaguy tdurakov: I would talk you through it, but I would like to test the docs :)
11:01:46 tdurakov johnthetubaguy: that's ok
11:01:50 tdurakov thank you
11:02:08 johnthetubaguy tdurakov: now worries, this is the rendered docs: http://docs.openstack.org/developer/nova/cells.html#step-by-step-for-common-use-cases
11:02:23 johnthetubaguy markus_z: http://docs.openstack.org/developer/nova/cells.html#step-by-step-for-common-use-cases and https://github.com/openstack/nova/blob/5dad7eaef7f8562425cce6b233aed610ca2d3148/doc/source/man/nova-manage.rst should be your friend
11:03:44 mdbooth Ah... attach/delete would be worse, which is probably why that takes an instance lock
11:04:47 johnthetubaguy mdbooth: I am wondering about the end API state vs what we have on the host, its not clear what we end up with
11:42:33 openstackgerrit Matthew Booth proposed openstack/nova master: Fix various races with detach_volume https://review.openstack.org/430213
11:42:56 openstackgerrit Sergey Nikitin proposed openstack/nova master: TrivialFix: replace list comprehension with 'for' https://review.openstack.org/430214
11:51:53 markus_z johnthetubaguy: thanks, I haven't yet read the new docs, I'll do that today.
11:52:38 johnthetubaguy markus_z: no worries, hopefully it should help
11:52:48 johnthetubaguy markus_z: if not we should fix the docs :)
11:58:39 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Remove stuck live-migration detection https://review.openstack.org/430218
12:00:26 johnthetubaguy pkoniszewski: kick started the discussion around that bug in the above patch^
12:01:22 mdbooth johnthetubaguy: Haven't looked at what it does yet, but +87, -163 looks good :)
12:01:33 johnthetubaguy mdbooth: heh :)
12:26:22 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Remove stuck live-migration detection https://review.openstack.org/430218
12:28:57 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Remove stuck live-migration detection https://review.openstack.org/430218
12:40:56 BobBall_AWOL johnthetubaguy: Could you re-approve https://review.openstack.org/#/c/424428/ pls? :)
12:43:33 openstackgerrit Vladik Romanovsky proposed openstack/nova master: neutron: retrieve physical netwrok name from a multi-provider network https://review.openstack.org/430026
12:44:36 mdbooth vladikr: nit: you have a typo in the title
12:46:34 vladikr mdbooth, ah, thanks! :)
12:47:06 mdbooth ;)
12:48:42 openstackgerrit Vladik Romanovsky proposed openstack/nova master: neutron: retrieve physical network name from a multi-provider network https://review.openstack.org/430026
12:53:53 gcb sfinucan, can you remove your -2 now ? https://review.openstack.org/#/c/323313/
13:02:28 sfinucan gcb: Sure
13:03:11 sfinucan gcb: Actually, per johnthetubaguy's comments, perhaps we should wait until next week to merge that?
13:03:32 gcb sfinucan, agree
13:04:14 sfinucan gcb: So the reason changes but the -2 remains until then :)
13:04:31 gcb sfinucan, okay
13:04:34 gcb :)
13:05:16 sfinucan but ping me next week and I'll re-review
13:28:32 openstackgerrit Sivasathurappan Radhakrishnan proposed openstack/nova master: Initiate network changes from source during live migration https://review.openstack.org/404607
13:35:19 openstackgerrit Matthew Booth proposed openstack/nova master: Fix various races with detach_volume https://review.openstack.org/430213
13:36:01 pumaranikar johnthetubaguy, cdent can you please check https://review.openstack.org/#/c/395194/ . Placement json formatter change
13:39:11 mdbooth lyarwood: A festival of obscure syntax: https://github.com/mdbooth/openstack-dev-hacks/commit/1e13942134c298dd27af674fdcaf7a5ac72b1e82 :)
13:39:17 johnthetubaguy pumaranikar: whats the plan for all the other handlers?
13:40:00 mdbooth To top it off, here's how I just called it: ci_log_fetch http://logs.openstack.org/09/395709/6/check/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/f829785/logs/ {c-{api,vol},n-{api,cond,cpu}}
13:40:03 pumaranikar johnthetubaguy, will be addressing in follow up patches
13:43:46 johnthetubaguy pumaranikar: that LOG.error in your handler, did the original bug want that added?
13:43:59 tdurakov pkoniszewski: hi, are you around?
13:44:37 pumaranikar johnthetubaguy, it was not a requirement to add logging here
13:45:33 johnthetubaguy pumaranikar: have you seen this, its worth a read: https://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html
13:47:46 pkoniszewski tdurakov: yes
13:48:26 lyarwood mdbooth: nice!
13:48:36 tdurakov pkoniszewski: there are some intersections in my schedule today, could you chair l-m meeting?
13:48:50 pkoniszewski i'm afraid i need to skip the meeting today
13:48:57 pkoniszewski leaving office in 10-15 minutes
13:49:59 tdurakov pkoniszewski: ok
13:51:40 pumaranikar johnthetubaguy, do you think error log level is not appropriate here ?
13:51:58 johnthetubaguy pumaranikar: yeah, added comments
13:52:02 tdurakov mdbooth: hi, are you ok chairing lm meeting today?
13:52:11 johnthetubaguy pumaranikar: "Error: Serious issue with cloud, administrator should be notified immediately via email/pager. On call people expected to respond."
13:52:25 mdbooth tdurakov: Not really, I'm totally not on top of it
13:52:38 johnthetubaguy pumaranikar: sometimes these errors raised by placement are almost expected, like not found error when you check something is deleted, etc
13:52:42 mdbooth tdurakov: If you'd asked me yesterday, maybe :)
13:53:35 pumaranikar johnthetubaguy, yes. I thought of error level logging since it is in exception block
13:53:43 tdurakov paul-carlton2 eliqiao shaohe_feng pkoniszewski yuntongjin kashyap kmartin claudiub tobasco abhishekk mrhillsman raj_singh wznoinsk siva_krishnan, folks, let's skip the l-m meeting this week.
13:53:59 johnthetubaguy pumaranikar: yeah, need to think about the consumer of the message, it doesn't rally help them in this case
13:54:20 johnthetubaguy pumaranikar: but a debug log might be a good idea, if we include information on the specific exception
13:55:03 johnthetubaguy tdurakov: did you see this: https://review.openstack.org/#/c/429798/
13:55:14 johnthetubaguy tdurakov: trying to get it into RC2, if possible
13:56:19 pumaranikar johnthetubaguy, ok. i will have look at your comment and make change accordingly
13:56:55 tdurakov johnthetubaguy: I'll review it in 1.5 hours, or earlier
13:57:54 johnthetubaguy tdurakov: I was wondering if the folks in the meeting agree if we should try get that into RC2 I guess
13:59:32 tdurakov johnthetubaguy: btw, have you seen this one https://review.openstack.org/#/c/408002
14:00:01 johnthetubaguy tdurakov: yeah, I need to look at that one again
14:00:23 johnthetubaguy tdurakov: thats a new option though, not suitable past feature freeze, but good for pike

Earlier   Later