Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-04-20
15:43:24 mriedem dansmith: where does this handle/log thing go? a migration? an object?
15:43:28 dansmith mriedem: well, I wasn't going to be that nice, I was going to say "Instance Extra (uuid) has no Instance, unable to migrate"
15:43:32 mdbooth imacdonn: It waits for the rest call to return an http response
15:43:35 imacdonn mdbooth lyarwood I know this quite well, because it times out sometimes ;)
15:43:43 dansmith mriedem: in the online migration function that nova-manage is calling
15:43:50 mdbooth But the http resonse doesn't (according to the docs) indicate that it has completed
15:44:06 mriedem dansmith: as you like
15:44:10 mriedem i won't block it
15:44:17 dansmith mriedem: well, I'll put it up and you can decide then
15:44:33 mriedem geddy lee told me that if i choose not to decide i've still made a choice
15:44:35 Tahvok I'm installing nova on ubuntu, and according to the guide I need to create the cell1 cell. In the command they say I need to run it with some sort of id? 109e1d4b-536a-40d0-83c6-5f121b82b650
15:44:37 imacdonn mdbooth lyarwood empirically, I can say that it does wait for the operation to complete .... how else would the status be returned ??
15:44:45 Tahvok What is that id? https://docs.openstack.org/ocata/install-guide-ubuntu/nova-controller-install.html
15:44:48 dansmith mriedem: heh
15:44:51 mdbooth imacdonn: It returns 202 ACCEPTED
15:45:13 mriedem Tahvok: which command? nova-manage cell_v2 create_cell?
15:45:22 Tahvok mriedem: yes
15:45:28 mriedem su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova
15:45:36 mriedem that doesn't say anything about providing an id
15:45:39 Tahvok Oh, it's the output
15:45:41 mriedem it outputs an id
15:45:42 mriedem yes
15:45:50 Tahvok Yes, I thought it's part of the command
15:45:55 Tahvok mriedem: thanks!
15:46:16 imacdonn mdbooth: the ZFSSA REST API call ??
15:46:18 mriedem Tahvok: also https://docs.openstack.org/developer/nova/man/nova-manage.html#nova-cells-v2 in case you need more info on the commands
15:46:21 lyarwood mdbooth: sorry back, isn't this a bug within this and any other driver that doesn't terminate_connection synchronously then?
15:46:23 mdbooth imacdonn: Yeah
15:46:38 lyarwood mdbooth: I skimmed over a few target drivers and they appeared to be sync
15:46:39 mdbooth lyarwood: Right, we're discussing zfssa, but other drivers will be different
15:46:54 Tahvok mriedem: is that create_cell command idempotent?
15:47:10 Tahvok Could I run it again, or it will try creating another cell?
15:47:10 imacdonn mdbooth: that doesn't sound right ... looking for a debug log to verify
15:47:12 lyarwood mdbooth: yeah, I think there's still value to adding the lock on our side if we also follow up with cinder to get all drivers in line there
15:48:02 openstack Launchpad bug 1511466 in OpenStack Compute (nova) "migrate flavor data impossible" [Medium,Confirmed]
15:48:02 zioproto dansmith: the bug is still open https://bugs.launchpad.net/nova/+bug/1511466 SOOO I guess any operator with a big deployment that managed to upgrade to liberty probably used this workaround. This means that soon this problem will explode :D
15:48:05 mriedem Tahvok: it's not
15:48:11 mdbooth imacdonn: That's definitely what it returns, and the documented semantics of that are that it's an async call which may not have completed. That doesn't necessarily mean that's what it's doing, of course.
15:48:11 mriedem Tahvok: it would fail with a duplicate entry error
15:48:27 dansmith zioproto: yeah, you know what you need to do to fix your data right?
15:48:33 mriedem Tahvok: "Returns 0 if the cell mapping was successfully created, 1 if the transport url or database connection was missing, and 2 if a cell is already using that transport url and database connection combination."
15:48:38 Tahvok mriedem: that's fine as long as it does not create any new records
15:48:50 mriedem Tahvok: $? would be 2
15:48:57 Tahvok mriedem: thanks a lot!
15:49:02 imacdonn mdbooth: heh, OK ... I'm fairly sure there's nothing in the ZFSSA driver that gets a result of the operation by any means other than the response to the REST request
15:49:13 zioproto dansmith: so you mean that if I delete from the instance_extra table all the records where the uuid is not in instances, then I dont need any patch ?
15:50:04 mdbooth imacdonn: https://docs.oracle.com/cd/E51475_01/html/E52433/makehtml-id-181.html
15:50:24 mdbooth Except it's doing a PUT, not a POST
15:50:42 dansmith zioproto: yeah, that's my expectation.. let me know if it doesn't and we can see if something else is wrong too
15:50:42 zioproto dansmith: I have to rush away from the office... I will ping you tomorrow about this
15:50:46 dansmith zioproto: okay thanks
15:51:01 imacdonn mdbooth: Yeah, so does "successful response" mean "I've completed it"? In practice I believe it does .. so not sure why they're using 202
15:51:58 mdbooth Meh, I wonder if it's reasonable to assert this is sync and make it the cinder driver's problem if it isn't
15:52:03 mdbooth lyarwood: ^^^ ?
15:52:21 imacdonn mdbooth: in any case, I think that nova-compute has to assume that the cinder API call won't return until the work as completed .. if that's not happening, it's either a bug in cinder, or a large other can of worms (need an async mechanism between nova and cinder
15:52:32 lyarwood mdbooth: yeah that's my view
15:52:49 mdbooth lyarwood: Ok
15:53:09 lyarwood mdbooth: but we should really make that assumption known to the cinder folks
15:53:17 lyarwood mdbooth: using this driver as an example
15:53:19 mdbooth Yep
15:53:36 openstackgerrit Dan Smith proposed openstack/nova master: Warn the user about orphaned extra records during keypair migration https://review.openstack.org/458564
15:53:43 dansmith mriedem: this is what I was thinking ^
15:54:04 dansmith mriedem: I'll mark the bug as invalid and also comment on the original bug warning people that they eff'd themselves
15:54:45 zioproto dansmith: with this patch https://review.openstack.org/#/c/458564 I should be able to go on without cleaning the db, right ?
15:54:54 zioproto it is the patch I applied manually, right?
15:55:10 dansmith zioproto: no
15:55:14 dansmith zioproto: it's just a warning
15:55:30 dansmith zioproto: it won't fix the problem, which will make the second keypair migration unable to proceed
15:55:34 dansmith zioproto: you have to clean the database
15:56:04 sfinucan vladikr, sahid: Little cleanup here, if you could take a gawk https://review.openstack.org/#/c/417378/
15:57:13 vladikr sfinucan, sure
15:58:54 openstackgerrit Matt Riedemann proposed openstack/nova master: Deprecate os-hosts API https://review.openstack.org/456504
15:59:01 mriedem alex_xu: ^ fixed that unit test
16:00:38 mdbooth lyarwood: My patch won't fix this, btw, because it needs to be global: https://review.openstack.org/#/c/430213/2/nova/compute/manager.py
16:01:53 mdbooth lyarwood: When using iscsi, I think all connect and disconnect calls must be serialised, where disconnect also include terminate connection
16:02:34 lyarwood mdbooth: global in terms of the compute node right?
16:02:40 mdbooth lyarwood: Yes
16:02:57 lyarwood mdbooth: I assumed the manager was, maybe I'm missing something
16:03:10 mdbooth lyarwood: The lock has to be global
16:03:22 mdbooth In my patch it's not, it's keyed on instance uuid
16:03:26 imacdonn lyarwood: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2261
16:04:11 mdbooth i.e. No 2 connect/disconnect operations affecting iscsi can occur concurrently, regardless of target or instance
16:04:50 mdbooth But that's not true of other volume drivers, which is evil
16:06:32 mdbooth So unless we want to push this restriction onto all volume backends, terminate connection would have to be managed by os-brick
16:07:10 mdbooth lyarwood johnthetubaguy: ^^^
16:07:34 lyarwood mdbooth: yeah sorry the cinderv3 meeting just started, I see what you mean know re global
16:07:39 lyarwood now*
16:08:18 sdague so, given the issues with https://bugs.launchpad.net/nova/+bug/1511466, how about bump to critical, and provide some better guidance to get through it
16:08:18 openstack Launchpad bug 1511466 in OpenStack Compute (nova) "migrate flavor data impossible" [Critical,Confirmed]
16:08:36 sdague because it does seem like it's coming up on google hunts :)
16:09:47 openstackgerrit Eric Fried proposed openstack/nova master: PowerVM Driver: spawn/destroy #2: functional https://review.openstack.org/438598
16:10:30 efried sdague mriedem ^^ Thanks for the review, back atcha.
16:11:06 openstackgerrit Chris Dent proposed openstack/nova master: Update resource tracker to PUT custom resource classes https://review.openstack.org/456915
16:11:07 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Idempotent PUT /resource_classes/{name} https://review.openstack.org/448791
16:15:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Using max api version in notification sample test https://review.openstack.org/443686
16:17:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Using max api version in notification sample test https://review.openstack.org/443686
16:22:52 openstackgerrit Eric Fried proposed openstack/nova master: PowerVM Driver: power_on/off and reboot https://review.openstack.org/427380
16:24:20 openstackgerrit Balazs Gibizer proposed openstack/nova master: Using max api version in notification sample test https://review.openstack.org/443686
16:24:21 openstackgerrit Balazs Gibizer proposed openstack/nova master: add tags field to instance.update notification https://review.openstack.org/407228
16:26:14 gibi mriedem: I update the patch that adds tags to the instance.update notification as we agreed https://review.openstack.org/#/c/407228/

Earlier   Later