| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-02 | |||
| 19:26:05 | mriedem | maybe that predates the old_flavor being stored on the instance? | |
| 19:26:08 | dansmith | no, | |
| 19:26:13 | dansmith | we set it to None before we call this here: | |
| 19:26:31 | dansmith | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3501-L3521 | |
| 19:26:34 | dansmith | son of a goat | |
| 19:26:41 | mriedem | is it stashed on the migration record? | |
| 19:26:46 | dansmith | no, only the id | |
| 19:26:55 | mriedem | # NOTE(danms): delete stashed migration information | |
| 19:26:55 | mriedem | ha | |
| 19:27:02 | mriedem | BUT WHY?! | |
| 19:27:02 | dansmith | wait | |
| 19:27:20 | dansmith | we pass old_instance_type for instance_type, | |
| 19:27:27 | dansmith | but clearly that can be None sometimes | |
| 19:27:30 | dansmith | but I don't know when | |
| 19:27:36 | dansmith | maybe if we're doing a migrate? | |
| 19:27:43 | mriedem | probably yeah | |
| 19:27:47 | mriedem | no old/new if it's not a resize | |
| 19:27:48 | dansmith | so maybe I can use old_instance_type or instance.flavor | |
| 19:27:54 | mriedem | f yes you can | |
| 19:30:53 | mriedem | dfisher: if you're using a recent devstack, it's running in superconductor mode which means you can't do retries from the compute | |
| 19:31:00 | mriedem | it can't upcall to the api db to get instance group info | |
| 19:31:16 | mriedem | if you're doing some crazy crap in the oracle virt driver, then you're on your own | |
| 19:32:49 | dfisher | i'm really not doing anything crazy … I don't think. | |
| 19:32:59 | dansmith | dfisher: well, except for that one crazy thing | |
| 19:33:18 | dfisher | like, I'm using nova.api.metadata.password.convert_password | |
| 19:33:23 | dfisher | which takes a context | |
| 19:35:07 | dfisher | dansmith: ssssh. don't tell them </loud whisper> | |
| 19:37:38 | mriedem | dfisher: if you're looking at trunk code, build_instances in the conductor is only hit if (1) youre' using cellsv1 which you shouldn't be or (2) if the compute is doing a retry of a build | |
| 19:37:48 | dansmith | okay I think I'm doing the right thing, and now I'm failing because of that 500 from placement | |
| 19:37:56 | dansmith | so I'll push this up and let someone else look at that in parallel | |
| 19:37:59 | mriedem | dfisher: so assuming you're hitting (2), if you're using devstack in the default superconductor mode, the retry isn't going to work | |
| 19:38:16 | dfisher | hmm. ok. | |
| 19:38:20 | mriedem | dfisher: b/c you're hitting the cell conductor which doesn't have access to the scheduler | |
| 19:38:23 | mriedem | or api db for that matter | |
| 19:38:36 | mriedem | if you're not using devstack, or you're using a devstack that's older than like 1 week, then *shrug* | |
| 19:38:50 | dfisher | no, devstack is from yesterday or maybe Monday | |
| 19:38:51 | melwitt | yeah. the error is saying the node handling the database read doesn't have the API database connection configured in nova.conf | |
| 19:39:02 | openstackgerrit | Dan Smith proposed openstack/nova master: remove provider allocs in confirm/revert resize https://review.openstack.org/488510 | |
| 19:39:03 | openstackgerrit | Dan Smith proposed openstack/nova master: Test resize to same host with placement api https://review.openstack.org/489973 | |
| 19:39:03 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Sum allocations in the scheduler when resizing to the same host https://review.openstack.org/490085 | |
| 19:39:19 | mriedem | dfisher: if you need retries, then set CELLSV2_SETUP=singleconductor in your stackrc | |
| 19:39:20 | dansmith | cdent: jaypipes^ check out 'tox -efunctional ServerMovingTest' in the top patch there ^ | |
| 19:39:23 | mriedem | and restackaroo | |
| 19:39:31 | dfisher | very cool. thank you | |
| 19:39:40 | dfisher | i really appreciate it | |
| 19:39:44 | mriedem | dfisher: also https://docs.openstack.org/nova/latest/user/cellsv2_layout.html | |
| 19:39:49 | mriedem | digest that | |
| 19:42:00 | jaypipes | dansmith: gotcha. looking at it now. | |
| 19:42:16 | dansmith | also note the FIXME about cheating | |
| 19:42:22 | jaypipes | k | |
| 19:42:26 | dansmith | that's clearly just a hack to do some math, but needs to be integrated in the right place | |
| 19:43:16 | dansmith | IndexError: list index out of range | |
| 19:43:16 | dansmith | specifically this here: consumer_id = allocs[0].consumer_id | |
| 19:43:48 | dansmith | maybe that somehow means I'm passing an empty list of things to set and placement isn't having ti? | |
| 19:43:50 | dansmith | *it | |
| 19:44:44 | jaypipes | not sure, still looking :) | |
| 19:44:57 | dansmith | oh, well, heh | |
| 19:45:01 | dansmith | I'm definitely not adding it back in | |
| 19:45:02 | dansmith | duh | |
| 19:46:14 | dansmith | heyyyyy | |
| 19:46:15 | dansmith | woo | |
| 19:46:25 | dansmith | fix comin' | |
| 19:46:27 | openstackgerrit | Dan Smith proposed openstack/nova master: remove provider allocs in confirm/revert resize https://review.openstack.org/488510 | |
| 19:46:33 | dansmith | that passes & | |
| 19:46:58 | jaypipes | k, lemme pull that | |
| 19:47:33 | dansmith | but, that means that placement will 500 if you pass it an empty set of allocations (/cc cdent) | |
| 19:47:44 | cdent | noted | |
| 19:47:52 | jaypipes | cdent: bug please if you would | |
| 19:48:02 | cdent | yeah, just finishing something else first | |
| 19:48:02 | jaypipes | that should be an easy fix. | |
| 19:52:19 | jaypipes | dansmith: am I kosher to work on fixing that ocata to pike func test failure or are you still working on that patch? | |
| 19:52:39 | jaypipes | dansmith: and yes, I confirm that latest patch is passing tox -efunctional ServerMovingTest | |
| 19:52:47 | dansmith | jaypipes: yeah I hadn't looked at that yet, so I'll pass the baton back to you | |
| 19:52:49 | dansmith | jaypipes: someone has to fix up the "cheating" part anyway | |
| 19:53:14 | openstackgerrit | Ed Leafe proposed openstack/nova master: Handle addition of new nodes/instances in ironic flavor migration https://review.openstack.org/487954 | |
| 19:53:30 | jaypipes | dansmith: did I miss the cheating part? where is that? | |
| 19:53:38 | jaypipes | oh, I see it now. | |
| 19:53:39 | jaypipes | nm | |
| 19:53:59 | dansmith | jaypipes: https://review.openstack.org/#/c/488510/14/nova/scheduler/client/report.py@1102 | |
| 19:54:09 | jaypipes | yup, see it now. | |
| 19:54:16 | dansmith | probably need to have RT pass in the allocationy thing to subtract in there | |
| 19:54:21 | dansmith | instead of the report client knowing about flavors | |
| 19:54:36 | dansmith | and I figure there's some shared code to take a flavor and turn it into allocations, but I didn't go looking | |
| 19:55:03 | dansmith | but anyway, cheating aside, that passes all the forwards and backwards single and multi host tests for pike-and-pike | |
| 19:55:04 | jaypipes | dansmith: that something you want to fix up in this patch or do you want to tackle that in a followup? | |
| 19:55:05 | dansmith | which is... good | |
| 19:55:16 | dansmith | jaypipes: don't you think it's way too ugly to merge as is? | |
| 19:55:29 | jaypipes | dansmith: ugly sure, but harmless :) | |
| 19:55:31 | dansmith | jaypipes: i.e. it doesn't account for ephemeral_gb | |
| 19:55:50 | dansmith | maybe that's an easy fix, adding ephemeral and swap in there | |
| 19:56:15 | jaypipes | dansmith: sure, just remember swap is in MB :) | |
| 19:56:19 | dansmith | but if we can make it work right, I'm okay leaving the ugly there | |
| 19:56:35 | dansmith | jaypipes: you can just fix that bit while you're fixing the other test right? | |
| 19:56:56 | jaypipes | dansmith: happy to. was just checking to make sure I wouldn't clobber anything you were working on. | |
| 19:57:08 | dansmith | it's all yours | |
| 19:57:11 | jaypipes | kk | |
| 19:58:16 | jaypipes | dansmith: you want to keep those LOG.info() lines in remove_provider_from_instance_allocation()? | |
| 19:58:30 | mriedem | this deals with the various disk things https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L154 | |
| 19:58:37 | mriedem | so throw that into a utility method if we need | |
| 19:58:43 | jaypipes | mriedem: yup, on it. | |