| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-06 | |||
| 20:46:15 | mriedem | so no compute involved | |
| 20:46:21 | dansmith | no, | |
| 20:46:35 | dansmith | this would be like a local delete | |
| 20:46:56 | dansmith | because we obviously need not call to compute in this case, so I'm asking in a normal local delete where compute is down, | |
| 20:47:04 | dansmith | do we have to clean up bdms and tags separately? | |
| 20:47:34 | mriedem | no, don't think so, those get removed via the db api | |
| 20:47:36 | mriedem | when you delete the instance | |
| 20:47:54 | dansmith | instance delete will delete BDMs | |
| 20:48:01 | mriedem | https://github.com/openstack/nova/blob/b79492f70257754f960eaf38ad6a3f56f647cb3d/nova/db/sqlalchemy/api.py#L1886 | |
| 20:48:05 | mriedem | https://github.com/openstack/nova/blob/b79492f70257754f960eaf38ad6a3f56f647cb3d/nova/db/sqlalchemy/api.py#L1894 | |
| 20:48:05 | dansmith | and tags | |
| 20:48:06 | dansmith | yeah | |
| 20:48:52 | dansmith | so when you delete the instance after the quota check those will get cleaned up if we have created them right? | |
| 20:49:45 | mriedem | well we don't delete the instance here, | |
| 20:49:54 | mriedem | we just set it to ERROR state | |
| 20:49:59 | mriedem | and then the user does the (local) delete | |
| 20:50:08 | dansmith | right | |
| 20:50:09 | mriedem | which should cleanup the bdms/tags | |
| 20:50:21 | openstackgerrit | Merged openstack/nova master: Fix broken link https://review.openstack.org/501391 | |
| 20:50:31 | mriedem | well, the bdms and tags wouldn't even exist in the cell db | |
| 20:50:37 | mriedem | if the quota check fails | |
| 20:50:47 | mriedem | oh but if we moved it to the end.. | |
| 20:51:03 | mriedem | yeah i think that would work | |
| 20:52:11 | dansmith | we can also do what mnaser wanted to do in the first place, which is just do this quick map fixup if we fail this quota check, | |
| 20:52:12 | melwitt | cells meeting in 8 minutes? | |
| 20:52:26 | dansmith | but then we will have an instance without those things in error state, and we'll be doing it from yet another place | |
| 20:52:41 | dansmith | I kinda want to keep it less spaghetti-like and do it in one place, personally | |
| 20:52:58 | dansmith | melwitt: I'm in an airport so we said this morning we'd cancel | |
| 20:53:08 | melwitt | oh, cool | |
| 20:53:11 | dansmith | if you have things to discuss we can, because I'm clearly connected, but, I didn't see the point | |
| 20:53:26 | melwitt | no, just forgot to check the log | |
| 20:53:36 | dansmith | ack | |
| 20:54:12 | mriedem | another alternative is just don't delete the build request in _cleanup_build_artifacts, or is that crazy? | |
| 20:54:24 | dansmith | no, I think we can't do that | |
| 20:54:36 | mriedem | the problem is that we haven't mapped the instance to a cell, and _cleanup_build_artifacts removes the build request, so the API literally can't find it anywhere | |
| 20:54:38 | mriedem | yeah? | |
| 20:54:44 | dansmith | or you mean in the failure case only? | |
| 20:54:47 | mriedem | yes | |
| 20:55:12 | mriedem | but then you'd have a build request in the api and an instance in the cell, unmapped | |
| 20:55:16 | dansmith | mriedem: we wouldn't know where to find the instance record to mark it as deleted when they deleted the buildreq, so we'd leave that undeleted but unfindable instance forever | |
| 20:55:18 | mriedem | and that instance in the cell db wouldn't get removed, | |
| 20:55:21 | dansmith | right | |
| 20:55:21 | mriedem | unless _cleanup_build_artifacts removes it | |
| 20:55:43 | mriedem | so, change _cleanup_build_artifacts to delete the instance from the cell and leave the build request for API requests? | |
| 20:56:10 | mriedem | unless we just overhaul this code to remove the 2 loops and do the quota check at the end | |
| 20:56:19 | melwitt | doing that would remove the instance fault message which explains the quota check failed | |
| 20:56:37 | mriedem | melwitt: can't the build request have a fault on it? | |
| 20:56:40 | dansmith | and we have the instance in the cell db, that's better than a buildreq, so we should use it | |
| 20:56:47 | melwitt | I didn't know it can | |
| 20:56:53 | dansmith | mriedem: I don't think we ever update that instance do we? | |
| 20:56:57 | dansmith | because it's fake right? | |
| 20:57:17 | mriedem | build request doesn't have a fault field, so yeah, can't do that | |
| 20:57:35 | melwitt | it has an instance field which would have a fault field I guess | |
| 20:57:47 | dansmith | no, I think the instance is fake and constructed on the fly | |
| 20:57:54 | melwitt | oh, okay | |
| 20:58:05 | mriedem | the BuildRequest.instance is persisted | |
| 20:58:10 | mriedem | it's just serialized json | |
| 20:59:29 | mriedem | but, that starts to get weird... | |
| 20:59:31 | dansmith | yeah, I guess so | |
| 20:59:36 | mriedem | managing state on an instance nested in a build requst | |
| 20:59:38 | dansmith | I'm not sure why that's better though | |
| 20:59:41 | dansmith | yeah | |
| 20:59:56 | mriedem | it's only better if we want to keep the 2 loops b/c we're afraid of changing them | |
| 20:59:56 | dansmith | this makes a bunch of other regular code just be used for anything else you do to this dead instance | |
| 21:00:15 | dansmith | I ain't skeered | |
| 21:00:20 | mriedem | dude, | |
| 21:00:28 | mriedem | i shit my pants everytime we touch nova these days | |
| 21:00:33 | dansmith | hah | |
| 21:01:03 | mnaser | (which is totally why i thought putting a tiny little call/fixup in the exception handling = less scary | |
| 21:01:05 | mnaser | :p | |
| 21:01:12 | melwitt | well, doesn't moving the quota check to the end and having only one loop address the issue fine since we know the BDMs and tags will get cleaned up at delete time? | |
| 21:01:20 | mriedem | melwitt: i think so | |
| 21:01:38 | melwitt | so the fake instance thing isn't the only way to get rid of the two loop thing | |
| 21:01:40 | dansmith | mnaser: yeah, but we already have lots of "clean this up in this snowflake way" calls | |
| 21:01:50 | mriedem | a test that creates an instance with a bdm and tag and asserts those are removed from the db would make me feel safer | |
| 21:01:54 | dansmith | melwitt: the fake instance is the way to keep the two loops he said | |
| 21:02:07 | dansmith | melwitt: fearing change | |
| 21:02:15 | mriedem | i fear the landmines | |
| 21:02:22 | mriedem | i have literally 2 toes left | |
| 21:02:37 | melwitt | oh. well, my thinking is the two loops are new and it's probably better to put it back to one now that we know we can | |
| 21:02:40 | dansmith | the easier way to do that is just do what mnaser wanted to do in the first place | |
| 21:02:45 | dansmith | melwitt: I agree | |
| 21:03:08 | dansmith | melwitt: and also, the fake instance in a build request is a special case and we should have that oooonly in the case where we haven't created the instance yet or can't, | |
| 21:03:19 | mnaser | for me it's the fact that it makes it more comfortable to backport this sort of thing (which i hope we can do to stable/pike) | |
| 21:03:21 | dansmith | so putting more things in that basket when we already have an instance created in a real cell seems wrong to me | |
| 21:03:26 | dansmith | mnaser: right, so, | |
| 21:03:29 | dansmith | I was going to suggest: | |
| 21:03:42 | dansmith | how about we let mnaser do the simpler but uglier thing of only mapping if we hit that quota check, | |
| 21:03:44 | melwitt | dansmith: yeah, agreed | |
| 21:03:54 | mriedem | and then collapse the loops in queens only | |
| 21:03:56 | mriedem | i like that | |
| 21:03:57 | dansmith | backport that mofo' and then do this cleanup separately with a functional test to get rid of the two loop | |
| 21:04:00 | dansmith | yeah | |
| 21:04:24 | mnaser | i think that would be best: so that stable can remain stable and you have plenty of time to review a much bigger change like that :> | |
| 21:04:28 | mnaser | and you're not worried about stable breaking | |
| 21:04:42 | mriedem | mnaser: btw, how is it that vexxhost is already on pike code? | |
| 21:04:47 | dansmith | I think that's reasonable | |
| 21:05:05 | dansmith | mriedem: you mean "thank you for already being on pike" right? | |
| 21:05:06 | mnaser | mriedem puppet+rdo packages, trusting the openstack CI and puppet-openstack-integration we use | |
| 21:05:07 | mnaser | aha | |