| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-11-27 | |||
| 18:16:32 | dansmith | what I mean is, you have a placement for the api cell, and a placement for each compute cell | |
| 18:16:42 | mriedem | belmoreira: is this when we delete an instance? it gets routed to the child cell api and that tries to delete the build request? | |
| 18:16:51 | dansmith | so I'm guessing the api puts in a buildrequest, but the compute cell tries to delete it during boot | |
| 18:17:23 | dansmith | which means you keep getting the buildrequest listed out of the api, and maybe compute goes to delete it, finds it deleted, and assumes the instance was deleted? | |
| 18:17:49 | belmoreira | dansmith, the plan was to enable placement also in api_cell just to be already there for the pike upgrade | |
| 18:18:59 | belmoreira | dansmith the issue is because the build_request is not deleted, the instance is created but remains in build state when the user list its instances | |
| 18:19:13 | dansmith | okay so the first half of what I said | |
| 18:19:24 | belmoreira | yes | |
| 18:19:32 | dansmith | belmoreira: so you'll need something in the instance sync back to the api cell that deletes the build request | |
| 18:20:27 | belmoreira | dansmith what I did was to change delete_build_request to delete the request from the nova_api DB in the top cell | |
| 18:20:53 | belmoreira | basically adding a new conf option to point to that DB that is used in this function | |
| 18:20:55 | dansmith | belmoreira: I think you want to delete the buildrequest here: https://github.com/openstack/nova/blob/master/nova/cells/messaging.py#L998 | |
| 18:21:41 | dansmith | if you can test that, I think we could probably merge that change | |
| 18:22:27 | belmoreira | but it does it in the nova_api DB? | |
| 18:23:13 | dansmith | that code will run at the top-level nova-cells service, which would be configured to point to the top-level nova_api db right? | |
| 18:24:31 | belmoreira | I wasn't aware that nova-cells would sync nova_api DB. Only nova DB in top cell. | |
| 18:24:52 | dansmith | well, it doesn't do anything other than what you tell it to | |
| 18:25:13 | dansmith | historically there was no api db, so it didn't do anything there, but if you put code in there to find/delete the buildrequest then it will start to do some of that :) | |
| 18:25:34 | belmoreira | I see :) | |
| 18:25:53 | belmoreira | do we already have an example that nova_api DB is updated? | |
| 18:26:31 | dansmith | in nova-cells? not that I know of | |
| 18:27:14 | dansmith | it'll need to be graceful about buildrequest being deleted already, for the people that have a unified api db for all their cellsv1 cells | |
| 18:28:53 | dansmith | are you asking because of consistency or just because you're not sure how to go about the lookup/delete/ | |
| 18:29:15 | belmoreira | my current patch changes the the delete_build_request in conductor/manager to delete it in the nova_api cell. I wasn't aiming to merge it upstream :) | |
| 18:29:16 | dansmith | because it should be roughly the thing done in delete_build_request | |
| 18:29:43 | dansmith | okay but wouldn't you rather a change that doesn't need a new config option and _is_ upstream? | |
| 18:30:12 | belmoreira | I started this conversation because I'm not sure if we are missing something else | |
| 18:30:37 | belmoreira | or this can have any consequence when upgrading then to ocata/pike | |
| 18:31:45 | belmoreira | dansmith sure I would prefer a proper fix | |
| 18:35:37 | belmoreira | dansmith thanks. I will have a look on how to have it in cells/messaging | |
| 18:36:19 | dansmith | belmoreira: | |
| 18:36:24 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Fix cellsv1 build request deletion if multiple api databases are used https://review.openstack.org/523187 | |
| 18:36:38 | dansmith | belmoreira: see if that ^ works and/or tweak until it does and then we can move on with it | |
| 18:37:46 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Fix cellsv1 build request deletion if multiple api databases are used https://review.openstack.org/523187 | |
| 18:40:07 | belmoreira | dansmith: thanks a lot. I will let you know | |
| 18:40:25 | dansmith | belmoreira: cool | |
| 18:50:36 | artom | mriedem, haven't had a chance to look at the latest version yet | |
| 18:50:41 | artom | Will do that now | |
| 18:54:57 | openstackgerrit | Jay Pipes proposed openstack/nova master: Use oslo_db Session in resource_provider.py https://review.openstack.org/523192 | |
| 18:54:59 | jaypipes | mriedem, efried_rollin: ^ | |
| 18:58:24 | mriedem | that is a large change, but looks like it's inevitable | |
| 18:59:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix 'force' parameter in os-quota-sets PUT schema https://review.openstack.org/523194 | |
| 19:00:00 | mriedem | sdague: ^ is that 1 line os-quota-sets 'force' regression fix. the functional api samples are just to show the regression | |
| 19:05:51 | artom | Ohhh, the "need to open every PS individually to leave comments on each" thing is gone! | |
| 19:06:04 | artom | Man, my life just keeps on improving | |
| 19:06:09 | artom | This morning I learned about pssh | |
| 19:06:52 | jaypipes | mriedem: well, I figured it would be silly to fix only the conn.execute() calls in the lookup_entry_in_table() function and leave the other call locations. | |
| 19:08:08 | mriedem | jaypipes: if we need to backport that fix, then it would be best to fix the one place and do the massive refactor on top | |
| 19:08:44 | jaypipes | mriedem: there's latent bugs in all those other call locations. | |
| 19:09:18 | jaypipes | mriedem: you want to backport >1 patch or just one? | |
| 19:09:37 | jaypipes | mriedem: also, I'm happy to create a backport patch for pike/ocata | |
| 19:10:17 | mriedem | i single backport patch would be ideal, however, with the amount of refactoring that's gone on already in queens, the backport is probably going to suck, but if you want to own the backport then that's ok with me for a single change | |
| 19:10:58 | mriedem | i don't know that it needs to go to ocata | |
| 19:11:13 | mriedem | i think this actually crept up in CI because posting allocations during scheduling failed | |
| 19:11:15 | mriedem | which wasn't in ocata | |
| 19:12:15 | jaypipes | mriedem: I'll handle the backport to pike, then, no prob. | |
| 19:13:22 | sdague | mriedem: +2, though I'm still rather down on all the deepcopy and modify pattern, I think it creates more of these bugs in the future | |
| 19:13:35 | sdague | vs. just do a full expand every time there is a new tweak on the schema | |
| 19:22:23 | mriedem | sdague: thanks | |
| 19:22:27 | mriedem | dansmith: here is my proposed reno https://gist.github.com/mriedem/3da0a2edf18be0487e8becad31293077 | |
| 19:22:36 | mriedem | for https://review.openstack.org/#/c/521186/ | |
| 19:22:44 | openstackgerrit | Merged openstack/nova master: finish refactor AllocCandidates._get_by_filters() https://review.openstack.org/516782 | |
| 19:23:04 | openstackgerrit | Merged openstack/nova master: required traits for no sharing providers https://review.openstack.org/517027 | |
| 19:23:20 | openstackgerrit | Merged openstack/nova master: Add ProviderSummary.resource_class_names @property https://review.openstack.org/517646 | |
| 19:31:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Refined fix for validating image on rebuild https://review.openstack.org/521186 | |
| 19:31:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add regression test for rebuild with new image doubling allocations https://review.openstack.org/521153 | |
| 19:31:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix doubling allocations on rebuild https://review.openstack.org/521662 | |
| 19:31:37 | mriedem | jaypipes: i added a reno on https://review.openstack.org/521186 which you were +2 on | |
| 19:33:43 | cfriesen | when updating the default quota class set, if I set a value lower than the current usage will that prevent any more resources from being consumed? updating quota-sets has the "force" option for this case, but I don't see that for quota class sets. | |
| 19:36:51 | mriedem | cfriesen: quota classes aren't tied to a tenant | |
| 19:37:00 | mriedem | so there is no 'current usage' on a quota class | |
| 19:37:38 | mriedem | as for "if I set a value lower than the current usage will that prevent any more resources from being consumed" - i'm not sure | |
| 19:37:54 | mriedem | it might | |
| 19:38:00 | cfriesen | mriedem: I was envisioning usage as being basically per tenant. | |
| 19:38:33 | mriedem | we check limits based on (1) is there a specific limit for this project, (2) if not, is there a specific limit for this quota class, (3) if not, what's the configured limit | |
| 19:38:37 | cfriesen | so you have the class default as 10, the tenant currently has 9, I change the class default to 8 | |
| 19:39:06 | mriedem | i think that will cause the next instance create to be overquota | |
| 19:39:27 | cfriesen | that matches my understanding, thanks. it's odd we have "force" for that when setting tenant quotas but not the global default | |
| 19:39:34 | mriedem | the point in changing the limits to be below the usage (like the force parameter in PUT /os-quota-sets) is to get to the desired limit regardless of current usage | |
| 19:40:01 | mriedem | if we checked that on the global default, we'd have to check usage for all projects | |
| 19:40:10 | mriedem | and, | |
| 19:40:26 | mriedem | we could be tracking quota for projects that no longer actually exist in keystone | |
| 19:40:35 | cfriesen | true enough | |
| 19:40:54 | openstackgerrit | Merged openstack/nova master: Implement query param schema for delete assisted vol https://review.openstack.org/513668 | |
| 19:41:07 | openstackgerrit | Merged openstack/nova master: Versioned notifications for service create and delete https://review.openstack.org/519588 | |
| 20:08:58 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Refined fix for validating image on rebuild https://review.openstack.org/523212 | |
| 20:08:58 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add regression test for rebuild with new image doubling allocations https://review.openstack.org/523213 | |
| 20:08:59 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Fix doubling allocations on rebuild https://review.openstack.org/523214 | |
| 20:15:25 | openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: SEA https://review.openstack.org/523216 | |
| 20:19:37 | cfriesen | mriedem: when querying quota details including usage for a tenant/user, does it only count resources created by that user? | |
| 20:20:42 | mriedem | cfriesen: you should talk to melwitt | |
| 20:27:39 | mriedem | cfriesen: in general quotas are on the usage in the project, | |
| 20:27:41 | mriedem | except for keypairs | |
| 20:28:25 | mriedem | you can see how usage is counted by the methods called for each CountableResource here https://github.com/openstack/nova/blob/master/nova/quota.py#L1903 | |
| 20:28:58 | cfriesen | mriedem: thanks | |
| 20:30:35 | mriedem | so i guess PUT /os-quota-sets/{tenant_id} is a bit weird if you're updating limits for keypairs, since those are per-user, so that should really be something like PUT /os-qouta-sets/{tenant_id}/key_pairs/{user_id} - or something like that | |
| 20:32:24 | cfriesen | quotas in general is a mess. the mapping between the HTTP API and the internal representation is really indirect. Looking at the quota for a user/tenant, it takes multiple queries to figure if the limit came from a tenant/user quota, a tenant quota, the default quota class, or the default in the code or config file. | |
| 20:34:38 | artom | mriedem, dansmith, I think we may have gotten a filter wrong, but I could just be being an idiot. Review left. | |
| 20:35:34 | mriedem | cfriesen: that's what i explained before | |
| 20:35:59 | mriedem | cfriesen: and why i wrote https://review.openstack.org/#/c/411035/ | |