| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-09 | |||
| 19:18:35 | mriedem | this kind of seems like a bug of it's own, but not sure what we'd do about it | |
| 19:19:05 | mriedem | we'd have to update any shelved instances in an az if we rename that az | |
| 19:20:33 | jmlowe_ | if nothing else, where ever an instance is unshelved it needs to update the device_owner because it is there and that is truth | |
| 19:20:50 | mriedem | jmlowe_: yeah i'm trying to get to that part | |
| 19:21:03 | mriedem | jmlowe_: while you've got az renames on the brain, maybe you'd like to reply to http://lists.openstack.org/pipermail/openstack-operators/2018-March/015029.html with your thoughts | |
| 19:22:06 | jmlowe_ | oh, I effectively did that by putting all my computes into rackwise az | |
| 19:22:28 | jmlowe_ | to get migration to work again I wound up editing the specs | |
| 19:22:47 | mriedem | i bet updating json blobs in the db is super fun | |
| 19:28:50 | mriedem | jmlowe_: ack confirmed http://paste.openstack.org/show/718769/ | |
| 19:36:08 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Cleanup RP and HM records while deleting a compute service. https://review.openstack.org/554920 | |
| 19:36:52 | jmlowe_ | mriedem: it was all the fun you might imagine it to be and then some | |
| 19:38:32 | jmlowe_ | if you could never change the az name because you could never change the spec then az's really become a nightmare for operators and should be avoided at all costs, probably not what people really want | |
| 19:45:38 | mriedem | jmlowe_: what if, for starters, as mentioned in that ML thread, we just didn't allow az renames while there were (alive) instances in those AZs? | |
| 19:45:49 | mriedem | so operators can't shoot themselves in the foot that way? | |
| 19:49:35 | jmlowe_ | Lets say you start with the default "nova", specs wind up with nova in it, how to you create any AZ if you didn't plan on it from the beginning? Do you have to get this right before any people start using your cloud and make sure your guesses never having run a cloud before were correct for all time? | |
| 19:49:55 | mriedem | is nova put into the request spec? | |
| 19:50:21 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L895 | |
| 19:50:35 | mriedem | 'nova' shouldn't be in the request spec unless the user specifically requested --availability-zone nova | |
| 19:51:09 | jmlowe_ | Maybe not anymore, but my "no free hosts" live migration problems were because nova az was in the specs | |
| 19:51:42 | jmlowe_ | Fixes you didn't know you needed but always wanted | |
| 19:51:44 | mriedem | hmm, i'm not saying users couldn't wedge themselves, | |
| 19:51:51 | mriedem | the server create API does say to not specify 'nova' here https://developer.openstack.org/api-ref/compute/#id10 | |
| 19:52:36 | jmlowe_ | I think I would make more mistakes with renames disallowed than I would with it allowed | |
| 19:53:15 | mriedem | huh, this inconsistency in the CLI for non-admins isn't good either http://paste.openstack.org/show/718771/ | |
| 19:53:32 | jmlowe_ | it's kind of bad data modeling to have it all based on name and not id | |
| 19:53:48 | mriedem | jmlowe_: yeah that's jaypipes' point in the ML thread, | |
| 19:53:53 | mriedem | but is also a bigger hairier change | |
| 19:54:26 | jmlowe_ | I really don't have any answers over here, just the ability to create more problems for myself | |
| 19:54:39 | mriedem | heh, fair enough | |
| 19:55:44 | jmlowe_ | I'm looking at the possibility of going all in on host aggregates to keep certain customers on hardware they bought sometime later this summer | |
| 19:56:07 | jmlowe_ | none of this is going to get easier for me | |
| 19:56:07 | openstackgerrit | Merged openstack/nova master: Fix comments at the 'save' method of objects.Instance https://review.openstack.org/559743 | |
| 19:57:07 | jmlowe_ | I've observed that cli behavior before | |
| 19:57:51 | openstack | Launchpad bug 1762534 in python-openstackclient "openstack availability zone list shows default 'nova' AZ to non-admin users" [Undecided,New] | |
| 19:57:51 | mriedem | https://bugs.launchpad.net/python-openstackclient/+bug/1762534 | |
| 19:57:57 | mriedem | i'll have to dig into that later | |
| 20:09:46 | openstackgerrit | Merged openstack/nova master: Fix cancel_all_events event name parsing https://review.openstack.org/558059 | |
| 20:18:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update port device_owner when unshelving https://review.openstack.org/559828 | |
| 20:18:43 | mriedem | jmlowe_: ^ | |
| 20:20:13 | jmlowe_ | That looks pretty good | |
| 20:29:00 | mriedem | oh it doesn't fix the bug in the single node devstack case... | |
| 20:29:16 | mriedem | because on shelve offload, we don't clear the port binding host id, so the conditional on unshelve isn't true | |
| 20:29:17 | jmlowe_ | really? looks like it should work | |
| 20:29:26 | mriedem | it would work in a >1 compute env :) | |
| 20:29:33 | jmlowe_ | oh, that's what somebody was on about when I first reported it | |
| 20:29:44 | jmlowe_ | clearing the port on shelve | |
| 20:29:52 | mriedem | that was me :) | |
| 20:30:03 | mriedem | and that's the todo i left in the code | |
| 20:30:06 | jmlowe_ | how convenient | |
| 20:32:37 | mriedem | i wanted to avoid fixing the shelve / clear case in the same patch since that gets slightly more complicated, and not really needed to fix this bug | |
| 20:32:56 | mriedem | unless your production env is 1 compute host... | |
| 20:33:01 | mriedem | maybe it's a mainframe | |
| 20:53:42 | openstackgerrit | Jay Pipes proposed openstack/nova master: mirror nova host aggregate members to placement https://review.openstack.org/553597 | |
| 21:12:39 | tblakes | dansmith: Could you please take a look at https://review.openstack.org/#/c/559158/ if you have a chance. It's gotten a +2 from Matt Riedemann but still needs to get a +1 on workflow. It's a cherry pick from master to stable/queens. | |
| 21:51:35 | imacdonn | dansmith: if you have a moment, could you evaluate https://review.openstack.org/#/c/558089/ , please? | |
| 22:00:16 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM: Add proc_units_factor conf option https://review.openstack.org/554688 | |
| 22:04:15 | dansmith | imacdonn: mriedem: sorry, I just can't get on board with that | |
| 22:04:41 | imacdonn | dansmith: OK.... do you have another idea ? | |
| 22:05:37 | dansmith | imacdonn: I don't think it's something we need to do at all, TBH, but if we're going to, we should make it useful | |
| 22:05:47 | imacdonn | efried: heads-up | |
| 22:05:57 | dansmith | imacdonn: we're going to hork out the resource provider a couple seconds later, so do that early or something | |
| 22:05:59 | dansmith | but really, | |
| 22:06:19 | dansmith | we should be fine to just log the problem, explain the issue and whatever | |
| 22:06:26 | imacdonn | dansmith: my original idea was to do away with the check completely ... but mriedem was concerned about the case where the scheduler doesn't require placement, but we want to force it anyway, for reporting purposes | |
| 22:07:06 | dansmith | imacdonn: okay I'm not sure why the current config-based check doesn't support that scenario | |
| 22:07:51 | imacdonn | dansmith: the issue is that the current check uses an option that's not required .. so it barfs on a perfectly valid config | |
| 22:08:10 | imacdonn | dansmith: I suppose an alternative would be to check for auth_type instead of region_name | |
| 22:08:27 | dansmith | imacdonn: ah, right, well, then no check at startup like I was saying I guess | |
| 22:08:28 | imacdonn | dansmith: if auth_type is always going to be required ............. | |
| 22:08:40 | dansmith | imacdonn: catch it when we go to fetch our RP record the first time, log the bejeezus out of it | |
| 22:09:02 | efried | IMO it would be nice to fail early with a nice message rather than let it fail "naturally" somewhere down the line. | |
| 22:09:08 | imacdonn | dansmith: I believe that there's a case where there would be no request for RP (depending on scheduler config) | |
| 22:09:28 | efried | right, and in ^ that case, "down the line" could be way later. | |
| 22:09:56 | dansmith | ? | |
| 22:09:56 | dansmith | efried: we don't check all of config on startup, but also, isn't this a thing that could be fixed at runtime in the service catalog | |
| 22:10:19 | dansmith | imacdonn: efried: no, the RP setup will happen just a few seconds after startup, when listing the nodes and running the resource tracker stuff | |
| 22:10:36 | efried | For this code path, you have to fix it by editing conf. | |
| 22:11:02 | efried | which I hear is going to be able to be picked up dynamically across the board soon anyway, so... | |
| 22:11:13 | dansmith | efried: it already can be | |
| 22:11:20 | dansmith | efried: if you mark these things as reloadable and they are, | |
| 22:11:22 | dansmith | then done deal | |
| 22:11:30 | efried | cool cool. | |
| 22:11:45 | efried | dansmith: So you're advocating removing the check entirely and letting it fail organically down the line | |
| 22:12:00 | dansmith | efried: if we can't do a legit check then yeah | |
| 22:12:07 | efried | Just so you're aware, @safe_connect has the potential to make that somewhat... unclean. | |
| 22:12:17 | efried | Wait, how is this check not legit? | |
| 22:12:24 | dansmith | "why the fsck are my computes requesting /foo/?bogus" would be high on my list | |
| 22:12:40 | dansmith | efried: I mean a legit check without spamming the logs in placement | |
| 22:12:59 | dansmith | a legit config-only check | |
| 22:13:05 | efried | one entry on startup is hardly spamming, but okay. auth_type it is, I guess. | |
| 22:13:41 | dansmith | I just restarted 1000 computes | |
| 22:13:55 | imacdonn | can we say (with reasonable confidence) that auth_type will always be a required option? TBH, I'm not sure why it doesn't default to 'password' | |
| 22:13:56 | dansmith | that's a lot of spam | |
| 22:14:15 | efried | This I cannot answer. | |
| 22:14:20 | dansmith | if I didn't, and I'm auditing logs, I find some /?bogus queries to placement and wonder who the fsck is on my network | |
| 22:14:29 | dansmith | and then realize it's ... mah compute nodes | |
| 22:14:34 | dansmith | I don't want to be on that customer call | |
| 22:15:06 | efried | imacdonn: In other places we go to load the auth from conf and error if it comes back None. | |
| 22:15:36 | dansmith | we call pre_start_hook before we even listen to RPC, and that calls update_available_resource, which will pull our RPs from placement | |
| 22:15:40 | efried | Which in this case might look a little bit like publicizing (bits of) SchedulerReportClient._create_client. | |