| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-03 | |||
| 14:01:38 | bauzas | https://etherpad.openstack.org/p/nova-stein-rc-potential is pretty done | |
| 14:02:38 | trident | mriedem: You are correct, forbidden traits never worked unless there were also a required trait in the same flavor. If there were, both would be used, if not, the forbidden trait was lost. | |
| 14:03:14 | bauzas | mriedem: https://review.openstack.org/#/c/649454/ wanting it for RC2 ? I can vote on my own change given melwitt proposed it | |
| 14:03:40 | bauzas | I think it should clarify our Stein docs | |
| 14:03:49 | bauzas | even if we don't honestly branch them | |
| 14:06:04 | mriedem | bauzas: i'm waiting to hear that yes we're doing an rc2 | |
| 14:06:15 | mriedem | and then i'd like to step through with the ptl which stein backports are going to go into it | |
| 14:06:28 | bauzas | soooooo... efried? | |
| 14:07:22 | mriedem | there is a 50% chance efried is the PTL coordinating the stein RC2 :) | |
| 14:08:54 | bauzas | oh, right, it's still melwitt's point :p | |
| 14:09:20 | efried | hi, sorry, catching up. | |
| 14:10:03 | mriedem | bauzas: you could start by backporting https://review.openstack.org/#/c/649409/ to stable/stein | |
| 14:10:36 | bauzas | mriedem: ok, I can do it | |
| 14:10:43 | efried | I think we need an RC2 for sure for https://review.openstack.org/#/c/649409/ at least | |
| 14:10:57 | efried | and if we're doing it, we might as well put the docs in, no risk there. | |
| 14:11:21 | mriedem | https://review.openstack.org/#/c/649454/ specifically yeah? | |
| 14:12:00 | efried | yes | |
| 14:13:11 | efried | mriedem: wanna +A that one please? | |
| 14:14:17 | mriedem | done | |
| 14:14:34 | efried | mriedem: Has anyone done docs on the SIGHUP thing? | |
| 14:14:40 | efried | L56 | |
| 14:14:44 | mriedem | no | |
| 14:14:59 | mriedem | i was just creating a devstack env to try and recreate the bug | |
| 14:15:50 | efried | .. note:: SIGHUP is broken, see `bug 1715374`_ | |
| 14:15:50 | efried | mriedem: I can go through and add a quick | |
| 14:15:52 | efried | to all the docs where it's mentioned if you like. | |
| 14:15:52 | openstack | bug 1715374 in OpenStack Compute (nova) "Reloading compute with SIGHUP prevents instances from booting" [High,In progress] https://launchpad.net/bugs/1715374 - Assigned to Ralf Haferkamp (rhafer) | |
| 14:16:57 | mriedem | well, if the nuance is it's broken for a window while servers are being created and waiting for an event when the sighup runs, that gets a bit hard to communicate in all the places we have sighup mentioned in the docs | |
| 14:17:36 | efried | .. note:: SIGHUP behavior is questionable, see `bug 1715374`_ if you try it and things go wobbly. | |
| 14:17:37 | efried | ? | |
| 14:17:37 | openstack | bug 1715374 in OpenStack Compute (nova) "Reloading compute with SIGHUP prevents instances from booting" [High,In progress] https://launchpad.net/bugs/1715374 - Assigned to Ralf Haferkamp (rhafer) | |
| 14:18:28 | mriedem | idk, could give a more detailed explanation of the bug in the reset() portion of https://docs.openstack.org/nova/latest/reference/services.html#the-nova-manager-module for just the compute service | |
| 14:19:06 | mriedem | i wouldn't want to mention a bug in 5 places just to have to remember after the bug is fixed to go back and remove all of those 5 places | |
| 14:20:18 | efried | as you wish | |
| 14:21:00 | openstackgerrit | Sylvain Bauza proposed openstack/nova stable/stein: Libvirt: gracefully handle non-nic VFs https://review.openstack.org/649630 | |
| 14:21:09 | bauzas | mriedem: ^ | |
| 14:27:36 | mriedem | is there anything else we want to consider? https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/stein | |
| 14:29:12 | bauzas | mriedem: https://review.openstack.org/#/c/647310/ is controversial for RC2 | |
| 14:31:24 | dansmith | I thought we said no to that? | |
| 14:31:52 | dansmith | and sounds like lyarwood is on board with the original plan too | |
| 14:32:58 | openstackgerrit | Helena proposed openstack/nova-specs master: Spec for a new nova virt driver to manage an RSD, composable infrastructure deployment https://review.openstack.org/648665 | |
| 14:32:59 | lyarwood | yarp, missed that it had already been discussed in the change once I got back online yesterday. gertty-- | |
| 14:34:26 | lyarwood | Does anyone have any idea why nova-stable-maint already has +2 on stable/stein btw? | |
| 14:34:52 | mriedem | i'm not sure if the release team distinguishes anymore | |
| 14:34:54 | mriedem | smcginnis: ^? | |
| 14:36:26 | jaypipes | mriedem, dansmith: for online data migrations, did we envision those migration routines living forever or are we envisioning being able to delete some over time? | |
| 14:36:38 | mriedem | jaypipes: we already have deleted some over time | |
| 14:36:42 | dansmith | jaypipes: we have deleted many | |
| 14:36:50 | jaypipes | oh, ok. | |
| 14:36:55 | mriedem | most recent memory is i removed the request spec and flavor migration ones | |
| 14:37:07 | dansmith | we're not super good at setting that timer and then doing it, like any other cleanup | |
| 14:37:15 | jaypipes | mriedem, dansmith: after looking into mnaser's troubles, I think the removing the keypairs migration entirely might be the best solution. | |
| 14:37:20 | dansmith | but most of them should be relatively no-op-ish if done properly | |
| 14:37:21 | jaypipes | it's Newton-era | |
| 14:37:38 | jaypipes | dansmith: unfortunately that one does a full table scan across instance_extra each time it's run. | |
| 14:37:45 | mriedem | jaypipes: yeah i started looking at that the other day, but i don't think we have a blocker migration or anything in place to make sure the migration has completed before we rip that out | |
| 14:37:47 | mriedem | like we did for flavors | |
| 14:37:47 | jaypipes | dansmith: looking for WHERE keypairs=NULL | |
| 14:38:06 | dansmith | mriedem: we might not be able to for that one | |
| 14:38:19 | jaypipes | dansmith: and for large DBs like mnaser's 3M+ instance_extra records, it really is a resource hog. | |
| 14:38:22 | dansmith | mriedem: but since it's so old, if it hadn't finished, we probably know that other things would be broken | |
| 14:38:24 | mriedem | if there are keypairs in the cell db wouldn't that be an indication? | |
| 14:38:26 | dansmith | jaypipes: I hear you | |
| 14:38:51 | mriedem | for the request spec migration, we didn't have a blocker migration, just a nova-status upgrade check: https://github.com/openstack/nova/commit/ed4fe3ead62c09ec7de7b6a11072295a99997b4f#diff-91e852cb498abb50ca653ef6418bd65a | |
| 14:39:05 | mriedem | oh i added the upgrade check in rocky, dropped the reqspec migration code in stein | |
| 14:39:11 | dansmith | jaypipes: I don't agree that going back to putting this inside the schema migration would make people happy, as his cloud would still be down while he waits for that to happen, but.. you know :) | |
| 14:40:02 | dansmith | jaypipes: hopefully you just meant some sort of version-like sentinel instead of scanning to determine if things needed to be done :) | |
| 14:40:04 | jaypipes | dansmith: why would his cloud be down? | |
| 14:40:14 | dansmith | jaypipes: if we did it in the middle of a schema migration? | |
| 14:40:34 | jaypipes | yeah. | |
| 14:40:52 | dansmith | because he'd be halfway between two schema versions with two versions of code that can't use it? | |
| 14:41:06 | dansmith | i.e. the reason we decoupled those tasks in the first place | |
| 14:41:11 | jaypipes | sorry, you misunderstand... | |
| 14:41:46 | mriedem | wouldn't dropping the keypairs migration be similar to dropping the instance groups migration? https://github.com/openstack/nova/commit/1160921c2d053ce33279ca4ec1f00572271e7c95#diff-91e852cb498abb50ca653ef6418bd65a | |
| 14:42:19 | mriedem | jaypipes: fyi https://review.openstack.org/#/q/topic:remove-newton-online-compat-code+(status:open+OR+status:merged) | |
| 14:42:21 | mriedem | for a blueprint | |
| 14:42:26 | jaypipes | dansmith: if we did the data migration as an Alembic/sqlalchemy-migrate migration, then we wouldn't have to ever run the "check to see if this pre-condition exists" (like the SELECT against the instance_extra table...) more than once. | |
| 14:42:35 | jaypipes | mriedem: ack | |
| 14:42:59 | mriedem | i seem to remember we've fucked up the schema migration scripts before as well | |
| 14:43:05 | jaypipes | dansmith: I'm not saying the data migration would be the *same* migration as the schema migration. just that we could do it *as* a migraiton. | |
| 14:43:12 | dansmith | jaypipes: that's what I said above about the sentinel approach for scanning, but definitely do not agree they should be in the same migration | |
| 14:43:14 | jaypipes | instead of the whole separate nova-manage thing. | |
| 14:43:17 | mriedem | and since you can't downgrade the schema to get back, you'd have to reset the version in the migrations table and re-run those if we had a bug | |
| 14:43:21 | jaypipes | but that train has passed... | |
| 14:43:27 | dansmith | jaypipes: okay, well, you can see why people might be confused about such a statement :) | |
| 14:43:40 | jaypipes | mriedem: not once, ever, has anyone ever done a downgrade of a schema migration. | |
| 14:43:54 | mriedem | that's not true | |
| 14:44:12 | mriedem | hell we used to gate on being able to upgrade and downgrade the schema | |
| 14:44:26 | jaypipes | gating != ever being done in production, ever. | |
| 14:44:27 | dansmith | it didn't actually work for real data | |
| 14:44:28 | dansmith | yeah | |
| 14:44:30 | dansmith | heh | |
| 14:44:41 | jaypipes | it just doesn't happen, sorry. | |
| 14:44:51 | dansmith | which is why I don't want to couple the two processes | |
| 14:45:08 | jaypipes | the procedure is a) backup your stuff, b) run migrations, c) test and if problem, d) restore from backup. | |
| 14:45:21 | dansmith | but jaypipes, we could of course just set up our own version counter to be a sentinel to avoid re-running migrations, | |
| 14:45:33 | jaypipes | nobody ever ran a schema downgrade in production. I can almost guarantee that. | |
| 14:45:44 | mriedem | i should introduce you to some chinese people i know then | |
| 14:45:59 | mriedem | anyway, i'm not saying it's something that should be done | |
| 14:46:03 | jaypipes | mriedem: then, clearly, SOMEONE is WRONG on the Internet. | |