Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-11-30
15:38:15 mdbooth leakypipes: Interesting that we're not the only one creating pandas from legacy workloads.
15:38:31 leakypipes mdbooth: having some sort of "register this thing as a VM but don't do anything with it (yet)" would be useful for getting data/systems into OpenStack I think.
15:38:32 mriedem pretty sure there are lots of products that have something that does something like this,
15:38:46 leakypipes mdbooth: oh, we have a whole friggin zoo of pandas.
15:38:46 mriedem there was a lot of chatter in wechat one night about how openstack didn't have this import feature
15:38:52 mriedem especially for DR scenarios
15:39:18 cdent DR is the context I keep hearing it too
15:39:23 mriedem e.g. operator lost the nova server data, but had the hypervisor/ovs/volume info and just wanted to recreate the records
15:39:52 leakypipes well, that's just... well, ewww. but yeah I can kinda see that.
15:39:59 mnaser i remember that discussion
15:40:03 mnaser it was in the scope of someone's database blowing up
15:40:09 mriedem yes
15:40:10 mnaser and now everything was running but no records for anything
15:40:12 leakypipes (ewww == "guh, you should have a backup of your data")
15:40:26 mdbooth The way our v2v works is we create an ephemeral conversion instance, create a volume, the instance connects out to tools on the legacy hypervisor/physical and copies data to the volume. Then it does unspeakable hacks to it.
15:40:27 mriedem and the majority of replies were either "openstack should totally be able to fix that for you!" to "why don't you backup your database dummy"
15:40:43 leakypipes mriedem: :)
15:40:50 mdbooth Then we throw away the conversion instance and create a new one for the migrated workload, using bfv.
15:40:54 leakypipes mriedem: I think you know where I stand on that. :P
15:41:26 mnaser i mean we can just say keep the volume and boot that when you need it
15:41:33 leakypipes mriedem: then again, there's a reason that the mysql command line client has a --i-am-a-dummy option.
15:41:35 mnaser but i also dont think that is a really trivial way for users
15:41:50 mnaser leakypipes: hah, except only people who are advanced users know about it... :p
15:41:57 leakypipes mnaser: I know right? :)
15:42:22 mdbooth mnaser: Right, but that means you need to keep that metadata around until you want to boot it at some unspecified time. I can understand why they don't like that solution.
15:42:39 mriedem beggars can't be choosers
15:42:43 mnaser mdbooth: yeah, that's the only reason that i feel like this might make value i guess
15:42:47 mriedem well, unless they are beggars with tons of cash
15:43:03 mnaser but i also dont think it's a trivial change given the whole logic of nova boot is to boot an instance
15:43:06 leakypipes (in case people thought I was joking... https://www.percona.com/blog/2017/03/06/mysql-i-am-a-dummy/)
15:43:10 mdbooth mriedem: They have *some* cash :)
15:43:41 mriedem mnaser: +1 to that
15:44:42 mriedem this might be a shocker, but given our current review rate to bp ratio, i won't hold my breath that this is something we're going to seriously consider soon, especially given we're just now getting deployments upgrading to the big changes we've worked on the last 2 years and they are having problems with them that we're trying to solve
15:44:54 mdbooth mnaser: It looked technically simple enough if we pass a flag in to not boot. I think the change in the libvirt driver, at least, would be relatively small.
15:45:11 mdbooth mriedem: ack. That's exactly why I'm not jumping on this.
15:45:38 mriedem you're talking about the power_on flag right? we use that to keep a stopped vm shutoff during cold migration
15:45:43 mnaser would it? dont we have to shortcircuit the whole: plug port, start up paused, wait for notification, resume thing
15:46:02 mriedem we already have plumbing to not wait for the vif plug if we didn't power on the vm
15:46:10 mnaser ah okay
15:46:17 mriedem it was one of my first big bug fixes in nova
15:47:04 mnaser https://github.com/openstack/nova/commit/9e7c7706a76ad76612ba75314d436a8ba419a3eb
15:47:15 mnaser my amazing nova start
15:48:50 leakypipes :)
15:51:39 mnaser btw
15:51:44 mnaser we have 2 more releases to go
15:51:44 mnaser https://review.openstack.org/#/c/619352/
15:52:09 mnaser not that i run pike but it'd be nice to flush it out
15:59:56 openstackgerrit Matt Riedemann proposed openstack/nova master: Default zero disk flavor to RULE_ADMIN_API in Stein https://review.openstack.org/603910
16:00:54 mriedem mnaser: not sure about your thoughts on a nova-status upgrade check for ^
16:01:14 mriedem to see if (1) you have root_gb=0 flavors and (2) your policy checks out for those
16:01:19 mriedem and dump a warning if not
16:01:52 mriedem i posted to the ML, but likely no one has read it, or grokked it, or cares because they aren't upgraded to that point of caring yet
16:02:47 mriedem checking the policy would be kind of tough though if there is a custom rule configured
16:03:08 mnaser mriedem: i saw it and mentally responded, i should write up a reply
16:03:34 mnaser i'd like to warn against it but also i feel like it might break upgrade tooling in a way
16:03:57 mnaser i guess it's about trying to see it as "behavioural change" vs "bug fix"
16:04:39 mnaser i wouldn't want to see a warning because my cloud is operating as expected.. maybe a warning in stable/rocky though?
16:08:09 openstackgerrit Lee Yarwood proposed openstack/nova stable/queens: Add regression test for bug #1764883 https://review.openstack.org/621199
16:08:09 openstack bug 1764883 in OpenStack Compute (nova) "Evacuation fails if the source host returns while the migration is still in progress" [Undecided,Fix released] https://launchpad.net/bugs/1764883 - Assigned to Lee Yarwood (lyarwood)
16:08:09 openstackgerrit Lee Yarwood proposed openstack/nova stable/queens: compute: Ensure pre-migrating instances are destroyed during init_host https://review.openstack.org/621200
16:12:58 mriedem mnaser: by operating as expected, you mean you have 0 disk flavors but allow non-admins to use them intentionally even if they aren't booting from volume?
16:13:33 mriedem btw, this is another area where https://review.openstack.org/#/c/511965/ would help
16:13:44 openstackgerrit Lee Yarwood proposed openstack/nova stable/pike: Update docs for _destroy_evacuated_instances https://review.openstack.org/621203
16:13:45 openstackgerrit Lee Yarwood proposed openstack/nova stable/pike: Add regression test for bug #1764883 https://review.openstack.org/621204
16:13:45 openstack bug 1764883 in OpenStack Compute (nova) "Evacuation fails if the source host returns while the migration is still in progress" [Undecided,Fix released] https://launchpad.net/bugs/1764883 - Assigned to Lee Yarwood (lyarwood)
16:13:45 openstackgerrit Lee Yarwood proposed openstack/nova stable/pike: compute: Ensure pre-migrating instances are destroyed during init_host https://review.openstack.org/621205
16:14:31 lyarwood temka: ^ there you go
16:16:32 Sundar efried: Please ping me on cyborg channel when you can. Thanks.
16:16:37 mnaser mriedem: sorry, i meant i have 0 disk flavors and i am ok with non-admins no longer being able to boot
16:17:38 temka lyarwood, ack
16:19:02 mriedem mnaser: yeah in my mind the upgrade check wouldn't warn on that - if the policy is the default, you're ok even with 0-disk flavors. i'd think the check would warn if you have 0-disk flavors and your policy was not the default (admin_api)
16:19:13 mriedem but like i said, the policy rule could be custom so it's hard to say
16:19:34 mnaser ah i see what you mean, i could have renamed by admin_api to foo_bar and it would warn anyways
16:19:41 mriedem right
16:19:51 mriedem that's why i'm hesitant to add something
16:20:10 mnaser + if someone is actually running their own policy
16:20:13 mnaser its very likely that its custom anyways
16:22:43 mriedem SteelyDan: before i make this change that leakypipes suggested in my cross-cell resize series, do you agree with it or a meh? https://review.openstack.org/#/c/614012/3/nova/db/sqlalchemy/models.py@794
16:24:58 SteelyDan mriedem: HMM
16:25:00 SteelyDan so,
16:25:17 SteelyDan on the one hand, expanding the schema for a boolean seems kinda silly when we could store something useful
16:25:48 SteelyDan BUT, the migration lives in the cell db, and I don't like ever telling the cell what cell it is, which it can kinda infer from that data
16:26:04 SteelyDan like, imagine you do some migrations and then later collapse or split a cell
16:26:49 mriedem as noted in the change, i get the source cell mapping from the host mapping tied to the source_compute in the migration object https://review.openstack.org/#/c/603930/15/nova/conductor/tasks/cross_cell_migrate.py@871
16:26:56 mriedem so i don't *need* the explicit uuid in the migrations table
16:27:00 SteelyDan right
16:27:07 mriedem the boolean flag is really for flow control
16:27:11 SteelyDan I know
16:27:37 mriedem i don't know if the source/dest cell uuids being in the migration record could be abused later
16:27:50 mriedem i mean, i'm sure they could, i'm just not sure in what case
16:28:21 SteelyDan well, it's not a matter of abuse so much as just separation of responsibilities
16:28:23 mriedem once the migration is done, we shouldn't really care about the record anymore, except reporting
16:28:45 mriedem and if an operator is going to split cells, they had better make sure they don't have migrations going on while doing so
16:29:06 SteelyDan yeah, not a matter of ongoing ones, but just those being wrong later
16:29:24 SteelyDan nothing else in the cell db knows about its position in the hierarchy and I think we should keep it that way
16:29:31 SteelyDan so while I was going to argue against just storing a boolean,
16:29:36 mriedem right. i mean, there could be a comment in there saying, "don't trust these after the migration is done"... but that's a bit hokey
16:29:39 SteelyDan I think I'd prefer that over the uuids just for that reason
16:29:43 SteelyDan heh
16:29:53 mriedem SteelyDan: ok care to reply so i can move on?

Earlier   Later