Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-04-26
16:08:36 mriedem if people want this, then they need to put up the dev resources to do it
16:08:41 mriedem rather than api hacks
16:09:09 mriedem cinder ephemeral imagebackend also allows us to deprecate lvm, rbd and scaleio (if that lands)
16:09:29 mriedem i think anyway
16:09:57 johnthetubaguy mriedem: my problem is the whole way we do disks is truly terrible, so to get to something clean buts a lot of burden on the users to switch to something radically different
16:10:00 mriedem having said this, there is probably a world of API operations that don't work for boot from volume servers, like shelve/unshelve and rebuild
16:10:23 mriedem users == operators?
16:10:28 johnthetubaguy mriedem: they should all work, they might be broken
16:10:33 johnthetubaguy mriedem: I was meaning API users
16:11:06 mriedem i don't see how a cinder ephemeral imagebackend is radically different from the API pov
16:11:11 mriedem the api doesn't change
16:11:30 mriedem the operator just says he wants to use cinder for storage on this compute host
16:11:46 mriedem which is what that spec is also doing, probably with host aggregates tied to the marked flavors
16:11:55 johnthetubaguy mriedem: the problem is we create user visible cinder volumes, that we didn't before, then we talk about customizing the volume type, and mixed diskless and non-diskless, and it all gets a bit tricky.
16:13:30 mriedem at the ptg we said that we could add the support for volume type via extra spec if we did cinder ephemeral imagebackend,
16:13:33 mriedem which was a win for cburgess
16:14:05 mriedem i agree you get a user visible cinder volume that you wouldn't have before,
16:14:17 edleafe ` /win 14
16:14:20 mriedem but i'm not sure that's a huge issue as long as it's marked for delete_on_termination=True
16:14:21 cburgess What do we mean by "wouldn't have before"?
16:14:34 mriedem wouldn't have before "automatically"
16:14:39 mriedem today you opt in to bfv
16:14:58 mriedem this would have nova do it for you depending on which compute host you land on
16:15:09 cburgess OK right but, presumably it would be a deployer, or at least cloud admin, option to use the new as of yet unspeced nova roots on cinder idea.
16:15:33 mriedem yes it would be
16:15:42 cburgess So I don't think its a problem that users would see the cinder volumes.
16:15:49 mriedem as would marking your flavors as local_disks=False like in https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/flavor-root-disk-none.html
16:15:56 cburgess In pure BFV environments today they already deal with that.
16:16:28 mriedem it's just with https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/flavor-root-disk-none.html if the user picks the wrong flavor or makes the wrong server create request, it will blow up
16:16:36 mriedem and then they have to go back and figure out how to add bdms via the nova boot cli
16:16:37 mriedem which is not fun
16:16:37 cburgess How so?
16:16:40 cburgess I'm not following.
16:16:54 mriedem "The use case for End Users is to not create ephemeral (root) disks locally on the hypervisor by accident. When launching an instance from a flavor with disabled local disks, the user will be informed to create a bootable volume in order to launch the instance with that volume."
16:17:05 mriedem "be informed" means 400
16:17:45 mriedem holy crap, just look at the proposed change minefield
16:17:47 mriedem "error, when flavor create request with local_disks=False and root_gb > 0 and/or emphemeral_gb > 0 and/or swap_gb > 0"
16:18:02 mriedem "return descriptive exception, when flavor with local_disks=False and requested image larger than given blockdevice layout"
16:18:07 mriedem "if local_disks is omitted or set to True, root_gb is mandatory again"
16:18:22 mriedem "if jupiter is aligned with saturn and it's a leap year, the api works, else 400"
16:18:37 cburgess Yeah this seems... sub-optimal.
16:18:44 cdent what about when mercury is in retrograde?
16:18:53 cburgess Why can't we use the rest of the info from the boot request and craft a BFV request for them.
16:18:59 mriedem cdent: then all bets are off
16:19:16 mriedem cburgess: that's probably https://review.openstack.org/#/c/432503/
16:19:18 cburgess Granted... turning off local disks is a really big deal/issue today and causes us issues.
16:20:02 johnthetubaguy right, its not possible today, due to root_gb=0 meaning "don't resize image"
16:20:07 cburgess mriedem Yeah but you killed that. So it seems to me that if we do local_disk=False we either need to build the BDM under the covers, or we need to allow the BDM in the flavor as well.
16:20:34 cburgess johnthetubaguy How does that impact this exactly?
16:20:40 mriedem if we build the bdm under the coveres if local_disk=False, that's the same thing as https://review.openstack.org/#/c/432503/
16:20:45 mriedem just a different attribute on the flavor
16:20:56 mriedem ^ was doing is_bdm_flavor=True on the flavor
16:20:58 mriedem same idea
16:21:01 cburgess OK
16:21:02 cburgess Well...
16:21:08 johnthetubaguy cburgess: people often assume 0 GB means no disk
16:21:20 cburgess Honestly it seems reasonable to me. But then I thought that spec was reasonable to begin with.
16:21:25 cburgess johnthetubaguy Well they are wrong :P
16:21:30 johnthetubaguy cburgess: agreed
16:21:35 mriedem cburgess: you're saying you like both ideas
16:21:45 mriedem the abandoned spec, and the extra specs / bfv thing
16:21:59 mriedem cburgess: you like anything that gets rid of your local patches :)
16:21:59 cburgess mriedem I'm saying disks are a total and complete mess and we have to find ways of making this easier and better to consume because right now the whole thing is a confusing mess.
16:22:33 cburgess mriedem Yes.. but... I have those local patches because things are a mess. So I like anything that makes a cleaner, easier to use system for users.
16:22:48 mriedem yeah i know,
16:23:00 mriedem if a cinder ephemeral imagebackend is never going to happen, then maybe we go with https://review.openstack.org/#/c/432503/
16:23:14 mriedem i just really dislike coupling two terrible tech debt things in nova, extra specs and bdms
16:23:21 mriedem because that's bound to be a maintenance nightmare
16:23:32 cburgess Well I think a cinder ephemeral imagebackend is a long term thing. Question is what, if anything, can we do in a cycle or two.
16:23:44 smcginnis I've been seeing more and more requests for using Cinder as ephemeral storage.
16:23:57 mriedem cburgess: nothing is happening in pike
16:24:05 mriedem b/c nothing was proposed before spec freeze
16:24:24 mriedem smcginnis: yes it has been for a long time, the question is how to best provide it w/o adding a bunch of tech debt to nova
16:24:50 cburgess agreed
16:24:57 mriedem as i said earlier, we already have apis that don't work with volume-backed instances
16:25:10 mriedem like, maybe we should fix some of that...
16:25:13 johnthetubaguy mriedem: but is that actually true? or just a bug?
16:25:17 cburgess mriedem I know Boston isn't supposed to be a summit but... I kind of feel like we need a working session at some point to lay out what the correct roadmap for storage is.
16:25:40 mriedem johnthetubaguy: bugs, but big hairy bugs
16:25:45 cburgess mriedem Because we have been dancing around these issues for 2 cycles nw.
16:25:45 cdent cburgess: hasn't that sesssion happened at every summit for the last several years?
16:25:57 mriedem johnthetubaguy: cfriesen was just asking about volume-backed instance rebuild yesterday and how it doesn't work
16:26:06 stackempty mriedem: so what would be my options for now .. to have something close to preventing local disk storage on the hypervisor ?
16:26:09 EmilienM mriedem: hi, I was afk this morning. Please let me know how can I help wrt wsgi
16:26:12 mgagne hehe, we used to have a similar need (default to BFV) in the past and implemented it with an ugly middleware: https://gist.github.com/mgagne/161bd7eaeb4c747fa1ae97c0bc43c093
16:26:14 cburgess cdent Sort of. We keep doing it in terms of a feature request or two from individual parties (multi-attach, volume-type, etc).
16:26:37 cburgess cdent I'm thinking we need a more high level, ignoring all individual feature requests, what do we have now and where do we need to involve it too
16:26:38 mriedem stackempty: it sounds like lots of providers have implemented downstream solutions
16:26:42 mriedem as cburgess and mgagne pointed out
16:26:50 johnthetubaguy mriedem: OK, I guess we have concrete examples then
16:27:19 mriedem johnthetubaguy: i had something in the ML recently about how shelve/unshelve is just totally ignorant wrt volues
16:27:21 mriedem *volumes
16:27:31 mriedem and we've had bugs open for over a year that haven't been resolved
16:27:37 cburgess mriedem Another case for removing that feature
16:27:39 johnthetubaguy mriedem: yeah, that whole feature is un finished, so I expected that of shelve
16:28:32 johnthetubaguy cburgess: thats not a crazy plan
16:28:36 mriedem mgagne: ok so that basically does https://review.openstack.org/#/c/432503/
16:28:40 mriedem just via paste middleware
16:29:13 johnthetubaguy so is this a middle ground?
16:29:20 johnthetubaguy we do https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/flavor-root-disk-none.html

Earlier   Later