Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-06-18
22:22:11 efried Sundar: Nope, placement knows what it's told. I need to tell it about more than just "I need one VGPU". Otherwise I have to do all my filtering later, which defeats the purpose of having used placement.
22:22:30 Sundar "conductor needs to draw up a placement query for allocation candidates, it asks cyborg (via os-acc?)" This is not needed for selecting a node
22:22:43 efried yes it is.
22:22:54 Sundar You can do GET /a-c without talking to Cyboirg
22:22:55 efried unless, as I say, you want to do all the filtering later, which defeats the purpose.
22:23:35 efried Sundar: n-cond had to get the config details of the accelerator from *somewhere*.
22:23:44 efried Sundar: Where did it get that from?
22:23:48 Sundar E.g. resources:ACCELERATOR_GPU=1; traits:RESOLUTION-1024-768=required (or whatever)
22:24:12 efried right, where does traits:RESOLUTION_1024X768=required come from?
22:24:27 Sundar CYborg (or whatever) published it
22:24:28 efried Sundar: Oh, this may be the source of confusion: are you thinking that comes from the flavor?
22:24:34 Sundar Yes
22:25:04 efried jaypipes, dansmith: Remind me why we're not putting accelerator stuff in flavors
22:25:22 efried (which I suppose may be the same reason we don't put neutron port stuff in flavors??)
22:25:28 jaypipes efried: we are putting that stuff in flavors and image metadata (at least, we are for *v*GPUs...)
22:26:25 Sundar jaypipes: +1 Same for FPGAs
22:26:39 efried well then
22:26:40 jaypipes efried: that, of course, is what is being *requested* by the user. something (Cyborg I was assuming) needs to decorate resource providers representing the physical accelerators with those traits.
22:26:57 Sundar Totally agreed
22:26:58 efried yeah, got that
22:27:01 jaypipes k
22:27:17 Sundar Great. I need to drop now for a 3:30 PM call
22:27:31 Sundar I am open to coming back tomorrow (or even later today) to close this
22:27:33 efried We need to get a consensus with dansmith.
22:27:46 Sundar OK, what's the best time for that?
22:27:58 efried Sundar: You don't need to be around for that, necessarily.
22:27:58 jroll jaypipes: I don't have specific advice, I was just curious what clint was referring to and then tried to help clarify :)
22:28:44 Sundar OK. Thanks, efried and jaypipes. Take care!
22:28:46 efried Sundar: I'll try to catch up with dansmith and see if it was just a misunderstanding or what.
22:29:49 efried jaypipes: FYI, this is the flow that dansmith and I talked through the other day: http://logs.openstack.org/45/575545/1/check/build-openstack-sphinx-docs/77f4d6a/html/specs/rocky/approved/nova-cyborg-flow.html
22:30:03 efried jaypipes: It closely follows the neutron port model with os-vif.
22:30:54 jaypipes jroll: k, no prob. was just curious if we should make a specific change.
22:32:49 jaypipes efried: I don't see *end users* ever doing `openstack acc create --type GPU <specs>`. Maybe some admin... maybe. though that said, I don't really know what <specs> would be.
22:33:11 jaypipes efried: end users (at least that I'm aware of) don't have the ability to create "accelerators".
22:33:12 efried jaypipes: same thing as whatever it would be in the flavor, more or less.
22:33:31 jaypipes efried: I guess I don't see why we need a separate service for that, but ok.
22:33:33 efried Okay, if this is all driven by the flavor, so be it.
22:33:58 efried jaypipes: If we don't, we don't. I've never been a huge fan of the `port create` pre-step.
22:34:34 efried I guess the difference is that every port needs to have specific *unique* config info (IP addr).
22:34:46 efried ...which doesn't make sense to be in a flavor.
22:36:25 jaypipes efried: we're talking about different things entirely.
22:37:42 jaypipes efried: I'll just leave this conversation like this... I don't see why Cyborg isn't the generic device manager...
22:37:56 efried I fully hope it will be
22:38:03 jaypipes in other words, I don't see the point of having `openstack acc $ANYTHING`.
22:38:20 efried I'm on board with that, if dansmith is.
22:38:46 jaypipes there should be a YAML inventory file describing devices (that can't for some reason be auto-discovered on a node) and that's that...
22:39:23 efried The way the conversation with Sundar started was that his specs talk about os-acc parallelling os-vif, but then doesn't actually do that.
22:39:50 jaypipes note that I didn't say that os-acc shouldn't exist.
22:40:40 efried I wanted os-acc to have Acc* objects (corresponding to VIF*) and Acc*Profile (=~ VIF*PortProfile); and to define plug/unplug methods which call to plugins registered/loaded stevedore-wise.
22:40:49 efried they didn't have any of that in their specs.
22:41:48 jaypipes efried: but I don't see the point of a Cyborg REST API. sorry if that offends anyone. I just really only see the point of a utility library (os-acc) that can be called to configure/init a particular device slot.
22:42:20 efried ...where os-acc has a plugin mechanism that in turn lets it call out to platform-specific code to do plugging etc.
22:42:31 jaypipes sure, that's fine with me.
22:42:55 efried I imagine the REST API may be useful for things like programming FPGAs or twiddling details in their database.
22:43:14 efried but Nova isn't going to touch it, as you say, because all Nova needs to interact with is os-acc.
22:43:46 efried now it's possible that os-acc will talk to the cyborg API.
22:43:48 efried isn't it?
22:43:50 jaypipes efried: nobody is going to be "prgramming FPGAs" via a REST API. :)
22:44:25 jaypipes efried: sure, I guess I'm fine with this sort of thing being in the Cyborg API: https://github.com/openstack/cyborg/blob/master/cyborg/api/controllers/v1/deployables.py
22:44:46 jaypipes efried: what I *don't* think should be in the Cyborg API is this: https://github.com/openstack/cyborg/blob/master/cyborg/api/controllers/v1/accelerators.py
22:44:55 jaypipes efried: because all that is is inventory and resource providers information.
22:45:05 efried jaypipes: Just meaning in the same sense that we have a REST API to create a server by identifying the UUID of an image to load up; cyborg would have an API to identify the UUID of a bitstream image to program onto the thing.
22:45:06 jaypipes efried: and that belongs in placement, IMHO.
22:45:08 efried is that off base?
22:45:37 jaypipes efried: no. https://github.com/openstack/cyborg/blob/master/cyborg/api/controllers/v1/accelerators.py <-- that is the equivalent of compute_nodes in Nova, not instances table.
22:46:28 efried I was still talking about "programming FPGAs via a REST API".
22:46:34 jaypipes oh.
22:47:05 jaypipes efried: sure, that's fine. that's the /deployables REST API endpoint in Cyborg.
22:52:28 efried jaypipes: The other thing dansmith talked about was this need for a "bind" step. This is after allocate but before plug, and (I guess) lets cyborg know the actual RP we're grabbing. I think the purpose is so that cyborg can do their on-the-fly inventory/trait adjustment or perhaps final checking stuff? Not sure why this couldn't happen at the front of plug() before the plugin is invoked.
22:53:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Wait for network-vif-plugged before starting live migration https://review.openstack.org/558001
22:53:36 dansmith efried: the point of bind would be to get the inventory adjustment step and start programming the thing as early as we know where it's going to be
22:53:36 openstackgerrit Matt Riedemann proposed openstack/nova master: Add check if neutron "binding-extended" extension is available https://review.openstack.org/523548
22:53:37 mriedem watch out
22:53:37 openstackgerrit Matt Riedemann proposed openstack/nova master: Add "bind_ports_to_host" neutron API method https://review.openstack.org/523604
22:53:38 openstackgerrit Matt Riedemann proposed openstack/nova master: Add "delete_port_binding" network API method https://review.openstack.org/552170
22:53:39 openstackgerrit Matt Riedemann proposed openstack/nova master: Add "activate_port_binding" neutron API method https://review.openstack.org/555947
22:53:40 openstackgerrit Matt Riedemann proposed openstack/nova master: Delete port bindings in setup_networks_on_host if teardown=True https://review.openstack.org/556333
22:53:41 openstackgerrit Matt Riedemann proposed openstack/nova master: Implement migrate_instance_start method for neutron https://review.openstack.org/556334
22:53:42 openstackgerrit Matt Riedemann proposed openstack/nova master: Add VIFMigrateData object for live migration https://review.openstack.org/515423
22:53:43 openstackgerrit Matt Riedemann proposed openstack/nova master: Add VIFMigrateData.get_dest_vif https://review.openstack.org/566931
22:53:44 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: factor out pre_live_migration plug_vifs call https://review.openstack.org/566932
22:53:45 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: use dest host port bindings during pre_live_migration https://review.openstack.org/566933
22:53:46 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: use dest host vif migrate details for live migration https://review.openstack.org/551370
22:53:48 openstackgerrit Matt Riedemann proposed openstack/nova master: Annotate flows and handle PortBindingDeletionFailed in ComputeManager https://review.openstack.org/551371
22:53:49 openstackgerrit Matt Riedemann proposed openstack/nova master: Port binding based on events during live migration https://review.openstack.org/434870
22:53:50 openstackgerrit Matt Riedemann proposed openstack/nova master: conductor: use port binding extended API in during live migrate https://review.openstack.org/522537
22:53:58 mriedem gibi: thanks for the reviews ^
22:54:10 efried dansmith: I don't think we have the actual "VF" at that point, though.
22:54:26 efried dansmith: Any case, is there a reason that couldn't be done at the front of plug() before the plugin's plug() is invoked?
22:54:36 dansmith efried: we have the host, and cyborg could decide which "VF" it's going to give away
22:54:47 efried cyborg doesn't decide that - the plugin does.
22:55:10 dansmith efried: no, but it just serializes things that are parallelizeable, and makes us wait later to actually let cyborg adjust its inventory if it needs to
22:55:25 dansmith which reduces the window where we might've promised two things that can't actually co-exist
22:56:03 efried dansmith: And, since you're here now, do you agree that there's no `openstack accelerator create` thing to get an attachment handle - because the accelerator details are going to be in the flavor?
22:56:13 dansmith I really wish I wasn't here, I'm supposed to be EOD
22:56:25 efried Then just agree, and you can go free.
22:57:09 dansmith I dunno, I thought that if we don't allow them to create via cyborg (like creating a port via neutron) we're not going to be able to represent all the extra shit they want, and/or have to pass it all through our API
22:58:03 efried If by "pass it all through" you mean it piggybacks on the flavor and the allocation record in the RequestSpec, then... yeah.
22:58:10 dansmith no,
22:58:24 dansmith I mean if we don't let them do anything other than set stuff in the flavor,

Earlier   Later