Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-06-18
22:28:44 Sundar OK. Thanks, efried and jaypipes. Take care!
22:28:46 efried Sundar: I'll try to catch up with dansmith and see if it was just a misunderstanding or what.
22:29:49 efried jaypipes: FYI, this is the flow that dansmith and I talked through the other day: http://logs.openstack.org/45/575545/1/check/build-openstack-sphinx-docs/77f4d6a/html/specs/rocky/approved/nova-cyborg-flow.html
22:30:03 efried jaypipes: It closely follows the neutron port model with os-vif.
22:30:54 jaypipes jroll: k, no prob. was just curious if we should make a specific change.
22:32:49 jaypipes efried: I don't see *end users* ever doing `openstack acc create --type GPU <specs>`. Maybe some admin... maybe. though that said, I don't really know what <specs> would be.
22:33:11 jaypipes efried: end users (at least that I'm aware of) don't have the ability to create "accelerators".
22:33:12 efried jaypipes: same thing as whatever it would be in the flavor, more or less.
22:33:31 jaypipes efried: I guess I don't see why we need a separate service for that, but ok.
22:33:33 efried Okay, if this is all driven by the flavor, so be it.
22:33:58 efried jaypipes: If we don't, we don't. I've never been a huge fan of the `port create` pre-step.
22:34:34 efried I guess the difference is that every port needs to have specific *unique* config info (IP addr).
22:34:46 efried ...which doesn't make sense to be in a flavor.
22:36:25 jaypipes efried: we're talking about different things entirely.
22:37:42 jaypipes efried: I'll just leave this conversation like this... I don't see why Cyborg isn't the generic device manager...
22:37:56 efried I fully hope it will be
22:38:03 jaypipes in other words, I don't see the point of having `openstack acc $ANYTHING`.
22:38:20 efried I'm on board with that, if dansmith is.
22:38:46 jaypipes there should be a YAML inventory file describing devices (that can't for some reason be auto-discovered on a node) and that's that...
22:39:23 efried The way the conversation with Sundar started was that his specs talk about os-acc parallelling os-vif, but then doesn't actually do that.
22:39:50 jaypipes note that I didn't say that os-acc shouldn't exist.
22:40:40 efried I wanted os-acc to have Acc* objects (corresponding to VIF*) and Acc*Profile (=~ VIF*PortProfile); and to define plug/unplug methods which call to plugins registered/loaded stevedore-wise.
22:40:49 efried they didn't have any of that in their specs.
22:41:48 jaypipes efried: but I don't see the point of a Cyborg REST API. sorry if that offends anyone. I just really only see the point of a utility library (os-acc) that can be called to configure/init a particular device slot.
22:42:20 efried ...where os-acc has a plugin mechanism that in turn lets it call out to platform-specific code to do plugging etc.
22:42:31 jaypipes sure, that's fine with me.
22:42:55 efried I imagine the REST API may be useful for things like programming FPGAs or twiddling details in their database.
22:43:14 efried but Nova isn't going to touch it, as you say, because all Nova needs to interact with is os-acc.
22:43:46 efried now it's possible that os-acc will talk to the cyborg API.
22:43:48 efried isn't it?
22:43:50 jaypipes efried: nobody is going to be "prgramming FPGAs" via a REST API. :)
22:44:25 jaypipes efried: sure, I guess I'm fine with this sort of thing being in the Cyborg API: https://github.com/openstack/cyborg/blob/master/cyborg/api/controllers/v1/deployables.py
22:44:46 jaypipes efried: what I *don't* think should be in the Cyborg API is this: https://github.com/openstack/cyborg/blob/master/cyborg/api/controllers/v1/accelerators.py
22:44:55 jaypipes efried: because all that is is inventory and resource providers information.
22:45:05 efried jaypipes: Just meaning in the same sense that we have a REST API to create a server by identifying the UUID of an image to load up; cyborg would have an API to identify the UUID of a bitstream image to program onto the thing.
22:45:06 jaypipes efried: and that belongs in placement, IMHO.
22:45:08 efried is that off base?
22:45:37 jaypipes efried: no. https://github.com/openstack/cyborg/blob/master/cyborg/api/controllers/v1/accelerators.py <-- that is the equivalent of compute_nodes in Nova, not instances table.
22:46:28 efried I was still talking about "programming FPGAs via a REST API".
22:46:34 jaypipes oh.
22:47:05 jaypipes efried: sure, that's fine. that's the /deployables REST API endpoint in Cyborg.
22:52:28 efried jaypipes: The other thing dansmith talked about was this need for a "bind" step. This is after allocate but before plug, and (I guess) lets cyborg know the actual RP we're grabbing. I think the purpose is so that cyborg can do their on-the-fly inventory/trait adjustment or perhaps final checking stuff? Not sure why this couldn't happen at the front of plug() before the plugin is invoked.
22:53:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Wait for network-vif-plugged before starting live migration https://review.openstack.org/558001
22:53:36 dansmith efried: the point of bind would be to get the inventory adjustment step and start programming the thing as early as we know where it's going to be
22:53:36 openstackgerrit Matt Riedemann proposed openstack/nova master: Add check if neutron "binding-extended" extension is available https://review.openstack.org/523548
22:53:37 mriedem watch out
22:53:37 openstackgerrit Matt Riedemann proposed openstack/nova master: Add "bind_ports_to_host" neutron API method https://review.openstack.org/523604
22:53:38 openstackgerrit Matt Riedemann proposed openstack/nova master: Add "delete_port_binding" network API method https://review.openstack.org/552170
22:53:39 openstackgerrit Matt Riedemann proposed openstack/nova master: Add "activate_port_binding" neutron API method https://review.openstack.org/555947
22:53:40 openstackgerrit Matt Riedemann proposed openstack/nova master: Delete port bindings in setup_networks_on_host if teardown=True https://review.openstack.org/556333
22:53:41 openstackgerrit Matt Riedemann proposed openstack/nova master: Implement migrate_instance_start method for neutron https://review.openstack.org/556334
22:53:42 openstackgerrit Matt Riedemann proposed openstack/nova master: Add VIFMigrateData object for live migration https://review.openstack.org/515423
22:53:43 openstackgerrit Matt Riedemann proposed openstack/nova master: Add VIFMigrateData.get_dest_vif https://review.openstack.org/566931
22:53:44 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: factor out pre_live_migration plug_vifs call https://review.openstack.org/566932
22:53:45 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: use dest host port bindings during pre_live_migration https://review.openstack.org/566933
22:53:46 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: use dest host vif migrate details for live migration https://review.openstack.org/551370
22:53:48 openstackgerrit Matt Riedemann proposed openstack/nova master: Annotate flows and handle PortBindingDeletionFailed in ComputeManager https://review.openstack.org/551371
22:53:49 openstackgerrit Matt Riedemann proposed openstack/nova master: Port binding based on events during live migration https://review.openstack.org/434870
22:53:50 openstackgerrit Matt Riedemann proposed openstack/nova master: conductor: use port binding extended API in during live migrate https://review.openstack.org/522537
22:53:58 mriedem gibi: thanks for the reviews ^
22:54:10 efried dansmith: I don't think we have the actual "VF" at that point, though.
22:54:26 efried dansmith: Any case, is there a reason that couldn't be done at the front of plug() before the plugin's plug() is invoked?
22:54:36 dansmith efried: we have the host, and cyborg could decide which "VF" it's going to give away
22:54:47 efried cyborg doesn't decide that - the plugin does.
22:55:10 dansmith efried: no, but it just serializes things that are parallelizeable, and makes us wait later to actually let cyborg adjust its inventory if it needs to
22:55:25 dansmith which reduces the window where we might've promised two things that can't actually co-exist
22:56:03 efried dansmith: And, since you're here now, do you agree that there's no `openstack accelerator create` thing to get an attachment handle - because the accelerator details are going to be in the flavor?
22:56:13 dansmith I really wish I wasn't here, I'm supposed to be EOD
22:56:25 efried Then just agree, and you can go free.
22:57:09 dansmith I dunno, I thought that if we don't allow them to create via cyborg (like creating a port via neutron) we're not going to be able to represent all the extra shit they want, and/or have to pass it all through our API
22:58:03 efried If by "pass it all through" you mean it piggybacks on the flavor and the allocation record in the RequestSpec, then... yeah.
22:58:10 dansmith no,
22:58:24 dansmith I mean if we don't let them do anything other than set stuff in the flavor,
22:58:59 dansmith but that doesn't provide enough detail to say things like "I want SSLv3 offload accelerator vs. I want generic crypto accelerator" or something
22:59:15 dansmith if it's only ever going to be "I want N of $foo" then flavor is enough,
22:59:18 dansmith but it's never that
22:59:31 efried heh
22:59:35 dansmith like neutron, it was just network,
22:59:48 dansmith but then it became mac, and qos, and subnet, and ip and....
22:59:53 efried well, I guess we can do it one way and then add the other way later. That's never a problem.
23:03:17 eandersson Is there a way to do live migration one-by-one in Mitaka? e.g. using some external script or similar?
23:04:05 eandersson with NUMA support
23:04:25 efried eandersson: I imagine you could do some kind of
23:04:25 efried for uuid in `nova list --params`; do nova migrate $uuid blah; done
23:04:25 efried But I suspect that's not what you're asking...
23:04:47 eandersson Ah yea, sorry left out an important detail about NUMA :p
23:05:44 eandersson Trying to figure out if there is a one-off script we could use to manually fix the NUMA calculation after live migrating VMs
23:06:48 efried eandersson: Ask stephenfin in about 12h
23:07:08 efried sean-k-mooney[m] may also be able to help, but he's also UTC.
23:07:20 efried cfriesen possibly?
23:08:27 eandersson Thanks efried
23:08:30 cfriesen eandersson: one issue is that you may not actually succeed in live migrating
23:09:12 cfriesen eandersson: since in the general case you need to recalculate any dedicated CPUs, guest-to-host NUMA mapping, etc
23:11:47 cfriesen If you could arrange things such that you have an empty compute node that is identical to the one you want to migrate off of, that would simplify things since they could all keep the same resources.
23:12:50 cfriesen eandersson: ^ (I suppose it would also work if dest compute node was larger than the source.)
23:20:02 eandersson cfriesen, yea we were thinking of disabling a compute node, and then live migrating to it
23:20:19 eandersson to avoid raceconditions etc
23:23:08 eandersson most of these vms are probably gonna take up a single numa zone, with two zones per host
23:30:09 openstackgerrit Eric Fried proposed openstack/os-vif master: doc: Fix arg specs and object types in docs https://review.openstack.org/575151
23:32:52 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Fixed auto-convergence option name in doc https://review.openstack.org/576282

Earlier   Later