| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-14 | |||
| 14:50:33 | feeblepipes | jangutter: what I'd like to see is a description of those features of the virtual switch technology that we could use instead of specific PCI vendor_id/product_id information. | |
| 14:52:00 | jangutter | Yep, so after a discussion yesterday with sean-k-mooney, it looks like it should be possible to push that check out to the OS-VIF side. | |
| 14:52:34 | feeblepipes | jangutter: another way to think about things... what is the difference between network_model.VNIC_TYPE_AGILIO_PASSTHROUGH and network_model.VNIC_TYPE_DIRECT or, frankly, network_model.VNIC_TYPE_DIRECT_PHYSICAL? | |
| 14:54:22 | jangutter | feeblepipes: Those two AGILIO VNIC types were added to in an effort not to re-use internal OpenStack code: they should go completely away. | |
| 14:56:00 | jangutter | feeblepipes: The idea is to have all the necessary whitelist info sent down via the os-vif port profile, so that the API will do matching. | |
| 14:57:24 | feeblepipes | jangutter: ok. that's cool with me. How exactly, however, do you plan to keep vendor_id/product_id information out of the os-vif modeling? In other words, what I'm looking for is an indication that you've thought of how to communicate the *functionality* that Agilio SmartNICs provide to the end user instead of passing a PCI vendor_id/product_id :) | |
| 14:58:59 | jangutter | feeblepipes: That's done in a couple of ways: a deployer runs a script that generates a nova.conf fragment, this adds the VF's to the compute node with in a full pci-whitelist format. | |
| 14:59:37 | feeblepipes | jangutter: sure, understood. that merely lets nova-compute "use" the Agilio NICs for tenants. | |
| 15:00:08 | jangutter | feeblepipes: Secondly, the OS-VIF object for the external plugin gets a set of fields. Likely this would be vendor/productids, we'll definitely have to coordinate with Mellanox on that. | |
| 15:00:47 | jangutter | feeblepipes: so when the API code at that place hits, the port profile is available, and the pci devices has been registered. | |
| 15:01:06 | feeblepipes | jangutter: right, so that's the thing I'd be -1 on :) we should definitely work with Moshe and others from Mellanox to ensure vendor_id/product_id information isn't needed to be passed via os-vif models. | |
| 15:03:45 | feeblepipes | jangutter: what I'm looking for is an object model that says "hey, I need a VIF plug that supports feature X, Y, and Z", not an object model that says "hey, I need a VIF plug that is specific to Mellanox Indigo NPS" | |
| 15:03:52 | feeblepipes | jangutter: that make sense? | |
| 15:03:53 | jangutter | feeblepipes: hmmm... that's interesting... you just made me think quite a lot... | |
| 15:04:34 | jangutter | feeblepipes: that would basically mean a "capability set" sort-of? | |
| 15:04:40 | feeblepipes | jangutter: BINGO :) | |
| 15:05:36 | feeblepipes | jangutter: ala the traits/capabilities that have begun to emerge here: https://review.openstack.org/#/c/448283/4 | |
| 15:06:17 | jangutter | feeblepipes: I can understand the appeal, but WOW, the parameter space is _big_. | |
| 15:06:29 | feeblepipes | jangutter: so, as an example, the Neutron port profile and os-vif object model would, say, have a list of these acceleration traits: https://review.openstack.org/#/c/448283/4/os_traits/hw/nic/accel.py | |
| 15:06:42 | feeblepipes | that would be required to match to an available PCI device on the target host. | |
| 15:07:52 | feeblepipes | jangutter: and Netronome and Mellanox would add a set of traits/features that their NICs supported to https://review.openstack.org/#/c/448283/4/os_traits/hw/nic/sriov.py | |
| 15:07:58 | jangutter | feeblepipes: I'm just worried that the acceleration traits would not dovetail much. For example, our NIC's got an insane amount of flexibility. The same HW can support many different features. | |
| 15:09:49 | jangutter | feeblepipes: our internal lingo for the firmware running on the chip is "application" - the image I get when l think of negotiating capabilities is some insane duel. | |
| 15:10:11 | feeblepipes | jangutter: an end user, though, only wants to launch a VM on a host that has a NIC with certain features. an end user doesn't say "hey, I want a VM with a Netronome Agilio SmartNIC" (because, frankly, the end user doesn't *know* that the underlying hardware has Agilio NICs). Instead, the end user says "hey, give me a VM that has access to an SR-IOV virtual function that has a particular set of features" | |
| 15:11:05 | jangutter | feeblepipes: Ah, I've been confusing operators with end-users. | |
| 15:11:47 | feeblepipes | jangutter: no worries. it's a common NFV problem :) in NFV land, there is no end user. The only user of the system is a privileged administrator user embodied in the MANO/VNFM system. | |
| 15:12:50 | feeblepipes | jangutter: but Nova is for end users -- the people/systems that launch VMs. So, we're constantly trying to shape the new NFV-specific features into a world that "normal users" can understand and utilize via the Nova APIs. | |
| 15:12:57 | jangutter | feeblepipes: OK, let's take the current flat/vlan SR-IOV mechanism as an example. An end user creates a port on a particular network and specs "it should be direct". (I think "direct" tends to go with VF's and "direct-physical" tend to go with PFs). | |
| 15:13:17 | feeblepipes | ok, continue | |
| 15:13:47 | jangutter | feeblepipes: if there are a mix of NICs then some instances would pop up with one type of driver required, and others would pop up with another, correct? | |
| 15:15:08 | jangutter | feeblepipes: It should be possible to do some flavor/image filtering or something, to match the flavors with the drivers they support? | |
| 15:16:06 | feeblepipes | jangutter: if there are a mix of NICs, they should be differentiated by their feature set, not by their vendor_id/product_id. | |
| 15:17:18 | feeblepipes | jangutter: so, for example, if a user launches a VM with nova boot --net port_id=<PORT_UUID> and that port has binding information that constructs a PCI request, the stuff in that PCI request should be a feature set, not a vendor_id/product_id. That's what I'm getting at :) | |
| 15:17:57 | jangutter | feeblepipes: with iommu passthrough, the instances get a raw PCI device, with the vendor/product exposed. I've been looking at that tuple as "a feature" :-p | |
| 15:18:00 | feeblepipes | jangutter: and that feature set is what should be used by the scheduler and the nova-compute pci device manager to match a particular NIC among many on the host. | |
| 15:18:30 | feeblepipes | jangutter: what about having "iommu_passthrough" as the feature, then? | |
| 15:19:44 | jangutter | feeblepipes: well, iommu_passthrough won't tell us anything of the device type that's going to appear inside the VM. | |
| 15:19:46 | feeblepipes | jangutter: would an end-user say "I want a vNIC that is associated to a VF that has IOMMU passthrough features" or would the end user say "I want a vNIC that is associated with a VF that is a Netronome Agilio SmartNIC PF"? | |
| 15:23:10 | jangutter | feeblepipes: my view of one usecase is this: an end-user says "I want a vNIC, associated with a VF, passed through via IOMMU, with <insert-hw-driver-here> PCI driver, additionally pin everything for me optimally, pretty please!" | |
| 15:23:29 | feeblepipes | :) | |
| 15:23:42 | feeblepipes | jangutter: ok, cool. we're getting somewhere here :) | |
| 15:23:56 | jangutter | feeblepipes: with DPDK applications being specifically compiled for specific PCI drivers, it's sometimes a _must_. | |
| 15:24:11 | feeblepipes | jangutter: the <insert-hw-driver-here> goes in the *image metadata*, though. | |
| 15:25:04 | feeblepipes | jangutter: because the image metadata is what describes the workload to be launched. that image metadata can describe the HW driver's installed on the image, and that information can/should be used by os-vif in setting up the VIF plug on the host. | |
| 15:25:07 | jangutter | feeblepipes: yeah, but that leads to the other bit of fun. Give me one vNIC on control, standard TAP, and vNIC 2 for my DPDK NFV application. | |
| 15:25:14 | feeblepipes | jangutter: so I think we're *almost* there. | |
| 15:25:46 | jangutter | feeblepipes: is it possible to spec stuff that fine-grained? | |
| 15:26:02 | feeblepipes | jangutter: ah, indeed. so perhaps that information (hw driver) does need to be in the port binding profile. | |
| 15:26:18 | feeblepipes | jangutter: yes, it is possible. via the port profile information. | |
| 15:26:34 | feeblepipes | jangutter: I still don't think we need to have vendor_id/product_id in there, though. :) | |
| 15:26:54 | feeblepipes | jangutter: in any case... I think this is also something we can hash out in the code rather than the spec. | |
| 15:27:33 | jangutter | feeblepipes: I am definitely not averse to having a "driver list". That's actually a pretty good idea from a user's perspective. | |
| 15:27:34 | feeblepipes | jangutter: I'm willing to +2 the spec on condition that you just place a note in the spec that we've discussed on IRC how to make the information passed between Neutron and Nova via os-vif as generic as possible. | |
| 15:27:50 | feeblepipes | jangutter: if you can do that quick, I'll +2 it and mriedem can as well. | |
| 15:30:53 | mriedem | sfinucan: how do i reference another link within the same docs using rst again? | |
| 15:31:08 | mriedem | like i want to link to a section in the nova-manage man page from the nova-cells man page | |
| 15:31:43 | jangutter | I shall definitely do so: to summarize: It is undesirable to send the VF selection parameters to Nova as PCI vendor/product ID, further work should be done to develop a capability or driver list that would abstract this away from users. | |
| 15:32:01 | mriedem | oh maybe :ref: | |
| 15:32:11 | efried | mriedem Looks like that kolla thing got sorted. Can you throw me a link to whatever dashboard/view I should be using, so I can maybe pick off some lhf? | |
| 15:32:42 | mriedem | efried: not sorted yet, docs change coming soon | |
| 15:32:55 | efried | I meant triaged | |
| 15:33:05 | efried | That's the goal of the current sweep, right? | |
| 15:33:23 | efried | Not to resolve, but to get 'em out of 'New'? | |
| 15:33:44 | mriedem | efried: yeah | |
| 15:33:54 | mriedem | like 'unexpected api error' is usually always invalid | |
| 15:33:58 | mriedem | and a misconfiguration | |
| 15:34:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add release note and update cell install guide for multi-cell limitations https://review.openstack.org/456923 | |
| 15:34:12 | mriedem | melwitt: ^ | |
| 15:35:40 | mriedem | EmilienM: owalsh: ^ | |
| 15:38:26 | efried | This thing about having to run a new discovery command to make stuff work... Could it not just be made to work out of the box if you only have one cell? | |
| 15:47:49 | efried | [irony: when --help doesn't automatically page, and you have to say --help|less] | |
| 15:50:14 | mriedem | efried: we have a config option if you want things discovered automatically, | |
| 15:50:20 | mriedem | but it's a bit heavy for large deployments | |
| 15:50:27 | mriedem | dan has a series up to improve that | |
| 15:50:32 | jangutter | Is there a way to refer to today's IRC chat in a spec? | |
| 15:50:34 | efried | mriedem Okay, neat. | |
| 15:50:40 | mriedem | jangutter: yeah, esc | |
| 15:50:41 | mriedem | *Sec | |
| 15:50:43 | efried | http://eavesdrop.openstack.org/irclogs/%23openstack-nova/ | |
| 15:50:47 | efried | jangutter ^^ | |
| 15:50:48 | mriedem | efried: https://blueprints.launchpad.net/nova/+spec/discover-hosts-faster | |
| 15:51:03 | mriedem | jangutter: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-04-14.log.html#t2017-04-14T14:44:06 | |
| 15:52:24 | jangutter | Whatever you write is now memorialized forever. Or until cats reclaim their rightful place on top of the food chain. | |
| 15:54:10 | mriedem | our cat can easily be defeated with a bag of chips | |
| 15:54:47 | openstackgerrit | Jan Gutter proposed openstack/nova-specs master: Spec to enable Netronome SmartNIC support https://review.openstack.org/456386 | |
| 15:57:26 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Tell people that the nova-cells man page is for cells v1 https://review.openstack.org/456925 | |
| 15:57:29 | jangutter | Another newbie question. I've replied to a bunch of comments on an older review. They're in "draft" in gerrit now, am I too late? | |
| 15:57:41 | mriedem | jangutter: no, you can still publish those | |
| 15:57:43 | efried | jangutter No - go to that patch set and they'll show up. | |
| 15:58:15 | efried | Still not entirely understanding why a compute node can't automatically 'discover' itself into a cell if there's only one cell... | |
| 15:58:31 | mriedem | efried: it would require an upcall from the compute to the api layer, which we don't want | |
| 15:58:35 | mriedem | upcalls are the devil | |
| 15:58:43 | mriedem | s/the/a/ | |
| 15:58:53 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Add Cinder v3 detach call to _terminate_volume_connections https://review.openstack.org/456896 | |
| 15:59:12 | mriedem | efried: also, that would require either an upcall or putting the nova api db credentials in the compute config | |
| 15:59:16 | mriedem | another thing we don't want | |
| 16:00:20 | efried | mriedem And all of that is different from how a compute node registered itself B.C. (before cells)? | |
| 16:00:52 | mriedem | yes | |
| 16:00:58 | mriedem | in the before times, | |
| 16:01:01 | mriedem | there was no api db | |