Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-09-05
15:40:27 avolkov efried: some meta group based on those properties? sounds good
15:40:41 efried This gets a little weird in flavor land, though, cause you'd need a separate flavor for each group I think.
15:41:16 efried So you would have one flavor that says "give me two VFs: one from Switch 1 + Group 1; one from Switch 2 + Group 1"
15:41:27 efried Then another that says "give me two VFs: one from Switch 1 + Group 2; one from Switch 2 + Group 2"
15:41:55 efried ...and somehow alternate which one you use to do spawns, so you get saturation of the different ports.
15:42:09 efried Using P1/P2/P3/P4 you have that same problem, only twice as bad :)
15:42:40 avolkov efried: yeah :), it's another question I wanted to ask
15:42:52 openstack Launchpad bug 1715182 in OpenStack Compute (nova) "_rollback_live_migration does not remove allocations from destination node" [High,Triaged]
15:42:52 mriedem johnthetubaguy: here you go https://bugs.launchpad.net/nova/+bug/1715182
15:42:56 edmondsw efried what does the group signify that knowing the switch doesn't tell you?
15:43:16 edmondsw i.e., isn't the switch tag essentially a group tag?
15:43:20 efried no
15:43:50 edmondsw oh, I guess you could have the switches wired differently
15:44:38 efried (sorry, got an interrupt, gimme a few mins...)
15:47:59 efried okay, so avolkov that's a point: how many VFs do you want?
15:48:04 edmondsw I guess what I'm getting at is that you shouldn't need to know switch or port... just groups
15:48:11 efried avolkov If you want four, then yeah, you only need one flavor.
15:48:14 openstackgerrit Merged openstack/os-vif master: Add plugin names as constants. https://review.openstack.org/500111
15:49:24 efried "give me four VFs: Switch1 x Group1; Switch1 x Group2; Switch2 x Group1; Switch2 x Group2"
15:49:30 mriedem dtantsur: i've started a rough agenda at the bottom of this etherpad https://etherpad.openstack.org/p/nova-ptg-queens
15:49:38 mriedem penciled in ironic for 3pm on wed
15:50:03 edmondsw efried why wouldn't that be (give me 2 from group 1 and 2 from group 2)?
15:50:08 dtantsur cool, lemme check our schedule
15:50:25 efried edmondsw Because then you might get both from the same switch in group 1
15:51:02 edmondsw efried not if they defined the groups properly...
15:51:10 mriedem sdague: lyarwood: stable/pike backport for a novaclient thing regressed since 9.0.0 https://review.openstack.org/#/c/495901/
15:51:10 dtantsur mriedem: do you remember when we have lunch?
15:51:15 dtantsur is it right after or...?
15:51:20 mriedem dtantsur: i was told 12-1
15:51:24 dtantsur thnx
15:51:37 efried edmondsw How can you define two groups to ensure you get four separate ports across two separate switches?
15:51:43 mriedem 3pm wednesday would be *after* lunch :)
15:52:52 edmondsw group 1 = one port to sw1 and one to sw2, group 2 = one port to sw1 and one to sw2, then ask for 2 ports in each group
15:53:49 efried edmondsw But placement doesn't know enough to not give you both VFs from group 1 on the same switch.
15:54:18 edmondsw efried there aren't 2 ports in group 1 with the same switch
15:54:35 efried Yes, there are multiple VFs on each pport.
15:54:44 edmondsw oh, VFs...
15:55:06 edmondsw I gotcha now
15:55:07 efried And yes, you could conceivably do this same thing with four groups - but enumerating switches might make more sense to the user; and you also want the model to extend to >2 switches, >2 ports per switch.
15:55:29 dtantsur mriedem: okay, sounds good
15:56:16 efried although... avolkov that might actually make more sense. If you always know you want four VFs, you could just tag your PFs in groups so they'll always be spread out.
15:56:20 edmondsw efried how about group 1 = 2 ports to sw1 and group 2 = 2 ports to sw2?
15:56:55 efried edmondsw Then again you'll ask for two VFs from group 1 and they might wind up coming from the same pport
15:57:17 edmondsw yep, k... better for PFs but doesn't help with VFs
15:57:30 efried eh?
15:57:41 edmondsw nm... it doesn't work, so it doesn't work :)
15:58:16 edmondsw I'm not following why you wouldn't tag them with the port, then
15:58:29 efried avolkov In your email example, it's no different than having labeled P1,P2,P3,P4 - but extending to more than four pports (or reducing the problem set to fewer than 4 desired VFs) it makes more sense to think of groups - where the total number of groups is the number of VFs you're going to want from a single allocation request.
15:58:34 efried edmondsw ^^
16:00:12 avolkov efried: groups are okay if you have the same requirements for each boot request
16:01:17 efried avolkov Yeah, I get it. If they're different, then it makes more sense for each PF to have its own label, and you do your HA/anti-affinity by constructing your flavors appropriately.
16:01:17 avolkov efried: with original properties you can ask distinct ports for one boot request and distict switches for another
16:02:22 efried Problem with that, though, is if you don't have exactly the same number and configuration of SR-IOV cards on all your hosts.
16:02:39 edmondsw will it be possible to request ports on separate identical cards (HA if a card fails)?
16:03:05 efried So it probably makes more sense to *call* them groups anyway, even if they usually/always map to PFs :)
16:03:15 sean-k-mooney efried: i would prefer if we did not model this in flavor or image properties though
16:03:20 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Cleanup allocations on invalid dest node during live migration https://review.openstack.org/500908
16:03:20 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Add functional recreate test for live migration pre-check fails https://review.openstack.org/500907
16:03:28 efried sean-k-mooney Which "this"?
16:03:36 sean-k-mooney efried: really we should try to model the bond requiremetn as an atribute of the neutron port
16:03:46 sean-k-mooney vf selection policy
16:04:25 avolkov sean-k-mooney: +1 not to use flavors )
16:04:38 efried Yeah, that makes sense, sorry.
16:04:54 openstackgerrit melanie witt proposed openstack/nova master: Request zero root disk for boot-from-volume instances https://review.openstack.org/428481
16:04:55 openstackgerrit melanie witt proposed openstack/nova master: Claim and report zero root disk for boot-from-volume instances https://review.openstack.org/428505
16:05:06 efried but wait
16:05:25 efried wouldn't we like to be able to do a spawn with SR-IOV VFs in one command rather than two?
16:05:33 sean-k-mooney efried: did you see the section i added to the ptg etherpad https://etherpad.openstack.org/p/nova-ptg-queens lines 107-125
16:05:40 sean-k-mooney efried: no
16:06:01 sean-k-mooney efried: and yes but not via flavor
16:06:12 efried Is a port bound to a host?
16:06:38 sean-k-mooney if we can do it with one command via nova-boot sure but i dont whant to create a flavor multiple time with just different number of interfaes
16:06:55 sean-k-mooney efried: it is after nova selects the host
16:07:10 sean-k-mooney efried: before that it is just a logical port in a db
16:07:32 sean-k-mooney efried: as part of portbinding nova compute updates the neutron port with the host id
16:08:05 efried So you want to create the port with an anti-affinity/HA spec which specifies the number of VFs and how they should be spread out...
16:08:25 sean-k-mooney efried: yep
16:08:49 openstackgerrit Matt Riedemann proposed openstack/nova master: Skip more racy rebuild failing tests with cells v1 https://review.openstack.org/499001
16:08:55 efried ...and then we schedule to the host and spawn attaches the right number of VFs with the right distribution.
16:10:02 efried There's a big hand-wavey part in the middle there, though, where the scheduler was able to figure out which compute host(s) would be able to honor that request. Is the scheduler (and/or, gods forbid, the placement API) supposed to introspect the port metadata to help with that decision??
16:10:14 sean-k-mooney efried: yep see my comments in https://review.openstack.org/#/c/463526/ and https://review.openstack.org/#/c/182242/ to this effect
16:11:32 sean-k-mooney no basically before the scheduler starts scheduling today the neutron v2 client api in nova retrives the port from neutorn
16:12:06 sean-k-mooney if that port is vnic_type direct/macvtap or virtio-forwarder it create a new pcieresutespec object
16:12:49 sean-k-mooney we need to extend that to also read the ha spec and add that to the picerequeste spec so that when tha tis passed to the sceduler/placement it can fufille the requirementes for ha
16:13:19 sean-k-mooney this is what we have imlemented for the feature based scheduing also
16:13:20 efried Yeah, okay, so that's how it works today; but I thought we were trying to move away from that kind of special-casing as we get into placement.
16:13:48 sean-k-mooney efried: yes so in placement i would like to be able to express affinity and anti affintiy
16:14:02 sean-k-mooney so we would ask for 2 vf with pf antiaffinity
16:14:27 sean-k-mooney placement would filer host based on that and then the scheduler would make the final desision
16:15:28 efried Has jaypipes weighed in yet on how affinity/anti-affinity might be made to work with placement?
16:16:09 dansmith efried: distance
16:16:13 sean-k-mooney proably its not the first time i have mentioned this to him but not aware of his current stance
16:16:22 dansmith efried: as mentioned earlier this morning, but also quite a bit in boston during that session
16:16:53 edmondsw there are anti-affinity needs at multiple layers... 1) device, 2) PF, 3) switch...
16:17:24 sean-k-mooney edmondsw: yes i was hoping we could model the switch as a trait on the pf if that made sense?
16:17:41 efried Right, so basically placement would need a generic, multi-layer-capable distance/affinity mechanism, and then consumers could model as they see fit within that framework.
16:18:32 sean-k-mooney efried: yes ideally. its just up to the consumers to model the dependcies with traits and netested providers correctly
16:18:41 sean-k-mooney that is easier said then done however
16:19:00 sean-k-mooney ideally i would like to see a request for a bonded port that looked someting like this
16:19:01 efried sean-k-mooney Definitely makes sense for the switch to be a trait on the PF. Or for the switch to be a RP with its PFs nested underneath it. Either way would work. But if these affinity gizmos are separate from traits, then it probably doesn't matter as much how the RPs are nested.

Earlier   Later