Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-09
14:45:07 sean-k-mooney fried_rice: generally we say the nic has a numa affinity to numa node x
14:45:16 fried_rice sean-k-mooney: Thanks
14:46:22 sean-k-mooney fried_rice: what that actully means is for a DMA or MMIO capable device the memory adress allocated to that device are from the adress range associated to numa node X
14:46:33 fried_rice sean-k-mooney: Dig
14:48:08 fried_rice Is master Rocky now?
14:48:10 fried_rice mriedem: ^
14:48:23 sean-k-mooney cpus get numa affinity because they are phyically closer to one memory controler then the rest. so address form that controler(numa node) can be loaded into cache faster then form a remote numa node
14:48:45 fried_rice sean-k-mooney: Phew, that gels with my (admittedly extremely limited) understanding.
14:49:41 fried_rice sean-k-mooney: And for PCI devices, there's something about the bus that goes to that particular I/O slot that makes the same kind of communication faster.
14:50:53 sean-k-mooney kindof, each socket on a modern system(e.g. it does not have a front side bus) has a pci controler route
14:51:23 sean-k-mooney each pci slot is phyically wiered to a partcalar pci controer/socket
14:51:43 sean-k-mooney so the pci slots on a mother board have socket affintiy
14:52:11 sean-k-mooney with processors with only one memory controler they have numa afinity too
14:53:01 sean-k-mooney with processors with 2 memory controlers teh numa affinity is determined by what memory controler the kenerl allocates the mmio address form for that device
14:53:16 fried_rice Okay, I said 'bus' but I guess I meant 'socket'. I'm a hardware 'tard.
14:53:30 sean-k-mooney linux puts all pci devices on the first memory controler/numa node on a socket by default
14:55:17 sean-k-mooney fried_rice: working at intel i kindof have to know these things but its all a little archane
14:56:50 openstackgerrit melissaml proposed openstack/nova master: fix typos https://review.openstack.org/542866
15:14:15 mriedem fried_rice: yes master is rocky since we have a stable/queens branch now, however, we likely don't want to land any major changes that are big refactors or delete code until we've fully released queens, because of backports for rc2 if we need them
15:14:17 mriedem hrw: hi
15:14:40 fried_rice mriedem: Roger wilco, thanks.
15:17:54 mriedem melwitt: i've updated launchpad to point rocky as the dev focus, and created the rocky milestones https://launchpad.net/nova/rocky
15:18:05 mriedem based on https://releases.openstack.org/rocky/schedule.html#r-release
15:18:19 melwitt mriedem: cool, thank you
15:18:27 mriedem i'll create a queens-rc2 milestone if we end up having one https://launchpad.net/nova/queens/
15:25:06 mriedem lyarwood: want to hit https://review.openstack.org/#/c/542323/ ?
15:26:12 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: Remove 2 redundant methods https://review.openstack.org/528325
15:30:37 lyarwood mriedem: ack done
15:35:59 openstackgerrit Balazs Gibizer proposed openstack/nova master: Poison context usage in periodic tasks https://review.openstack.org/542891
15:39:40 openstackgerrit Balazs Gibizer proposed openstack/nova master: Poison context usage in periodic tasks https://review.openstack.org/542891
15:49:57 hrw mriedem: went through your comments and pushed update to https://review.openstack.org/#/c/541728/
15:50:23 hrw mriedem: one of our team members will check for me entries I skipped
15:51:05 leakypipes fried_rice: can you give me another 30 mintues?
15:51:13 fried_rice leakypipes: Okay.
15:51:20 fried_rice leakypipes: Whyncha ping me when ready?
15:51:33 leakypipes fried_rice: yeah, thanks man
15:55:31 openstackgerrit Merged openstack/nova master: Ensure the JSON-Schema covers the legacy v2 API https://review.openstack.org/540154
16:02:59 bauwser meh, one day without being upstream
16:03:01 bauwser sad.
16:07:14 leakypipes fried_rice: https://etherpad.openstack.org/p/going-the-distance
16:07:57 leakypipes fried_rice: I'm ready when you are.
16:08:27 fried_rice leakypipes: Okay, we gonna go through this or should I pre-read?
16:08:43 edleafe fried_rice: it'll take an hour to read :)
16:09:06 leakypipes fried_rice, edleafe: please spend some time to read and then let's get on a hangout?
16:09:16 fried_rice finucannot, sean-k-mooney, edleafe, cfriesen: Heads up, we're gonna do this thing soon.
16:09:20 fried_rice leakypipes: Roger that.
16:09:41 edleafe leakypipes: can I clear the pink color?
16:11:05 leakypipes edleafe: yessir, absolutely.
16:11:20 edleafe ah, much easier on the eyes
16:11:31 fried_rice edleafe: You need to make it solarized now.
16:11:35 leakypipes edleafe: don't like the pepto-pink? :)
16:12:07 edleafe black on color == :(
16:12:41 edleafe ok, I think I have the gist of it
16:13:54 sean-k-mooney edleafe: actully black on colour is easier to read on paper then black on white if you have dislexia but on pcs not so much. thats why they make tinted glasses for peopel with dislexia
16:14:20 leakypipes sean-k-mooney: sense makes that me to.
16:14:27 edleafe sean-k-mooney: good to know
16:14:46 leakypipes finucannot: that's the GOOD stuff, man...
16:14:52 edleafe finucannot: there's not much else :)
16:14:58 finucannot leakypipes: You have a problem, sir.
16:15:04 leakypipes finucannot: yes, well aware :)
16:15:19 sean-k-mooney finucannot: so you read the titles and nothing else :P
16:15:52 sean-k-mooney sql is precise
16:17:58 cfriesen are we still planning on using this for numa as well?
16:18:10 finucannot ^ that was my question
16:18:13 leakypipes cfriesen: yes
16:18:22 fried_rice leakypipes: I'm ready. Hangout?
16:18:29 leakypipes cfriesen: though... meh? :)
16:18:43 leakypipes cfriesen: it would be a giant headache for operators to do NUMA affinity this way.
16:19:13 leakypipes cfriesen: due to the pinning logic that needs to occur in the filtering, I'd just as likely recommend keeping the NUMATopologyFilter as-is in Nova scheduler side.
16:19:13 fried_rice oh. Well, that's a point in favor of the spec I'm writing up. It handles NUMA specifically, but not much else.
16:20:15 cfriesen leakypipes: why would operators need to do anything?
16:20:25 leakypipes cfriesen: the primary use cases I've had in my mind with this are the use cases around multi-failure-domain (HA) placement as well as affinity placement for groups of instances (with tolerances for relative distances between where those instances land)
16:21:02 leakypipes cfriesen: well, perhaps the operator wouldn't... maybe the virt driver could auto-create the aggregates and set distances.
16:21:09 leakypipes cfriesen: for numa nodes, etc
16:22:01 cfriesen leakypipes: makes sense. do you see us being able to drop the server group affinity/antiaffinity filters?
16:22:44 leakypipes cfriesen, finucannot, sean-k-mooney, fried_rice, edleafe: hmm, perhaps it wasn't obvious, but I am not hardcoding the whole "region", "rack", "row", "power-unit" "power-domain" entries in that distance_types table. those would be for operators to fill in what they liked.
16:23:10 fried_rice Which would allow for "numa-node" in theory.
16:23:25 finucannot Yeah, they're all...distance_codes?
16:23:32 cfriesen leakypipes: for the numa case we'd need something like "give me cpus and ram on the same numa node, and some more cpus and ram from a different numa node of the same host, and a pci device from the same host"
16:23:49 edleafe leakypipes: so they would have to define the distance codes, as well as the value for the distances?
16:24:42 leakypipes cfriesen: re: the server group filters... maybe. the concept that I'm espousing here does away with the concept of a server group entirely and relies on point-in-time policy from the caller of nova boot. so instead a scheduler hint for the instance group, the caller is specifying "I want to be near (for *this* definition of "near") this other instance..."
16:25:07 leakypipes edleafe: yep. but keep in mind those distances are simply relative, nothing more.
16:25:50 edleafe I would certainly like to get input from operators about this, then
16:25:50 leakypipes edleafe: we could of course provide tutorial/examples for common use cases.
16:25:52 sean-k-mooney leakypipes: yes i assumed that the operator would fill thoes in but a compute driver could also in some cases
16:26:07 fried_rice One thing that concerns me is how much the operator needs to know about how resource providers are modeled by the driver.
16:26:30 leakypipes fried_rice: in what way? could you elaborate?
16:26:44 edleafe fried_rice: I was just going to say that it seems that we are making the operators fit the solution rather than the other way around
16:27:20 fried_rice If the providers are modeled in tree fashion, the operator needs to know that so she can know which provider(s) to specify what distances for.
16:27:35 leakypipes edleafe: I'm not sure how any solution that didn't rely on the operator describing the relationship of their compute nodes to their groupings of compute nodes (i.e. placement aggregates) would work.
16:27:36 sean-k-mooney leakypipes: you konw the distance types you are proposing are similar to the crush map valuse used by default in ceph
16:27:42 leakypipes edleafe: I mean, there's so many variations...
16:27:42 fried_rice Which is different if they're modeled in *this* shape tree versus *that* shape tree.
16:28:57 edleafe leakypipes: I'm not saying that this isn't acceptable. I'm just saying that it's important to get operators input on whether this is acceptable.
16:29:11 leakypipes sean-k-mooney: I'm not familiar with ceph crush maps :( however, this distance modeling is kinda similar to the distance maps for CPU cores to different levels of cache in NUMA setups, for sure.
16:29:41 leakypipes edleafe: ack, totes.
16:29:43 fried_rice leakypipes: Can you give an example (in the etherpad) for the CLI syntax that would deploy an instance which gets its storage from a sharing provider "near" its compute host?
16:30:09 leakypipes fried_rice: sure, one sec.
16:30:52 fried_rice And when you're done with that, I wouldn't mind delving a little bit into how real NUMA use cases would be modeled/expressed. Like the one cfriesen mentions above.
16:31:58 sean-k-mooney leakypipes: http://docs.ceph.com/docs/master/rados/operations/crush-map/#types-and-buckets just as an fyi. if you swap osd for RP that could work for placement too

Earlier   Later