| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-13 | |||
| 17:26:47 | stephenfin | bauzas: I hadn't heard of it before so I would assume not | |
| 17:26:56 | bauzas | stephenfin: https://en.wikipedia.org/wiki/Non-uniform_memory_access#/media/File:Hwloc.png | |
| 17:27:08 | openstackgerrit | Merged openstack/nova-specs master: Allow for merging traits and aggregates https://review.openstack.org/552122 | |
| 17:27:23 | bauzas | stephenfin: ccNUMA means that you can have multiple cache levels | |
| 17:27:50 | bauzas | I mean, for the same NUMA node | |
| 17:27:53 | stephenfin | bauzas: Oh, that sounds a bit like the cluster-on-die thing sean-k-mooney has been on about for years | |
| 17:28:22 | bauzas | no worries | |
| 17:28:27 | stephenfin | We just use whatever libvirt is reporting to us. Based on that hwloc output, we'd see four NUMA nodes/sockets for that system | |
| 17:28:42 | bauzas | so, the only resouces per NUMA node are just memory and devices, right? | |
| 17:28:55 | stephenfin | Hugepages and PCI devices, yup | |
| 17:28:58 | bauzas | I mean, the ones we know | |
| 17:29:06 | stephenfin | Be they PCI passthrough or SR-IOV | |
| 17:29:07 | bauzas | k | |
| 17:29:43 | stephenfin | I'm trying to add PCI devices used by vSwitches and I guess vGPU will use that at some point too | |
| 17:30:02 | bauzas | stephenfin: yeah, hence why I'm writing a spec | |
| 17:30:11 | bauzas | for NUMA affinity for vGPUs | |
| 17:30:24 | bauzas | at least, that spec is just about providing a RP tree for NUMA nodes | |
| 17:30:40 | stephenfin | Gotcha. Stick me on that once you've something drafted. Would be interested in seeing what it looks like | |
| 17:30:55 | bauzas | if we have PCI devices attached to a specific NUMA node, we could imagine those being nested RPs | |
| 17:31:05 | stephenfin | Yup, that's the plan | |
| 17:31:12 | stephenfin | a cycle or two from now though | |
| 17:31:16 | stephenfin | at least, afaik | |
| 17:31:16 | cdent | jaypipes: if you can put your +2 back on here https://review.openstack.org/#/c/540049/ then 3 things can merge and there will much rejoicing | |
| 17:31:17 | bauzas | but the memory would just be a resource class | |
| 17:31:31 | bauzas | even for hugepages | |
| 17:31:41 | stephenfin | Not nested under NUMA nodes? | |
| 17:31:42 | bauzas | that was my question about L2 caches | |
| 17:31:48 | bauzas | yeah | |
| 17:31:54 | stephenfin | Hmm, that seems...odd | |
| 17:32:04 | bauzas | each NUMA node has some memory, right? | |
| 17:32:13 | stephenfin | Then again, so does two views into what are essentially the same thing | |
| 17:32:19 | stephenfin | They should | |
| 17:32:31 | bauzas | okay, lemme draft the spec then | |
| 17:32:41 | stephenfin | Though I can't remember what happens if you leave some sockets on the motherboard empty | |
| 17:33:00 | bauzas | because IMHO, unless some cache memory isn't shared across the NUMA node, I see that as just a resource class for the NUMA node RP | |
| 17:33:10 | stephenfin | Yup | |
| 17:33:23 | bauzas | like CPUs | |
| 17:33:42 | bauzas | ie. I have a 2-sockets with 4 cores each, HT | |
| 17:33:43 | stephenfin | But remember, hugepages are essentially normal memory | |
| 17:33:50 | stephenfin | You can have 4k huge pages | |
| 17:34:04 | stephenfin | *"huge pages" | |
| 17:34:13 | stephenfin | We currently have open bugs around not tracking that stuff correctly | |
| 17:34:16 | bauzas | in that case, I'd have 2 NUMA nodes with 8 CPUs each | |
| 17:34:30 | stephenfin | Yup | |
| 17:34:40 | openstackgerrit | Merged openstack/nova master: crypto: Remove unused functions https://review.openstack.org/550772 | |
| 17:34:55 | bauzas | ie. 2 child RPs, with each of those having some memory and a VCPU RC of 8 | |
| 17:35:14 | bauzas | ok, I'm going to model this | |
| 17:35:26 | bauzas | stephenfin: hugepages sound a different resource class, right? | |
| 17:36:02 | bauzas | stephenfin: https://blueprints.launchpad.net/nova/+spec/numa-topology-with-rps is the boilerplate | |
| 17:36:57 | stephenfin | bauzas: Yup, but I don't know if we have any way to identify 4k "huge pages" from normal, non huge-page memory | |
| 17:37:08 | stephenfin | I need to root out those bugs to get the full context again | |
| 17:37:22 | bauzas | right | |
| 17:37:46 | melwitt | mriedem: on https://blueprints.launchpad.net/nova/+spec/add-server-group-remove-member-notifications, I'm not sure we want to notify about server group member removal upon overquota. the "remove server group member" is not a public API and the overquota in this case would be the one that occurs during a quota recheck, and I didn't think we'd want to expose that to an end user | |
| 17:38:35 | stephenfin | bauzas: Enjoy drafting that. I'm off for a run o/ | |
| 17:38:47 | melwitt | mriedem: guess I'll comment on the bp whiteboard | |
| 17:39:52 | openstackgerrit | Merged openstack/nova master: ca: Remove 'nova/CA' directory https://review.openstack.org/550773 | |
| 17:40:09 | openstackgerrit | Merged openstack/nova master: conf: Remove 'nova.crypto' opts https://review.openstack.org/550774 | |
| 17:43:10 | jaypipes | cdent: done | |
| 17:43:47 | lyarwood | stephenfin: https://docs.openstack.org/nova/latest/configuration/config.html - under passthrough_whitelist - are the quotes all rendering as unicode left “ and right ” quotation marks for you? Have a guess what this does if you copy things into nova.conf ^_^ | |
| 17:45:08 | efried | I have never found smart quotes to be as advertised. | |
| 17:45:17 | cdent | thanks jaypipes | |
| 17:46:27 | openstackgerrit | Elod Illes proposed openstack/nova master: WIP: Transform scheduler.select_destinations notification https://review.openstack.org/508506 | |
| 17:54:53 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Make archive_deleted_rows handle a missing CONF.api_database.connection https://review.openstack.org/550967 | |
| 17:59:45 | jaypipes | dansmith: not sure I follow you with "dynamic section like cinder does for backends". could you clue me in on that with a link to what you're referring to? | |
| 18:00:49 | dansmith | jaypipes: do you remember the discussion we had on thursday in the lunch room? | |
| 18:01:19 | dansmith | jaypipes: cinder has a thing where you define backend names, and then it assumes you have created a section in config named "backend_%(name)s", with keys in it | |
| 18:01:24 | jaypipes | dansmith: I don't believe I was there, no. (I don't remember sharing lunch with you on any of the days) | |
| 18:01:34 | dansmith | which is a way to encode dicts in ini format in less nasty form | |
| 18:01:42 | dansmith | jaypipes: I was speaking directly to you, you were there :) | |
| 18:01:56 | jaypipes | dansmith: which soup was it? :) | |
| 18:01:57 | cdent | that was the twin | |
| 18:02:04 | dansmith | this was when we relocated to the lunch room of the hotel on thursday afternoon for the neutron discussion | |
| 18:02:11 | dansmith | jaypipes: no soup, it was not over lunch | |
| 18:03:26 | jaypipes | k. gibi had indeed brought up the option of doing something like physnet_mapping_%(physnetname)s instead of a single config option. I wasn't keen on that. | |
| 18:03:36 | jaypipes | dansmith: sorry, I just don't remember that conversation :( | |
| 18:04:10 | dansmith | I _hate_ encoding json in our config files | |
| 18:04:14 | dansmith | I think it's a pain we should not inflict on humans | |
| 18:04:35 | dansmith | there's prior art for this in at least cinder | |
| 18:05:14 | dansmith | we previously broke operators with the pci whitelist when we accidentally changed the format from a list to a dict, because it was json and thus obscure that anything was changing | |
| 18:05:28 | dansmith | that caused people a lot of pain, on top of the pain required to view/edit it in general | |
| 18:05:30 | jaypipes | dansmith: so we'd have multiple options like this? physnet_mapping_physnet0_name = "tenant_data_0"; physnet_mapping_physnet0_numa_nodes = 0,1; | |
| 18:05:49 | jaypipes | dansmith: personally I find encoded that to be less nice than just putting it in a JSON blob. | |
| 18:06:08 | dansmith | no | |
| 18:06:28 | mriedem | melwitt: notifications are on the rpc queue, so the user wouldn't see them... | |
| 18:07:06 | jaypipes | dansmith: mind etherpadding up your proposed conf option format? | |
| 18:07:06 | dansmith | jaypipes: https://pastebin.com/zP7eyAZw | |
| 18:07:10 | jaypipes | lol | |
| 18:07:12 | jaypipes | jinx | |
| 18:07:15 | mriedem | unless you mean user = admin/operator looking at notifications; as for the actual usefulness of that specific case, i don't think it's really useful either since it's normal during a quota recheck | |
| 18:07:19 | jaypipes | always ten steps ahead of me :) | |
| 18:07:26 | mriedem | i.e. we don't send a versioned notification when server create fails due to overquota on ram/cpu | |
| 18:07:45 | jaypipes | dansmith: heh, ok. sure, I can go for that. | |
| 18:07:58 | jaypipes | dansmith: excellent use of the words "bloopy" and "poopy" | |
| 18:08:23 | dansmith | um, excuse me. | |
| 18:08:26 | dansmith | it's "boopy" | |
| 18:08:34 | jaypipes | oh, my mistake. yes, boopy. | |
| 18:09:37 | mriedem | lyarwood: i don't see the problem, clear as day https://docs.openstack.org/nova/latest/configuration/config.html#pci.passthrough_whitelist | |
| 18:09:44 | jaypipes | dansmith: k. added a link on the spec and mentioned I could go for that. | |
| 18:09:57 | dansmith | ack | |
| 18:14:40 | lyarwood | mriedem: so for me that renders with unicode quotes that when copied into nova.conf makes n-cpu bork | |