| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-03 | |||
| 16:27:10 | spatel_ | I think document isn't clear in blueprint so i am totally confused :( | |
| 16:28:06 | spatel_ | If i remove "aggregate_instance_extra_specs:pinned='true', hw:cpu_policy='dedicated'" from flavor then i am able to launch instance anywhere in NUMA with SR-IOV support | |
| 16:29:03 | sean-k-mooney | aggregate_instance_extra_specs:pinned='true' is not a standard thing | |
| 16:30:03 | spatel_ | hmmm! i didn't get it | |
| 16:30:20 | sean-k-mooney | spatel_: you should not need to set anything in the aggragte to use the pci policies | |
| 16:30:29 | openstackgerrit | Jay Pipes proposed openstack/nova stable/ocata: Re-use existing ComputeNode on ironic rebalance https://review.openstack.org/607626 | |
| 16:30:46 | spatel_ | oh! so you are saying i should remove aggregate_instance_extra_specs:pinned | |
| 16:30:55 | jaypipes | mriedem: k, hopefully correct now. | |
| 16:31:02 | jaypipes | thx for the help again. | |
| 16:31:20 | sean-k-mooney | spatel_: yes | |
| 16:31:31 | spatel_ | lets say if i remove "aggragte" then does my vCPU get Pinned ? | |
| 16:33:57 | spatel_ | removing aggrate and going to launch instance | |
| 16:36:25 | spatel_ | sean-k-mooney: didn't work error 'No valid host was found. There are not enough hosts available' | |
| 16:36:49 | spatel_ | look like something is missing.. | |
| 16:38:38 | sean-k-mooney | spatel_: so jsut to confirm you dont have any aggragte metadata set and have hw:pci_numa_affinity_policy=preferred set | |
| 16:38:48 | sean-k-mooney | in the flavor | |
| 16:39:15 | spatel_ | This is what i have currently in flavor -> properties | hw:cpu_policy='dedicated', hw:numa_nodes='2', hw:pci_numa_affinity_policy='preferred' | |
| 16:39:57 | spatel_ | If i remove all 3 option then i am successfully able to launch instance | |
| 16:39:57 | stephenfin | spatel_, sean-k-mooney: We didn't implement it with a flavor extra spec in the end | |
| 16:40:13 | sean-k-mooney | so this has be set in the pci whitelist then | |
| 16:40:17 | stephenfin | spatel_: Yes | |
| 16:40:23 | stephenfin | Oops, sean-k-mooney ^ | |
| 16:40:45 | spatel_ | oh!! wait wait.. so what i need to do in pci whitelist ? | |
| 16:40:49 | sean-k-mooney | spatel_: https://github.com/openstack/nova/blob/master/nova/pci/request.py#L16-L25 | |
| 16:41:13 | sean-k-mooney | sorry thats not what you want but yes you do | |
| 16:41:28 | spatel_ | so i need to add that snippet in compute nova.conf in [PCI] section ? | |
| 16:41:29 | stephenfin | spatel_: Have you seen this? https://docs.openstack.org/nova/latest/admin/networking.html#numa-affinity | |
| 16:41:41 | stephenfin | spatel_: Ignore that - wrong feature :) | |
| 16:41:47 | spatel_ | ok | |
| 16:41:52 | spatel_ | I am running queens | |
| 16:42:09 | stephenfin | spatel_: https://docs.openstack.org/nova/latest/configuration/config.html#pci | |
| 16:42:30 | stephenfin | See the alias configuration key | |
| 16:42:42 | stephenfin | spatel_: But, to be clear, is this for a PCI device or an SR-IOV device? | |
| 16:42:52 | spatel_ | SR-IOV device | |
| 16:43:20 | spatel_ | We are running high performance network application and need high speed network or high PPS rate | |
| 16:43:29 | sean-k-mooney | stephenfin: looking at the whitelist code i dont think we supprot it in the whitelist | |
| 16:44:12 | stephenfin | sean-k-mooney: Doesn't seem like it. I'm trying to think why it was done that way | |
| 16:44:28 | sean-k-mooney | which would mean the polices only work for device requeted via flavor alias which would be dumb | |
| 16:44:47 | sean-k-mooney | are you sure we did not supprot this in the flavor extraspecs /image metadata | |
| 16:44:52 | spatel_ | I am going to add alias and get back to you.. | |
| 16:44:53 | stephenfin | definitely not | |
| 16:45:19 | sean-k-mooney | stephenfin: was the whole point of this feature to fix neutron sriov | |
| 16:45:53 | spatel_ | is product_id and vendore_id mandatory because in i am using devname here "pci_passthrough_whitelist = "{ "physical_network":"vlan", "devname":"eno2" }"" | |
| 16:46:23 | sean-k-mooney | spatel_: no | |
| 16:47:58 | sean-k-mooney | a white list can be in any of these forms https://github.com/openstack/nova/blob/master/nova/pci/devspec.py#L182-L192 | |
| 16:48:20 | sean-k-mooney | actully the alias yes that need to use vendor_id and product_id | |
| 16:48:34 | stephenfin | sean-k-mooney: If it was, it seems something may have slipped through the cracks here | |
| 16:48:48 | sean-k-mooney | alias are not for networking they are for passthrough devices | |
| 16:48:56 | stephenfin | Yup, I get that | |
| 16:49:12 | spatel_ | type-PCI, type-PF and type-VF what i should pick ? | |
| 16:49:17 | spatel_ | VF ? | |
| 16:49:19 | stephenfin | From the quick glance here, it should really be configured via the whitelist. I'm not sure why I went with the alias | |
| 16:49:22 | stephenfin | spatel_: yup | |
| 16:49:27 | spatel_ | doing it.. | |
| 16:49:33 | sean-k-mooney | stephenfin: ya i am go to confirm https://bugs.launchpad.net/nova/+bug/1795920 | |
| 16:49:34 | openstack | Launchpad bug 1795920 in OpenStack Compute (nova) "SR-IOV shared PCI numa not working " [Undecided,Confirmed] | |
| 16:50:05 | sean-k-mooney | stephenfin: interested in working on this? if not ill add it to my list but this need to be fixed | |
| 16:50:15 | sean-k-mooney | and backported | |
| 16:50:21 | stephenfin | I won't tackle it tonight but I can do so, yeah | |
| 16:50:36 | stephenfin | not sure if we can backport though. It'll be a config file change | |
| 16:50:39 | sean-k-mooney | cool we liekly need to repreose the old spec | |
| 16:51:10 | sean-k-mooney | stephenfin: i was suggesting we need to add the flavor and image extraspecs so no config file change | |
| 16:51:47 | stephenfin | sean-k-mooney: Possibly, but before doing so I'd suggest going back and reading the spec reviews | |
| 16:51:59 | stephenfin | There was a reason we didn't do that, though I don't recall it now :/ | |
| 16:52:18 | spatel | sean-k-mooney: this is what i change in nova.conf http://paste.openstack.org/show/731417/ | |
| 16:52:22 | stephenfin | If it's image metadata changes, we can't backport those due to object changes | |
| 16:52:24 | spatel | can you verify | |
| 16:52:33 | sean-k-mooney | yes i rembere i was very against using the alisa but i never recalled dropping the extra specs | |
| 16:53:12 | spatel | going to launch instance now, figure cross | |
| 16:53:32 | sean-k-mooney | spatel: its goning to fail. | |
| 16:53:39 | spatel | ?? | |
| 16:53:43 | spatel | why? | |
| 16:53:48 | sean-k-mooney | looking at the code the feature was not finished | |
| 16:54:01 | spatel | Damn it :( | |
| 16:54:12 | spatel | so what is the deal here ? | |
| 16:54:25 | sean-k-mooney | the numa policies are only repected for flavor based pci device passhtoruh e.g. for things like gpu or acllorator cards | |
| 16:54:44 | sean-k-mooney | spatel: the spec was approve and the full feature was not merged | |
| 16:55:09 | spatel | in my case its VF | |
| 16:55:34 | spatel | currently i can't utilize both numa :( | |
| 16:55:47 | sean-k-mooney | spatel: in your case its a neutron port with vnic_type direct correct | |
| 16:55:52 | spatel | yes | |
| 16:56:04 | sean-k-mooney | spatel: the workaround is to make the guest have 2 numa nodes | |
| 16:56:11 | melwitt | sean-k-mooney: which spec was not properly finished? | |
| 16:56:30 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle IndexError in _populate_neutron_binding_profile https://review.openstack.org/607650 | |
| 16:56:34 | mriedem | spatel: fyi ^ | |
| 16:56:47 | sean-k-mooney | melwitt: the numa pci polices | |
| 16:57:00 | sean-k-mooney | ill get the link on sec | |
| 16:57:19 | melwitt | thanks. I didn't see a link in the backscroll | |
| 16:57:23 | spatel | sean-k-mooney: "guest have 2 numa nodes" can you explain this? | |
| 16:57:40 | sean-k-mooney | melwitt: https://review.openstack.org/#/c/361140/ | |
| 16:57:54 | openstackgerrit | Vlad Gusev proposed openstack/nova master: Not set instance to ERROR if set_admin_password failed https://review.openstack.org/555160 | |
| 16:57:55 | sean-k-mooney | spatel: in the flavor set hw:numa_nodes=2 | |
| 16:58:08 | spatel | let me try hold on... | |
| 16:58:23 | sean-k-mooney | this will create a guest with 2 numa nodes with half the cpus and ram on each | |
| 16:58:44 | spatel | sean-k-mooney: FYI, i have tried this and it failed "hw:cpu_policy='dedicated', hw:numa_nodes='2', hw:pci_numa_affinity_policy='preferred'" | |
| 16:59:11 | spatel | now i am going to remove "hw:cpu_policy='dedicated" and "hw:pci_numa_affinity_policy='preferred'" to see if that work | |
| 16:59:13 | sean-k-mooney | melwitt: the flavor and image extraspecs are appently not implemente meaning that this does not work for neutron sriov ports as it should | |
| 16:59:18 | melwitt | sean-k-mooney: ok, according to the notes on the blueprint, people thought it was completed "The last functional patch for this was merged on Dec 30, 2017" https://blueprints.launchpad.net/openstack/nova/+spec/share-pci-between-numa-nodes | |
| 16:59:55 | spatel | melwitt: that is why i am chasing that blueprint because it says completed 2017 | |
| 17:00:12 | sean-k-mooney | melwitt: i taught it was complted im checking the code to confirm but apparently its not working | |