| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-03 | |||
| 16:12:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix logging parameter in _populate_pci_mac_address https://review.openstack.org/607628 | |
| 16:12:54 | openstack | Launchpad bug 1795064 in OpenStack Compute (nova) "SR-IOV error IndexError: pop from empty list" [Undecided,New] | |
| 16:12:54 | mriedem | sean-k-mooney: you might be interested in https://bugs.launchpad.net/nova/+bug/1795064 | |
| 16:13:06 | mriedem | something something sriov and kernel versions | |
| 16:13:26 | sean-k-mooney | looking | |
| 16:13:47 | jaypipes | mriedem: https://review.openstack.org/#/c/607626/ is that stable/ocata backport for the duplicate hypervisor_hostname thingie | |
| 16:13:56 | jaypipes | mriedem: thx for your help earlier. | |
| 16:15:11 | mriedem | jaypipes: so you cherry-picked that from master? | |
| 16:15:16 | mriedem | https://review.openstack.org/#/c/508555/ | |
| 16:15:39 | mriedem | or did you cherry pick from the pike backport but forget the -x option on the cherry-pick command? | |
| 16:15:44 | spatel_ | Hi folks | |
| 16:15:50 | jaypipes | mriedem: no, I cherry-picked the SHA1 from the stable/pike patch | |
| 16:16:06 | jaypipes | mriedem: oh, sorry, I don't know about -x :( | |
| 16:16:28 | spatel_ | I am having issue with SR-IOV with shared PCI device between numa and reading this blueprint : https://blueprints.launchpad.net/nova/+spec/share-pci-between-numa-nodes | |
| 16:17:25 | spatel_ | I have set hw:pci_numa_affinity_policy='preferred' in flavor but still its not allowing me to run instance on NUMA-1 | |
| 16:17:25 | openstack | Launchpad bug 1795064 in OpenStack Compute (nova) "SR-IOV error IndexError: pop from empty list" [Undecided,New] | |
| 16:17:25 | sean-k-mooney | spatel_: this bug https://bugs.launchpad.net/nova/+bug/1795064? or another? | |
| 16:17:58 | spatel_ | sean-k-mooney: that problem got resolved by downgrading kernel to 3.x | |
| 16:18:39 | sean-k-mooney | spatel_: i think you issue with 4.18 was that you did not have a netdev associated with the vf | |
| 16:19:04 | spatel_ | Is that configuration issue or BUG? | |
| 16:19:46 | sean-k-mooney | spatel_: i would say config issue. i would guess the default options for the gernel module cahnged and or you are using a different driver by default | |
| 16:19:47 | jaypipes | mriedem: apologies. how can I fix appropriately? do I need to re-do the git cherry-pick with -x? or can/should I just edit the commit message with seomthing? | |
| 16:20:27 | sean-k-mooney | spatel_: for example if the device was bound to vfio_pci instead fo the broadcom driver then it would existit in lspci but not have a netdev | |
| 16:20:59 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Service version check for NUMA live migration https://review.openstack.org/566723 | |
| 16:20:59 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: Libvirt live migration: update NUMA XML for dest https://review.openstack.org/575179 | |
| 16:21:01 | spatel_ | hmmm! | |
| 16:21:02 | sean-k-mooney | spatel_: do you whitelist devices using the devname option? | |
| 16:21:04 | mriedem | jaypipes: see my other comment in the ocata backport about documenting the conflicts? | |
| 16:21:18 | spatel_ | sean-k-mooney: yes i am using devname option to specify my interface | |
| 16:21:29 | spatel_ | pci_passthrough_whitelist = "{ "physical_network":"vlan", "devname":"eno2" }" | |
| 16:21:36 | sean-k-mooney | spatel_: in general i advise against that for this exact reason | |
| 16:21:52 | sean-k-mooney | spatel_: if you use the pci adddress instead 4.18 would likely be fine | |
| 16:22:14 | spatel_ | pci address ? | |
| 16:22:27 | sean-k-mooney | spatel_: the whitelist can have 3 modes of whitelisting | |
| 16:22:35 | spatel_ | you mean vendor_id or product_id ? | |
| 16:23:09 | sean-k-mooney | spatel_: you can used devname, (vendor_id and product_id) or you can pass a pci address | |
| 16:23:24 | spatel_ | sean-k-mooney: i will give it a try and report back to BUG | |
| 16:23:41 | openstack | Launchpad bug 1795920 in OpenStack Compute (nova) "SR-IOV shared PCI numa not working " [Undecided,New] | |
| 16:23:41 | spatel_ | sean-k-mooney: currently i am dealing with this issue :( https://bugs.launchpad.net/nova/+bug/1795920 | |
| 16:23:58 | spatel_ | Do you know what wrong i am doing here | |
| 16:24:14 | spatel_ | I have 2 NUMA node and running SR-IOV with shared PCI | |
| 16:24:18 | sean-k-mooney | https://docs.openstack.org/mitaka/networking-guide/config-sriov.html has the doc | |
| 16:24:23 | sean-k-mooney | am let me look | |
| 16:24:33 | spatel_ | I am only able to use one side of NUMA | |
| 16:25:27 | sean-k-mooney | by default unless you set a pci numa affinity policy in the flavor or image we require strict numa afinity | |
| 16:25:32 | spatel_ | Its not allowing me to launch SR-IOV instance on NUMA-2 ( because PCI is attach to NUMA-1 ) | |
| 16:25:52 | spatel_ | All i did is hw:pci_numa_affinity_policy=preferred in flavor | |
| 16:25:57 | spatel_ | what else i need to do? | |
| 16:26:39 | sean-k-mooney | spatel_: let me check. i taught that was enough but you might also need to set the policy in the whitelist | |
| 16:26:40 | spatel_ | I do have aggregate_instance_extra_specs:pinned='true', hw:cpu_policy='dedicated' in flavor | |
| 16:27:10 | spatel_ | I think document isn't clear in blueprint so i am totally confused :( | |
| 16:28:06 | spatel_ | If i remove "aggregate_instance_extra_specs:pinned='true', hw:cpu_policy='dedicated'" from flavor then i am able to launch instance anywhere in NUMA with SR-IOV support | |
| 16:29:03 | sean-k-mooney | aggregate_instance_extra_specs:pinned='true' is not a standard thing | |
| 16:30:03 | spatel_ | hmmm! i didn't get it | |
| 16:30:20 | sean-k-mooney | spatel_: you should not need to set anything in the aggragte to use the pci policies | |
| 16:30:29 | openstackgerrit | Jay Pipes proposed openstack/nova stable/ocata: Re-use existing ComputeNode on ironic rebalance https://review.openstack.org/607626 | |
| 16:30:46 | spatel_ | oh! so you are saying i should remove aggregate_instance_extra_specs:pinned | |
| 16:30:55 | jaypipes | mriedem: k, hopefully correct now. | |
| 16:31:02 | jaypipes | thx for the help again. | |
| 16:31:20 | sean-k-mooney | spatel_: yes | |
| 16:31:31 | spatel_ | lets say if i remove "aggragte" then does my vCPU get Pinned ? | |
| 16:33:57 | spatel_ | removing aggrate and going to launch instance | |
| 16:36:25 | spatel_ | sean-k-mooney: didn't work error 'No valid host was found. There are not enough hosts available' | |
| 16:36:49 | spatel_ | look like something is missing.. | |
| 16:38:38 | sean-k-mooney | spatel_: so jsut to confirm you dont have any aggragte metadata set and have hw:pci_numa_affinity_policy=preferred set | |
| 16:38:48 | sean-k-mooney | in the flavor | |
| 16:39:15 | spatel_ | This is what i have currently in flavor -> properties | hw:cpu_policy='dedicated', hw:numa_nodes='2', hw:pci_numa_affinity_policy='preferred' | |
| 16:39:57 | stephenfin | spatel_, sean-k-mooney: We didn't implement it with a flavor extra spec in the end | |
| 16:39:57 | spatel_ | If i remove all 3 option then i am successfully able to launch instance | |
| 16:40:13 | sean-k-mooney | so this has be set in the pci whitelist then | |
| 16:40:17 | stephenfin | spatel_: Yes | |
| 16:40:23 | stephenfin | Oops, sean-k-mooney ^ | |
| 16:40:45 | spatel_ | oh!! wait wait.. so what i need to do in pci whitelist ? | |
| 16:40:49 | sean-k-mooney | spatel_: https://github.com/openstack/nova/blob/master/nova/pci/request.py#L16-L25 | |
| 16:41:13 | sean-k-mooney | sorry thats not what you want but yes you do | |
| 16:41:28 | spatel_ | so i need to add that snippet in compute nova.conf in [PCI] section ? | |
| 16:41:29 | stephenfin | spatel_: Have you seen this? https://docs.openstack.org/nova/latest/admin/networking.html#numa-affinity | |
| 16:41:41 | stephenfin | spatel_: Ignore that - wrong feature :) | |
| 16:41:47 | spatel_ | ok | |
| 16:41:52 | spatel_ | I am running queens | |
| 16:42:09 | stephenfin | spatel_: https://docs.openstack.org/nova/latest/configuration/config.html#pci | |
| 16:42:30 | stephenfin | See the alias configuration key | |
| 16:42:42 | stephenfin | spatel_: But, to be clear, is this for a PCI device or an SR-IOV device? | |
| 16:42:52 | spatel_ | SR-IOV device | |
| 16:43:20 | spatel_ | We are running high performance network application and need high speed network or high PPS rate | |
| 16:43:29 | sean-k-mooney | stephenfin: looking at the whitelist code i dont think we supprot it in the whitelist | |
| 16:44:12 | stephenfin | sean-k-mooney: Doesn't seem like it. I'm trying to think why it was done that way | |
| 16:44:28 | sean-k-mooney | which would mean the polices only work for device requeted via flavor alias which would be dumb | |
| 16:44:47 | sean-k-mooney | are you sure we did not supprot this in the flavor extraspecs /image metadata | |
| 16:44:52 | spatel_ | I am going to add alias and get back to you.. | |
| 16:44:53 | stephenfin | definitely not | |
| 16:45:19 | sean-k-mooney | stephenfin: was the whole point of this feature to fix neutron sriov | |
| 16:45:53 | spatel_ | is product_id and vendore_id mandatory because in i am using devname here "pci_passthrough_whitelist = "{ "physical_network":"vlan", "devname":"eno2" }"" | |
| 16:46:23 | sean-k-mooney | spatel_: no | |
| 16:47:58 | sean-k-mooney | a white list can be in any of these forms https://github.com/openstack/nova/blob/master/nova/pci/devspec.py#L182-L192 | |
| 16:48:20 | sean-k-mooney | actully the alias yes that need to use vendor_id and product_id | |
| 16:48:34 | stephenfin | sean-k-mooney: If it was, it seems something may have slipped through the cracks here | |
| 16:48:48 | sean-k-mooney | alias are not for networking they are for passthrough devices | |
| 16:48:56 | stephenfin | Yup, I get that | |
| 16:49:12 | spatel_ | type-PCI, type-PF and type-VF what i should pick ? | |