Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-09-03
15:26:51 sean-k-mooney which one. the auto allocation of a pcie route per numa node?
15:28:35 stephenfin kashyap: http://paste.openstack.org/show/729349/
15:28:52 stephenfin sean-k-mooney: Fortunately for you, I have zilch on my plate rn. Send em my way
15:29:33 sean-k-mooney ill create an ether pad with them the issue is i have not got around to filing/writing them all.
15:31:44 sean-k-mooney stephenfin: im not sure that is correct for num_pcie_ports=2
15:33:04 sean-k-mooney stephenfin: your machine type is pc-i440fx-bionic so you are not enableing q35 chipset so no pcie at all
15:33:57 stephenfin I don't think so either. I was expecting to see a <pci model='pcie-root-port'> element
15:34:45 stephenfin Sorry - <controller type='pci' model='pcie-root-port'>
15:35:01 stephenfin So why didn't that property get applied...
15:35:35 sean-k-mooney well currently everything in that xml is connect to the pci bus not pcie. also as i sad you are not using the q35 chipset so no pcie is available
15:35:52 stephenfin Oh yeah, I meant the 'hw:machine_type' property
15:36:17 sean-k-mooney i think its only read for arm...
15:36:50 stephenfin waaaait
15:36:56 sean-k-mooney it should be read on all archs but just going on the glance metadata docs i think its ignored on x86. i would prefer if it was not
15:37:09 stephenfin I misread kashyap's comments. You can't do this with extra specs
15:41:10 stephenfin kashyap: I misread your instructions. Here's the q35-only output http://paste.openstack.org/show/729350/
15:44:38 stephenfin kashyap: And with the addition of the num_pcie_ports option http://paste.openstack.org/show/729351/
15:44:45 sean-k-mooney stephenfin: kashyap that also looks incorrect but for different reasons. you asked for 2 pcie-root-ports and have 4
15:44:46 stephenfin sean-k-mooney: That looks better ^
15:45:05 stephenfin sean-k-mooney: Nope, first one is without that option
15:45:54 sean-k-mooney stephenfin: that still has 4 pcie-root-ports
15:46:06 stephenfin Indeed. I'd expect to see 3
15:46:27 sean-k-mooney why 3 didnt you request 2?
15:46:58 stephenfin because someone can't count :) https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5114-L5119
15:48:02 stephenfin So that option determines how many extra to add above the original amount
15:48:10 stephenfin *above the minimum required
15:49:06 stephenfin moshele: Quick question: how does one create a netdev for a connectx-3 VF?
15:49:20 stephenfin Assuming that's something you can do? I'm getting warnings without it
15:50:58 moshele stephenfin: https://community.mellanox.com/docs/DOC-1484#jive_content_id_Configuring_8_VFs_on_a_dual_port_NIC_while_all_VFs_are_probed_on_port_1
15:51:08 stephenfin moshele: ta
15:51:24 moshele stephenfin: connectx-3 VF is not working with openstack
15:51:33 stephenfin ohhh, really?
15:51:48 sean-k-mooney stephenfin: really? that is not what the melonox docs say
15:52:12 sean-k-mooney oh wait connext-x 3
15:52:40 moshele stephenfin: there is a missing patch https://review.openstack.org/#/c/409526/
15:52:57 moshele stephenfin: because we one PF for 2 ports
15:53:21 moshele stephenfin: basicly it will not work with duel port nic of cx3
15:53:26 sean-k-mooney moshele: oh connet-x3 is that card
15:53:31 stephenfin moshele: I saw that. Definitely quirky
15:53:36 moshele stephenfin: brb
15:53:58 stephenfin Guess I can revive that patch and see if we can move it forward. Would be nice to be able to use this NIC
15:55:19 sean-k-mooney stephenfin: that patch is not realated to your issue
15:57:44 sean-k-mooney stephenfin: there is another issue with that card on the neutron side but the failure you are hitting is in nova/libvirt way after teh port is already bound
15:58:12 stephenfin sean-k-mooney: Just realized that code snippet I sent is actually adding two different devices. L5114 is a PCIe root controller while the other is a PCIe root _port_ controller. My bad https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5114-L5119
15:58:23 stephenfin So I'm guessing the first two ports are automatically added by libvirt
15:58:37 sean-k-mooney stephenfin: looking at https://community.mellanox.com/docs/DOC-2365 did you set probe_vf ?
15:59:37 sean-k-mooney stephenfin: you should set probe_vf=1
16:01:45 sean-k-mooney actully im not sure..
16:09:46 moshele stephenfin: yes you should to make it work with a single port vf
16:10:12 moshele stephenfin: what is the issue exactly?
16:12:54 sean-k-mooney moshele: the issue stephenfin is having is that there are no netdevs allcoated for the VFs
16:13:07 stephenfin moshele: Yeah, and I'm seeing this for macvtap as a result http://paste.openstack.org/show/729339/
16:13:12 stephenfin I'll try the probe option now
16:13:20 stephenfin It wasn't clear what that was needed for
16:13:29 sean-k-mooney moshele: so while ip link shows the vfs allocated form the pf there are no nedevs for the vfs
16:15:01 moshele sean-k-mooney: there should be if you set probe_vf=X
16:15:40 sean-k-mooney moshele: currntly i dont think stephen set that so i assume based on the behavior he is seeing that it defaults to 0
16:15:53 moshele sean-k-mooney: yes you are right
16:16:45 sean-k-mooney in which case if stephenfin set probe_vf=1 then vnic-type=macvtap may start working
16:17:23 sean-k-mooney if there is no netdev then get_ifname_by_pci_address is going to fail
16:19:08 stephenfin moshele: What's the difference between mlx4.conf and mlx4_core.conf?
16:20:19 moshele stephenfin: no difference you can use ether one I think the mlx4_core.conf comes from Mellanox OFED
16:20:32 moshele stephenfin: brb
16:30:33 pvc No compute node record for host when enabling ironic on packstack
16:31:49 stephenfin moshele: Yup, that was the issue. I know for next time. I am seeing another issue now though, if you have time to look at it http://paste.openstack.org/show/729352/
16:32:04 stephenfin (the issue is at the end. The rest is the configuration I have)
16:33:56 sean-k-mooney stephenfin: if you do a virsh nodedev-list can you see net_enp2s1d1_fa_16_3e_38_ea_a6
16:34:41 stephenfin I can't. I'm not sure where it got that from
16:34:52 stephenfin Actually, it only started happening once I created an instance with macvtap
16:35:02 stephenfin I wonder if this is another reflection of the dual-port issues
16:35:08 stephenfin *another example
16:36:48 pvc No compute node record for host when enabling ironic on packstack
16:36:50 pvc anyone
16:36:51 pvc can help me
16:37:50 sean-k-mooney pvc: sorry you proably wont find that many people with packstack knowladge here
16:43:32 pvc but the main problem is
16:43:36 pvc it cannot find any hypervisor
16:43:41 pvc because i enable the ironic driver instead of libvir
16:44:42 sean-k-mooney pvc: when you enable ironic it should still run at least one instance of the nova-compute agent that loads the ironic driver. this agent should create at least one compute service record
16:45:33 sean-k-mooney each node you enrole in ionic should then be exposed as a seperate compute node/ hypervior record mapped to that compute servcice
16:46:44 sean-k-mooney if the ironic nova-compute agent does not start or fails for some reason you will have no compute node entries
16:49:27 pvc http://paste.openstack.org/show/729355/
16:50:06 sean-k-mooney pvc: yes.. did you manually add nodes to ironic. also this does not show you the compute noes
16:50:38 pvc ahmm no. for now we're not manually adding nodes to ironic
16:50:41 pvc we just install openstack using packstack
16:50:45 pvc i found something simirlar
16:50:49 pvc http://getvasanth.blogspot.com/2015/09/getting-start-with-ironic.html
16:51:35 sean-k-mooney pvc: what happens if you run openstack compute service list and openstack compute agent list
16:52:20 pvc http://paste.openstack.org/show/729356/
16:53:27 sean-k-mooney pvc: well it looks like you have 3 registers compute services that said i would have expected to also see them in the agent list
16:54:08 pvc but the server i'm using have an ipmi
16:54:14 sean-k-mooney in either case teh service state is up so i would suspec the nova pices is running correctly
16:54:18 pvc i guess packstack doesnt install ironic well
16:54:54 sean-k-mooney pvc: well installing ironic and enroling node in ironic are two very different things
16:55:37 pvc i see
16:55:42 pvc so i need to manually add it
16:56:49 sean-k-mooney pvc: i think packstack installed the ironic compute agent on nova01,nova02 and network hosts but it does not have any baremetal nodes registered for it to contorl
16:58:01 pvc can i use this guide sir? http://getvasanth.blogspot.com/2015/09/getting-start-with-ironic.html
16:59:25 sean-k-mooney pvc: looking at it quickly it seam rather old. it may still work but i would gues several things have changed in the last 3 years. if you do want to deploy with packstack i would recommend following a more up to date guide
17:03:50 pvc can i use the ironic official document?

Earlier   Later