Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-09-03
15:33:04 sean-k-mooney stephenfin: your machine type is pc-i440fx-bionic so you are not enableing q35 chipset so no pcie at all
15:33:57 stephenfin I don't think so either. I was expecting to see a <pci model='pcie-root-port'> element
15:34:45 stephenfin Sorry - <controller type='pci' model='pcie-root-port'>
15:35:01 stephenfin So why didn't that property get applied...
15:35:35 sean-k-mooney well currently everything in that xml is connect to the pci bus not pcie. also as i sad you are not using the q35 chipset so no pcie is available
15:35:52 stephenfin Oh yeah, I meant the 'hw:machine_type' property
15:36:17 sean-k-mooney i think its only read for arm...
15:36:50 stephenfin waaaait
15:36:56 sean-k-mooney it should be read on all archs but just going on the glance metadata docs i think its ignored on x86. i would prefer if it was not
15:37:09 stephenfin I misread kashyap's comments. You can't do this with extra specs
15:41:10 stephenfin kashyap: I misread your instructions. Here's the q35-only output http://paste.openstack.org/show/729350/
15:44:38 stephenfin kashyap: And with the addition of the num_pcie_ports option http://paste.openstack.org/show/729351/
15:44:45 sean-k-mooney stephenfin: kashyap that also looks incorrect but for different reasons. you asked for 2 pcie-root-ports and have 4
15:44:46 stephenfin sean-k-mooney: That looks better ^
15:45:05 stephenfin sean-k-mooney: Nope, first one is without that option
15:45:54 sean-k-mooney stephenfin: that still has 4 pcie-root-ports
15:46:06 stephenfin Indeed. I'd expect to see 3
15:46:27 sean-k-mooney why 3 didnt you request 2?
15:46:58 stephenfin because someone can't count :) https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5114-L5119
15:48:02 stephenfin So that option determines how many extra to add above the original amount
15:48:10 stephenfin *above the minimum required
15:49:06 stephenfin moshele: Quick question: how does one create a netdev for a connectx-3 VF?
15:49:20 stephenfin Assuming that's something you can do? I'm getting warnings without it
15:50:58 moshele stephenfin: https://community.mellanox.com/docs/DOC-1484#jive_content_id_Configuring_8_VFs_on_a_dual_port_NIC_while_all_VFs_are_probed_on_port_1
15:51:08 stephenfin moshele: ta
15:51:24 moshele stephenfin: connectx-3 VF is not working with openstack
15:51:33 stephenfin ohhh, really?
15:51:48 sean-k-mooney stephenfin: really? that is not what the melonox docs say
15:52:12 sean-k-mooney oh wait connext-x 3
15:52:40 moshele stephenfin: there is a missing patch https://review.openstack.org/#/c/409526/
15:52:57 moshele stephenfin: because we one PF for 2 ports
15:53:21 moshele stephenfin: basicly it will not work with duel port nic of cx3
15:53:26 sean-k-mooney moshele: oh connet-x3 is that card
15:53:31 stephenfin moshele: I saw that. Definitely quirky
15:53:36 moshele stephenfin: brb
15:53:58 stephenfin Guess I can revive that patch and see if we can move it forward. Would be nice to be able to use this NIC
15:55:19 sean-k-mooney stephenfin: that patch is not realated to your issue
15:57:44 sean-k-mooney stephenfin: there is another issue with that card on the neutron side but the failure you are hitting is in nova/libvirt way after teh port is already bound
15:58:12 stephenfin sean-k-mooney: Just realized that code snippet I sent is actually adding two different devices. L5114 is a PCIe root controller while the other is a PCIe root _port_ controller. My bad https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5114-L5119
15:58:23 stephenfin So I'm guessing the first two ports are automatically added by libvirt
15:58:37 sean-k-mooney stephenfin: looking at https://community.mellanox.com/docs/DOC-2365 did you set probe_vf ?
15:59:37 sean-k-mooney stephenfin: you should set probe_vf=1
16:01:45 sean-k-mooney actully im not sure..
16:09:46 moshele stephenfin: yes you should to make it work with a single port vf
16:10:12 moshele stephenfin: what is the issue exactly?
16:12:54 sean-k-mooney moshele: the issue stephenfin is having is that there are no netdevs allcoated for the VFs
16:13:07 stephenfin moshele: Yeah, and I'm seeing this for macvtap as a result http://paste.openstack.org/show/729339/
16:13:12 stephenfin I'll try the probe option now
16:13:20 stephenfin It wasn't clear what that was needed for
16:13:29 sean-k-mooney moshele: so while ip link shows the vfs allocated form the pf there are no nedevs for the vfs
16:15:01 moshele sean-k-mooney: there should be if you set probe_vf=X
16:15:40 sean-k-mooney moshele: currntly i dont think stephen set that so i assume based on the behavior he is seeing that it defaults to 0
16:15:53 moshele sean-k-mooney: yes you are right
16:16:45 sean-k-mooney in which case if stephenfin set probe_vf=1 then vnic-type=macvtap may start working
16:17:23 sean-k-mooney if there is no netdev then get_ifname_by_pci_address is going to fail
16:19:08 stephenfin moshele: What's the difference between mlx4.conf and mlx4_core.conf?
16:20:19 moshele stephenfin: no difference you can use ether one I think the mlx4_core.conf comes from Mellanox OFED
16:20:32 moshele stephenfin: brb
16:30:33 pvc No compute node record for host when enabling ironic on packstack
16:31:49 stephenfin moshele: Yup, that was the issue. I know for next time. I am seeing another issue now though, if you have time to look at it http://paste.openstack.org/show/729352/
16:32:04 stephenfin (the issue is at the end. The rest is the configuration I have)
16:33:56 sean-k-mooney stephenfin: if you do a virsh nodedev-list can you see net_enp2s1d1_fa_16_3e_38_ea_a6
16:34:41 stephenfin I can't. I'm not sure where it got that from
16:34:52 stephenfin Actually, it only started happening once I created an instance with macvtap
16:35:02 stephenfin I wonder if this is another reflection of the dual-port issues
16:35:08 stephenfin *another example
16:36:48 pvc No compute node record for host when enabling ironic on packstack
16:36:50 pvc anyone
16:36:51 pvc can help me
16:37:50 sean-k-mooney pvc: sorry you proably wont find that many people with packstack knowladge here
16:43:32 pvc but the main problem is
16:43:36 pvc it cannot find any hypervisor
16:43:41 pvc because i enable the ironic driver instead of libvir
16:44:42 sean-k-mooney pvc: when you enable ironic it should still run at least one instance of the nova-compute agent that loads the ironic driver. this agent should create at least one compute service record
16:45:33 sean-k-mooney each node you enrole in ionic should then be exposed as a seperate compute node/ hypervior record mapped to that compute servcice
16:46:44 sean-k-mooney if the ironic nova-compute agent does not start or fails for some reason you will have no compute node entries
16:49:27 pvc http://paste.openstack.org/show/729355/
16:50:06 sean-k-mooney pvc: yes.. did you manually add nodes to ironic. also this does not show you the compute noes
16:50:38 pvc ahmm no. for now we're not manually adding nodes to ironic
16:50:41 pvc we just install openstack using packstack
16:50:45 pvc i found something simirlar
16:50:49 pvc http://getvasanth.blogspot.com/2015/09/getting-start-with-ironic.html
16:51:35 sean-k-mooney pvc: what happens if you run openstack compute service list and openstack compute agent list
16:52:20 pvc http://paste.openstack.org/show/729356/
16:53:27 sean-k-mooney pvc: well it looks like you have 3 registers compute services that said i would have expected to also see them in the agent list
16:54:08 pvc but the server i'm using have an ipmi
16:54:14 sean-k-mooney in either case teh service state is up so i would suspec the nova pices is running correctly
16:54:18 pvc i guess packstack doesnt install ironic well
16:54:54 sean-k-mooney pvc: well installing ironic and enroling node in ironic are two very different things
16:55:37 pvc i see
16:55:42 pvc so i need to manually add it
16:56:49 sean-k-mooney pvc: i think packstack installed the ironic compute agent on nova01,nova02 and network hosts but it does not have any baremetal nodes registered for it to contorl
16:58:01 pvc can i use this guide sir? http://getvasanth.blogspot.com/2015/09/getting-start-with-ironic.html
16:59:25 sean-k-mooney pvc: looking at it quickly it seam rather old. it may still work but i would gues several things have changed in the last 3 years. if you do want to deploy with packstack i would recommend following a more up to date guide
17:03:50 pvc can i use the ironic official document?
17:05:00 sean-k-mooney pvc: are you just trying to deploy ironic or do you want an actual packstack install. there are several installer that support ironic that you could use if all you want is to have an ironic enabled cloud
17:05:43 pvc an actual packstack install. i'll just install the ironic manually since my compute server have an IPMI
17:06:08 sean-k-mooney pvc: the ironic offila documtaiton will tell you how to achive that manually
17:06:23 sean-k-mooney pvc: this might be of relevence to you also https://review.openstack.org/#/c/440258/
17:06:28 pvc noted on this. wait i willl just check it

Earlier   Later