Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-03-25
16:55:53 melwitt the spec says blocker migration for being able to remove compat code/stop legacy counting
16:55:58 melwitt understood
16:56:21 dansmith right, so, when we remove that code, we should have a blocker, so depending on how the spec words it, it may not be wrong :)
16:56:33 melwitt (although we have to keep legacy counting indefinitely until partitioning of placement allocations is possible)
16:56:49 dansmith okay
16:57:20 dansmith we could also do it a different way and just roll the blocker and new constraints together in one migration, but we definitely need to give some buffer between adding the thing and requiring the thing I thnk
16:58:11 melwitt yeah, understood and agreed
17:01:04 cdent melwitt: speaking of partitioning placement allocations. Is that a thing you were planning to spec or is that a broader all of us kind of thing?
17:01:40 melwitt cdent: broader thing
17:01:41 cdent or "dunno" is of course a reasonable answer
17:02:07 cdent ✔
17:11:07 melwitt mriedem, stephenfin, gmann: fyi, I have changes proposed for re-enabling testing of console with TLS in nova-next https://review.openstack.org/#/q/topic:bug/1819794+status:open
17:11:40 gmann melwitt: thanks.
17:12:37 sean-k-mooney cdent: is partioning placemetn allocation the "owner thing"
17:13:24 cdent sean-k-mooney: it's basically a way of saying "this allocation of VCPU is from this thing (which happens to be nova X)"
17:13:49 cdent which is somewhat different from the resource provider allocation, which is supposed to be a way of saying "these rps below to cloud X"
17:13:59 cdent (which I think is something jaypipes has been thinking about, if I remember right)
17:14:28 sean-k-mooney cdent: oh its partioning in the sense of multiple nova sharing the same placment
17:14:54 cdent sean-k-mooney: not necessarily. there could be other things that consume vcpu
17:15:03 cdent which may or may not be a nova
17:15:18 sean-k-mooney sure but i thin this started as an edge usecase.
17:15:20 cdent but as you can see there's a lot of overlap and ambiguity here, which is why it needs a bit more drive and discussion
17:15:35 sean-k-mooney ya
17:16:04 cdent yes, it started as an edge usecase, with two different models for the distribution: one placement many clouds, or one placement many novas but same keystone
17:16:10 sean-k-mooney cdent: what are the other consumer of vcpu beyond nova out of interest?
17:16:16 cdent I don't know
17:16:34 cdent I'm not too savvy about either of these use cases, just have heard people mention them
17:16:45 cdent from my standpoint, until somebody steps up a bit more, they aren't yet real
17:16:54 cdent just speculative stuff to think about in the back of the mind
17:16:55 sean-k-mooney i guess maybe zun? but i would have asumed it had its only RP
17:17:14 melwitt to be clear, I didn't mention it because it's a pressing issue right now, just to note that there's another larger reason why we wouldn't be able to remove quota usage counting compat code than just a blocker migration
17:18:10 cdent It does seem like the sort of thing where if there was somebody who had the cycles to make it go, it would be a good thing to have
17:18:17 cdent but until then...
17:19:41 jhinman I'm trying to get an instance up with ovs-dpdk. but the vif doesn't get exposed because of this code: vifs_to_expose = {vif.address: vif for vif in vifs if ('tag' in vif and vif.tag) or vlans_by_mac.get(vif.address)}
17:19:59 sean-k-mooney cdent: having the list of usecasue for the feature would be a good starting point. im not entirly sure that the usecsue we all vaguly recall someone talking about at some point are actully the same feature or should be :)
17:20:23 openstackgerrit Matt Riedemann proposed openstack/nova master: Error out migration when confirm_resize fails https://review.openstack.org/647546
17:20:31 sean-k-mooney jhinman: where is that code ?
17:20:33 cdent sean-k-mooney: yeah, it is a bit vague
17:20:54 jhinman the code is in /virt/libvirt/driver.py
17:21:23 jhinman where does the tag come from?
17:21:55 sean-k-mooney jhinman: what release are you running
17:22:08 sean-k-mooney vifs_to_expose is not in that file on master
17:22:26 jhinman rocky release
17:23:09 sean-k-mooney jhinman: its not there on rocky either are you deploying a downstream version of openstack
17:23:59 jhinman I'm deploying kolla-ansible, so whatever that container has
17:24:41 sean-k-mooney jhinman: kolla-ansible has several modes. are you usign ubuntu source vs cento binary
17:25:04 openstackgerrit Sylvain Bauza proposed openstack/nova master: Add doc on VGPU allocs and inventories for nrp https://review.openstack.org/647519
17:25:19 jhinman ubuntu
17:25:53 sean-k-mooney jhinman: the tag i think is the interface tagging feature where you can associate a generic name to an interface and look it up in the metadata service
17:27:15 jhinman but how does the tag get assigned? is it something passed from neutron. or maybe the tag is not the problem? is there supposed to be a vlan associated
17:27:45 sean-k-mooney its passed in by the user on nova boot. it is not related to the neutron segmentation id
17:27:55 sean-k-mooney its a value like "wan" or "lan"
17:28:30 efried stephenfin: Are we waiting to merge https://review.openstack.org/#/c/645991/ ?
17:28:33 sean-k-mooney so this is the code that is failing https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L9284-L9286
17:29:44 sean-k-mooney looks like we just merged somthing
17:29:46 sean-k-mooney https://github.com/openstack/nova/blob/0dfbcd74642b16432b800b51f304a89d762e3ff1/nova/virt/libvirt/driver.py#L9290-L9292
17:30:54 sean-k-mooney jhinman: there should be nothing ovs-dpdk specific in that code. it shoudl be totally generic
17:34:59 sean-k-mooney jhinman: actully if that is the code that is runnign that should not run for vhost-user interfaces
17:35:00 jhinman ultimately, it ends up with line 9204: LOG.debug('No VIF found with MAC %s, not building metadata', dev.mac_addr)
17:35:09 sean-k-mooney the section i linked if for sriov
17:37:03 sean-k-mooney jhinman: can you provide a paste of the trackback on paste.openstack.org
17:37:34 sean-k-mooney vifs_to_expose = {vif.address: vif for vif in vifs
17:37:36 sean-k-mooney if ('tag' in vif and vif.tag) or
17:37:38 sean-k-mooney vlans_by_mac.get(vif.address)}
17:37:52 sean-k-mooney is in the sriov code path and should not be called when handeling vhost-user
17:39:08 sean-k-mooney well unless we use that fucntion to do generic dispatch to the others. lets see where is it called
17:39:39 sean-k-mooney actully so yes that is how it is working
17:39:50 sean-k-mooney we get the interface metadata here
17:39:52 sean-k-mooney https://github.com/openstack/nova/blob/0dfbcd74642b16432b800b51f304a89d762e3ff1/nova/virt/libvirt/driver.py#L9310
17:41:23 sean-k-mooney jhinman: we emiit that debug message when the user does not specify any device role tags
17:41:53 sean-k-mooney so its not a error to see that in the logs
17:42:11 openstackgerrit Elod Illes proposed openstack/nova stable/pike: Fix incompatible version handling in BuildRequest https://review.openstack.org/647557
17:43:57 sean-k-mooney jhinman: this code is for this spec https://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/virt-device-role-tagging.html are you having a specic issue or did you jsut see that debug message and were trying to figure out how to scilance it?
17:45:03 jhinman allright, I was looking at that because I was going back from another error: I can't find any reason for qemu not to have permission to write the socket: Failed to connect socket /var/run/openvswitch/vhu7a90d3cd-d7: Permission denied
17:46:26 sean-k-mooney jhinman: the socket error can happen for two reasons. first apparmor can block it, that should not be an issue with rocky but it was in the past
17:46:57 sean-k-mooney second /var/run/openvswitch need to be writabel by the user that qemu is running with
17:46:58 jhinman I disable apparmor
17:47:34 jhinman but nova is the user of qemu, so should have permission for anything on the instances, right?
17:47:54 sean-k-mooney qemu and nova need to be runign in the same group
17:47:59 sean-k-mooney *qemu and ovs
17:49:13 sean-k-mooney jhinman: on rocky qemu should be acting as teh vhost-user server and will create the vhost-user socket
17:49:41 sean-k-mooney as such qemu need to have write permission to the /var/run/openvswith directory that is owned by ovs
17:51:08 jhinman so I need to add qemu to the ovs group?
17:51:53 sean-k-mooney jhinman: this should all be done by kolla ansible
17:53:12 openstackgerrit Merged openstack/nova stable/queens: Replace openstack.org git:// URLs with https:// https://review.openstack.org/646685
17:53:21 openstackgerrit Merged openstack/nova stable/ocata: Replace openstack.org git:// URLs with https:// https://review.openstack.org/646683
17:54:30 sean-k-mooney jhinman: its been a few years since i implented the support in kolla-ansibel so i cant remember of the top of my head how i fixed this issue
17:54:37 jhinman I think I remember a patch for qemu.conf, to set user to nova. but I think its in the nova group. is there a common group?
17:56:45 sean-k-mooney jhinman: this is where the contain is created https://github.com/openstack/kolla-ansible/blob/stable/rocky/ansible/roles/ovs-dpdk/handlers/main.yml#L56-L76
17:57:36 sean-k-mooney we are not setting a user in the yaml or in teh command using su no
17:57:43 sean-k-mooney https://github.com/openstack/kolla-ansible/blob/stable/rocky/ansible/roles/ovs-dpdk/templates/ovsdpdk-vswitchd.json.j2
17:58:18 sean-k-mooney so i think that means ovs woudl be running as root maybe
17:59:22 jhinman not obvious to me what you see, but I'll study it
17:59:53 sean-k-mooney qemu is running as nova
17:59:54 sean-k-mooney https://github.com/openstack/kolla-ansible/blob/stable/rocky/ansible/roles/nova/templates/qemu.conf.j2
18:00:31 jhinman yes, so how to give it permission to access ovs's file?
18:00:43 sean-k-mooney so nova would have to have write permission to /var/run/openvswitch
18:00:49 sean-k-mooney ya
18:04:29 jhinman sean: in order to get hostconfig stuff working, I added a new start-ovs script, overriding the embedded script. Is that the right way, or can the commands be added to the script in the tools directory (ovs-dpdkctl.sh)?
18:06:00 sean-k-mooney before or after this issue

Earlier   Later