| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-17 | |||
| 18:48:28 | artom | That's... bad | |
| 18:48:38 | gregwork | artom: yeah :/ | |
| 18:48:50 | melwitt | dansmith: I thought that was what you were getting at. nothing calls these methods anymore as of queens, so I thought it didn't matter what they return or not. but I knew I couldn't remove them if they were remotable | |
| 18:49:01 | artom | gregwork, and is most likely a nova and/or libvirt bug | |
| 18:49:09 | artom | Because that should never happen | |
| 18:49:49 | dansmith | melwitt: no, I'm saying we should keep the contract until we bump, but not for out-of-nova uses, just in case someone has an ocata compute running by accident or something so we do our best to not just cause it to break in weird ways | |
| 18:49:50 | artom | Well, assuming the hardware actually supports what you're trying to do, and you've configured nova correctly | |
| 18:49:58 | gregwork | artom: i dont think so, i believe this is related to ovs being configured to use mellanox dpdk poll mode driver, and kvm is not setting up the guest correctly to plumb the networking | |
| 18:50:07 | artom | But even a misconfiguration shouldn't go all the way down to a qemu error | |
| 18:50:22 | dansmith | melwitt: since it's all conductor-side for those actions, just attempt to fake the result that won't cause any of our code in the last few cycles to choke and call it good | |
| 18:50:34 | mriedem | sdague: can i get you to poke through these stable/newton changes? https://review.openstack.org/#/c/493206/ https://review.openstack.org/#/c/506323/ https://review.openstack.org/#/c/509441/ https://review.openstack.org/#/c/508902/ | |
| 18:50:37 | mriedem | tomorrow is newton-eol | |
| 18:50:52 | gregwork | artom: eg. https://paste.fedoraproject.org/paste/GWEv1HyVo1y9sekJbZFBeQ | |
| 18:51:00 | melwitt | dansmith: oh, gotcha. I understand now. I'll go over it again and look for those possibilities and update it. thanks | |
| 18:51:20 | melwitt | I at least can remove the non-remotable one in a new PS | |
| 18:52:00 | artom | gregwork, you're very rapidly bringing my out of my (admittedly limited) comfort zone :/ | |
| 18:52:58 | gregwork | artom: so i am testing vnf's on my openstack, where the compute nodes are running ovs in dpdk mode, with a poll mode driver (the nic is not managed by the kernel) | |
| 18:54:49 | artom | gregwork, sorry I can't be of more help, but I'm out of my league here | |
| 18:55:14 | mriedem | gregwork: sean-k-mooney might be able to help with dpdk questions | |
| 18:55:32 | artom | We have some folks from mellanox who hang out here, but they're (I think) on Israel time | |
| 18:55:42 | mriedem | moshe is yeah | |
| 18:55:43 | sean-k-mooney | mriedem: gregwork sure let me read back | |
| 18:56:31 | sean-k-mooney | gregwork: so yes you have a coumpute node with ovs-dpdk. what is the question? | |
| 18:57:20 | sean-k-mooney | gregwork: ah you dont have hugepages requested for the vm | |
| 18:58:12 | sdague | mriedem: done. I was a little surprised you wanted some of those back to newton, but that's fine | |
| 18:58:35 | gregwork | sean-k-mooney: have you seen that error before ? | |
| 18:58:42 | gregwork | the one i posted in the pastebin | |
| 18:58:49 | mriedem | sdague: the bfv one was needed for another thing someone was reporting in newton but w/o better debug couldn't be determined | |
| 18:59:23 | gregwork | sean-k-mooney: im using the mellanox rhosp11 plugin which has configured ofed on my nodes and we are pushing a nutron dpdk config | |
| 18:59:23 | sdague | mriedem: ok | |
| 18:59:48 | gregwork | sean-k-mooney: the deploy works great, but when i spawn instances i get that error in qemu-kvm on the compute node | |
| 19:00:03 | gregwork | sean-k-mooney: networking doesnt appear to be working (most likely due to that error) | |
| 19:00:14 | gregwork | vm boots and all but is an island | |
| 19:00:15 | sean-k-mooney | gregwork: the one you posted is indicating that the guest memory is not backed by file descriptor backed memory that is mapped shared | |
| 19:00:26 | gregwork | sean-k-mooney: yar | |
| 19:01:26 | sean-k-mooney | vhost user requires file decriptor backed shared memeory that is preallocated and locked to work | |
| 19:01:51 | sean-k-mooney | in ocata that means the guest must use hugepage backed memory or it will boot but have no networking | |
| 19:02:17 | gregwork | what am i missing from a flavor pov? im reading about some keys i need to set | |
| 19:02:47 | sean-k-mooney | yes you need to add hw:mem_page_size=large | |
| 19:03:12 | mriedem | gregwork: https://docs.openstack.org/nova/pike/admin/huge-pages.html | |
| 19:04:17 | dansmith | efried: I assume you're good with this since you pushed the latest edit? https://review.openstack.org/#/c/470575 | |
| 19:04:30 | dansmith | efried: no +1 so just wanted to make sure before I sent it to oblivio | |
| 19:04:33 | dansmith | *oblivion | |
| 19:04:45 | efried | dansmith oblivio away | |
| 19:04:56 | dansmith | efried: are you going to rebase the rest? | |
| 19:05:16 | efried | I could, if they need it. Do they need it? | |
| 19:05:21 | sean-k-mooney | mriedem: sfinucan wrote a section about vhost-user/dpdk also in the docs im trying to find that | |
| 19:05:22 | dansmith | I think so | |
| 19:05:27 | efried | ight. | |
| 19:05:31 | dansmith | efried: I think your edit(s) orphaned them | |
| 19:05:42 | openstackgerrit | Eric Fried proposed openstack/nova master: Send Allocations to spawn https://review.openstack.org/511879 | |
| 19:05:49 | efried | dansmith On it. Meanwhile ^ | |
| 19:06:14 | dansmith | efried: no, meanwhile lunch | |
| 19:06:21 | mriedem | sean-k-mooney: probably in the networking guide | |
| 19:06:24 | efried | dansmith Enjoy | |
| 19:06:56 | mriedem | https://docs.openstack.org/ocata/networking-guide/config-ovs-dpdk.html | |
| 19:07:43 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: add nested resource providers https://review.openstack.org/377138 | |
| 19:07:43 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: integrate ProviderTree to report client https://review.openstack.org/415921 | |
| 19:07:44 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807 | |
| 19:07:44 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215 | |
| 19:07:45 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693 | |
| 19:08:09 | efried | dansmith Done ^ | |
| 19:08:39 | sean-k-mooney | mriedem: gregwork yep https://docs.openstack.org/ocata/networking-guide/config-ovs-dpdk.html was what i was looking for | |
| 19:10:23 | gregwork | reading now | |
| 19:10:57 | sean-k-mooney | gregwork: that rather crypti libvirt warning + the complete lack of networking are the only signs that this requirement was not met | |
| 19:12:16 | sean-k-mooney | gregwork: by the way if you are on ubuntu you might need to aslo need to run sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd and restart libvirtd | |
| 19:12:58 | sean-k-mooney | the reason for this is apparmor may block acess to /var/run/openvswitch when qemu tries to create/connect to the vhost user socket | |
| 19:14:30 | sean-k-mooney | if you request hugepages and it fails to boot check dmesg to see if app armor blocked it and remove teh libvirt prfile and restart the libvirt service if it did | |
| 19:20:01 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215 | |
| 19:20:02 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693 | |
| 19:20:02 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807 | |
| 19:20:03 | openstackgerrit | Eric Fried proposed openstack/nova master: Reformat _get_all_with_shared https://review.openstack.org/512799 | |
| 19:21:29 | gregwork | sean-k-mooney: on rhel7 with rhosp11 | |
| 19:22:05 | sean-k-mooney | gregwork: ah ok. selinux might complain in a similar way but i think that issue only happens on ubuntu | |
| 19:22:25 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: Reformat _get_all_with_shared https://review.openstack.org/512799 | |
| 19:22:31 | sean-k-mooney | you can set selinux to permmissive if it dose as a workaround | |
| 19:22:50 | gregwork | sean-k-mooney: what hardware are you using | |
| 19:23:05 | gregwork | sean-k-mooney: we are testing connect x-5 100gbe dual qsfp cards | |
| 19:23:18 | dansmith | efried: thanks | |
| 19:23:26 | efried | yahyoubetcha. | |
| 19:23:36 | efried | Also addressed cdent -1 in the middle of that series | |
| 19:23:50 | dansmith | gregwork: sean-k-mooney: your discussion seems unrelated to nova dev (see topic), maybe you could take it to #openstack? | |
| 19:24:00 | dansmith | efried: cool | |
| 19:24:03 | sean-k-mooney | gregwork: i have tested i350,e1000,x520 and xl70(both 4*10 and 2*40G) | |
| 19:24:22 | gregwork | dansmith: sure | |
| 19:24:30 | efried | dansmith You should eat slower. Gonna give yourself indigestion. | |
| 19:24:30 | sean-k-mooney | dansmith: sure :) | |
| 19:24:37 | gregwork | actually ill just msg you directly if you dont mind | |
| 19:24:41 | dansmith | efried: I'm parallelizing | |
| 19:24:48 | efried | efficient | |
| 19:31:12 | mlavalle | johnthetubaguy, mriedem: any chance you could take a look again on https://review.openstack.org/#/c/375580/? | |
| 19:31:30 | mriedem | 0 chance miguel | |
| 19:31:43 | mriedem | i've had the tab open all morning :) | |
| 19:33:09 | mriedem | should be able to get to it before parent / teacher conferences this afternoon, which are not going to go well | |
| 19:33:13 | mriedem | dansmith: ^ you know why | |
| 19:33:23 | dansmith | lol | |
| 19:33:28 | dansmith | wish I could go | |
| 19:34:32 | mlavalle | mriedem: thanks. good luck with the parent / teacher conference :-) I remember those days | |
| 19:37:18 | efried | sean-k-mooney Took another swing at the bandwidth spec | |
| 19:43:26 | melwitt | mriedem: I wonder if the problem is with cryptsetup and not luks (in your recreate patch). in the bug report lisali mentioned dmcrypt which would imply cryptsetup IIUC | |