| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-17 | |||
| 18:45:44 | gregwork | artom: right | |
| 18:45:50 | melwitt | commit() used to set reservations = None too, so that's a thing that's considered valid | |
| 18:45:58 | artom | gregwork, nova does do hugepages and CPU pinning | |
| 18:46:27 | melwitt | dansmith: yeah, I think only the object itself uses it. or at least that's the intention. but ppl could access it if they wanted, I don't know why they would tho | |
| 18:46:31 | gregwork | artom: i am getting a qemu error trying to initialize the network with dpdk, i am trying to troubleshoot by making sure qemu-kvm is being invoked correctly | |
| 18:46:51 | artom | gregwork, I dunno what version you're on, for for pike the docs are https://docs.openstack.org/nova/pike/admin/huge-pages.html and https://docs.openstack.org/nova/pike/admin/cpu-topologies.html | |
| 18:47:08 | dansmith | melwitt: fwiw, I think you're overthinking this.. just set it to something and grep around for the uses of .reservations and make sure it'll be happy.. should be easy I think :) | |
| 18:47:34 | mriedem | artom: +2 | |
| 18:47:36 | melwitt | dansmith: I was getting confused by the idea of accounting for code that isn't in nova | |
| 18:47:54 | dansmith | melwitt: eh? no need to do that, like always | |
| 18:48:00 | dansmith | melwitt: what are you thinking of specifically? | |
| 18:48:11 | artom | mriedem, woothanks | |
| 18:48:24 | gregwork | artom: working with ocata atm | |
| 18:48:24 | artom | gregwork, wait, you're getting a qemu error while spawning a VM through nova? | |
| 18:48:28 | artom | That's... bad | |
| 18:48:38 | gregwork | artom: yeah :/ | |
| 18:48:50 | melwitt | dansmith: I thought that was what you were getting at. nothing calls these methods anymore as of queens, so I thought it didn't matter what they return or not. but I knew I couldn't remove them if they were remotable | |
| 18:49:01 | artom | gregwork, and is most likely a nova and/or libvirt bug | |
| 18:49:09 | artom | Because that should never happen | |
| 18:49:49 | dansmith | melwitt: no, I'm saying we should keep the contract until we bump, but not for out-of-nova uses, just in case someone has an ocata compute running by accident or something so we do our best to not just cause it to break in weird ways | |
| 18:49:50 | artom | Well, assuming the hardware actually supports what you're trying to do, and you've configured nova correctly | |
| 18:49:58 | gregwork | artom: i dont think so, i believe this is related to ovs being configured to use mellanox dpdk poll mode driver, and kvm is not setting up the guest correctly to plumb the networking | |
| 18:50:07 | artom | But even a misconfiguration shouldn't go all the way down to a qemu error | |
| 18:50:22 | dansmith | melwitt: since it's all conductor-side for those actions, just attempt to fake the result that won't cause any of our code in the last few cycles to choke and call it good | |
| 18:50:34 | mriedem | sdague: can i get you to poke through these stable/newton changes? https://review.openstack.org/#/c/493206/ https://review.openstack.org/#/c/506323/ https://review.openstack.org/#/c/509441/ https://review.openstack.org/#/c/508902/ | |
| 18:50:37 | mriedem | tomorrow is newton-eol | |
| 18:50:52 | gregwork | artom: eg. https://paste.fedoraproject.org/paste/GWEv1HyVo1y9sekJbZFBeQ | |
| 18:51:00 | melwitt | dansmith: oh, gotcha. I understand now. I'll go over it again and look for those possibilities and update it. thanks | |
| 18:51:20 | melwitt | I at least can remove the non-remotable one in a new PS | |
| 18:52:00 | artom | gregwork, you're very rapidly bringing my out of my (admittedly limited) comfort zone :/ | |
| 18:52:58 | gregwork | artom: so i am testing vnf's on my openstack, where the compute nodes are running ovs in dpdk mode, with a poll mode driver (the nic is not managed by the kernel) | |
| 18:54:49 | artom | gregwork, sorry I can't be of more help, but I'm out of my league here | |
| 18:55:14 | mriedem | gregwork: sean-k-mooney might be able to help with dpdk questions | |
| 18:55:32 | artom | We have some folks from mellanox who hang out here, but they're (I think) on Israel time | |
| 18:55:42 | mriedem | moshe is yeah | |
| 18:55:43 | sean-k-mooney | mriedem: gregwork sure let me read back | |
| 18:56:31 | sean-k-mooney | gregwork: so yes you have a coumpute node with ovs-dpdk. what is the question? | |
| 18:57:20 | sean-k-mooney | gregwork: ah you dont have hugepages requested for the vm | |
| 18:58:12 | sdague | mriedem: done. I was a little surprised you wanted some of those back to newton, but that's fine | |
| 18:58:35 | gregwork | sean-k-mooney: have you seen that error before ? | |
| 18:58:42 | gregwork | the one i posted in the pastebin | |
| 18:58:49 | mriedem | sdague: the bfv one was needed for another thing someone was reporting in newton but w/o better debug couldn't be determined | |
| 18:59:23 | sdague | mriedem: ok | |
| 18:59:23 | gregwork | sean-k-mooney: im using the mellanox rhosp11 plugin which has configured ofed on my nodes and we are pushing a nutron dpdk config | |
| 18:59:48 | gregwork | sean-k-mooney: the deploy works great, but when i spawn instances i get that error in qemu-kvm on the compute node | |
| 19:00:03 | gregwork | sean-k-mooney: networking doesnt appear to be working (most likely due to that error) | |
| 19:00:14 | gregwork | vm boots and all but is an island | |
| 19:00:15 | sean-k-mooney | gregwork: the one you posted is indicating that the guest memory is not backed by file descriptor backed memory that is mapped shared | |
| 19:00:26 | gregwork | sean-k-mooney: yar | |
| 19:01:26 | sean-k-mooney | vhost user requires file decriptor backed shared memeory that is preallocated and locked to work | |
| 19:01:51 | sean-k-mooney | in ocata that means the guest must use hugepage backed memory or it will boot but have no networking | |
| 19:02:17 | gregwork | what am i missing from a flavor pov? im reading about some keys i need to set | |
| 19:02:47 | sean-k-mooney | yes you need to add hw:mem_page_size=large | |
| 19:03:12 | mriedem | gregwork: https://docs.openstack.org/nova/pike/admin/huge-pages.html | |
| 19:04:17 | dansmith | efried: I assume you're good with this since you pushed the latest edit? https://review.openstack.org/#/c/470575 | |
| 19:04:30 | dansmith | efried: no +1 so just wanted to make sure before I sent it to oblivio | |
| 19:04:33 | dansmith | *oblivion | |
| 19:04:45 | efried | dansmith oblivio away | |
| 19:04:56 | dansmith | efried: are you going to rebase the rest? | |
| 19:05:16 | efried | I could, if they need it. Do they need it? | |
| 19:05:21 | sean-k-mooney | mriedem: sfinucan wrote a section about vhost-user/dpdk also in the docs im trying to find that | |
| 19:05:22 | dansmith | I think so | |
| 19:05:27 | efried | ight. | |
| 19:05:31 | dansmith | efried: I think your edit(s) orphaned them | |
| 19:05:42 | openstackgerrit | Eric Fried proposed openstack/nova master: Send Allocations to spawn https://review.openstack.org/511879 | |
| 19:05:49 | efried | dansmith On it. Meanwhile ^ | |
| 19:06:14 | dansmith | efried: no, meanwhile lunch | |
| 19:06:21 | mriedem | sean-k-mooney: probably in the networking guide | |
| 19:06:24 | efried | dansmith Enjoy | |
| 19:06:56 | mriedem | https://docs.openstack.org/ocata/networking-guide/config-ovs-dpdk.html | |
| 19:07:43 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: integrate ProviderTree to report client https://review.openstack.org/415921 | |
| 19:07:43 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: add nested resource providers https://review.openstack.org/377138 | |
| 19:07:44 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215 | |
| 19:07:44 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807 | |
| 19:07:45 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693 | |
| 19:08:09 | efried | dansmith Done ^ | |
| 19:08:39 | sean-k-mooney | mriedem: gregwork yep https://docs.openstack.org/ocata/networking-guide/config-ovs-dpdk.html was what i was looking for | |
| 19:10:23 | gregwork | reading now | |
| 19:10:57 | sean-k-mooney | gregwork: that rather crypti libvirt warning + the complete lack of networking are the only signs that this requirement was not met | |
| 19:12:16 | sean-k-mooney | gregwork: by the way if you are on ubuntu you might need to aslo need to run sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd and restart libvirtd | |
| 19:12:58 | sean-k-mooney | the reason for this is apparmor may block acess to /var/run/openvswitch when qemu tries to create/connect to the vhost user socket | |
| 19:14:30 | sean-k-mooney | if you request hugepages and it fails to boot check dmesg to see if app armor blocked it and remove teh libvirt prfile and restart the libvirt service if it did | |
| 19:20:01 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: allow filter providers in tree https://review.openstack.org/377215 | |
| 19:20:02 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: adds REST API for nested providers https://review.openstack.org/384807 | |
| 19:20:02 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: update client to set parent provider https://review.openstack.org/385693 | |
| 19:20:03 | openstackgerrit | Eric Fried proposed openstack/nova master: Reformat _get_all_with_shared https://review.openstack.org/512799 | |
| 19:21:29 | gregwork | sean-k-mooney: on rhel7 with rhosp11 | |
| 19:22:05 | sean-k-mooney | gregwork: ah ok. selinux might complain in a similar way but i think that issue only happens on ubuntu | |
| 19:22:25 | openstackgerrit | Eric Fried proposed openstack/nova master: placement: Reformat _get_all_with_shared https://review.openstack.org/512799 | |
| 19:22:31 | sean-k-mooney | you can set selinux to permmissive if it dose as a workaround | |
| 19:22:50 | gregwork | sean-k-mooney: what hardware are you using | |
| 19:23:05 | gregwork | sean-k-mooney: we are testing connect x-5 100gbe dual qsfp cards | |
| 19:23:18 | dansmith | efried: thanks | |
| 19:23:26 | efried | yahyoubetcha. | |
| 19:23:36 | efried | Also addressed cdent -1 in the middle of that series | |
| 19:23:50 | dansmith | gregwork: sean-k-mooney: your discussion seems unrelated to nova dev (see topic), maybe you could take it to #openstack? | |
| 19:24:00 | dansmith | efried: cool | |
| 19:24:03 | sean-k-mooney | gregwork: i have tested i350,e1000,x520 and xl70(both 4*10 and 2*40G) | |
| 19:24:22 | gregwork | dansmith: sure | |
| 19:24:30 | sean-k-mooney | dansmith: sure :) | |