Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-10-17
18:38:56 artom But I have so many
18:39:26 melwitt dansmith: for example, the reserve() method passes straight through to the quota engine code and then onto the DB API code. so to prevent breaking the contract, we have to keep all of that no?
18:39:54 melwitt I had thought I could just put "pass" in them. maybe we can since reserve() didn't return anything IIRC
18:40:07 dansmith melwitt: well it depends.. if the side effects of the code are no longer desired, and the older remote can't tell a difference, then it doesn't matter
18:40:10 dansmith that's the point of the isolation
18:40:31 dansmith you're not breaking the contract by changing the behavior of the remotable, only if you change the remotely-visible behavior
18:40:48 dansmith just like we make save() do new things like generate uuids even though older clients don't expect that
18:40:54 melwitt yeah. I'm trying to think if the other side can or can't tell the difference. reserve() currently sets the 'reservations' attribute
18:40:58 dansmith that's the new behavior, we just have to make sure we honor what they're expecting
18:41:10 dansmith but not if NoOp is used right?
18:41:25 melwitt yeah not if NoOp is used
18:41:37 artom mriedem, ah, OK, the rebase thing. So your fix should merge before we merge the backport, which would squash your fix
18:41:45 melwitt so it should already be resilient to that
18:41:45 mriedem artom: yes
18:41:48 dansmith so presumably we can keep the interface stable but neuter the new-side code so it's not doing anything right?
18:41:50 artom Honestly, we can do internal backport to newton, so no massive rush
18:41:54 gregwork artom: http://www.mellanox.com/related-docs/prod_software/MLNX_DPDK_Quick_Start_Guide_v16.11_1.5.pdf
18:42:04 artom But upstream would obviously be appreciated :)
18:42:12 gregwork artom: section 3.2 is the recommended settings for starting a kvm guest
18:42:18 melwitt dansmith: yeah. looks like we can safely do that for any method overloaded by NoOp since NoOp does a pass for those
18:42:19 openstackgerrit Artom Lifshitz proposed openstack/nova stable/newton: Catch InstanceNotFound exception https://review.openstack.org/467321
18:42:34 dansmith melwitt: you'd think yeah
18:42:37 artom mriedem, ^^ that's the tenant usage instancenotfound thing
18:42:58 artom mriedem, I guess I can give you the +1 on your fix, but that's about the extent of my helping to push it through
18:43:28 melwitt dansmith: yeah, not guaranteed. some code out there might be assuming only the default driver and thus assume something like 'reservations' attribute would be set. I'm not sure if we need to account for that
18:43:29 mriedem i need a stable hero to review these: https://review.openstack.org/#/c/493206/ https://review.openstack.org/#/c/506323/ https://review.openstack.org/#/c/509441/ https://review.openstack.org/#/c/508902/
18:43:35 artom With great power comes great responsibility, with no power you just sorta chill
18:44:12 dansmith melwitt: you can probably just make sure reservations is set to something that will make it not choke right?
18:44:55 melwitt dansmith: yeah, maybe empty list would be good enough. hopefully no one could assume non-empty list
18:45:15 artom gregwork, ah, OK. So they're not saying "in your openstack instance, make sure this command line is present"
18:45:29 dansmith melwitt: I don't have the usage of that field in my head so I'd have to look, but AFAIR, it's just passed back to commit or something
18:45:37 artom gregwork, you just happen to be using openstack, whereas the doc is written for bare kvm/qemu
18:45:44 gregwork artom: right
18:45:50 melwitt commit() used to set reservations = None too, so that's a thing that's considered valid
18:45:58 artom gregwork, nova does do hugepages and CPU pinning
18:46:27 melwitt dansmith: yeah, I think only the object itself uses it. or at least that's the intention. but ppl could access it if they wanted, I don't know why they would tho
18:46:31 gregwork artom: i am getting a qemu error trying to initialize the network with dpdk, i am trying to troubleshoot by making sure qemu-kvm is being invoked correctly
18:46:51 artom gregwork, I dunno what version you're on, for for pike the docs are https://docs.openstack.org/nova/pike/admin/huge-pages.html and https://docs.openstack.org/nova/pike/admin/cpu-topologies.html
18:47:08 dansmith melwitt: fwiw, I think you're overthinking this.. just set it to something and grep around for the uses of .reservations and make sure it'll be happy.. should be easy I think :)
18:47:34 mriedem artom: +2
18:47:36 melwitt dansmith: I was getting confused by the idea of accounting for code that isn't in nova
18:47:54 dansmith melwitt: eh? no need to do that, like always
18:48:00 dansmith melwitt: what are you thinking of specifically?
18:48:11 artom mriedem, woothanks
18:48:24 gregwork artom: working with ocata atm
18:48:24 artom gregwork, wait, you're getting a qemu error while spawning a VM through nova?
18:48:28 artom That's... bad
18:48:38 gregwork artom: yeah :/
18:48:50 melwitt dansmith: I thought that was what you were getting at. nothing calls these methods anymore as of queens, so I thought it didn't matter what they return or not. but I knew I couldn't remove them if they were remotable
18:49:01 artom gregwork, and is most likely a nova and/or libvirt bug
18:49:09 artom Because that should never happen
18:49:49 dansmith melwitt: no, I'm saying we should keep the contract until we bump, but not for out-of-nova uses, just in case someone has an ocata compute running by accident or something so we do our best to not just cause it to break in weird ways
18:49:50 artom Well, assuming the hardware actually supports what you're trying to do, and you've configured nova correctly
18:49:58 gregwork artom: i dont think so, i believe this is related to ovs being configured to use mellanox dpdk poll mode driver, and kvm is not setting up the guest correctly to plumb the networking
18:50:07 artom But even a misconfiguration shouldn't go all the way down to a qemu error
18:50:22 dansmith melwitt: since it's all conductor-side for those actions, just attempt to fake the result that won't cause any of our code in the last few cycles to choke and call it good
18:50:34 mriedem sdague: can i get you to poke through these stable/newton changes? https://review.openstack.org/#/c/493206/ https://review.openstack.org/#/c/506323/ https://review.openstack.org/#/c/509441/ https://review.openstack.org/#/c/508902/
18:50:37 mriedem tomorrow is newton-eol
18:50:52 gregwork artom: eg. https://paste.fedoraproject.org/paste/GWEv1HyVo1y9sekJbZFBeQ
18:51:00 melwitt dansmith: oh, gotcha. I understand now. I'll go over it again and look for those possibilities and update it. thanks
18:51:20 melwitt I at least can remove the non-remotable one in a new PS
18:52:00 artom gregwork, you're very rapidly bringing my out of my (admittedly limited) comfort zone :/
18:52:58 gregwork artom: so i am testing vnf's on my openstack, where the compute nodes are running ovs in dpdk mode, with a poll mode driver (the nic is not managed by the kernel)
18:54:49 artom gregwork, sorry I can't be of more help, but I'm out of my league here
18:55:14 mriedem gregwork: sean-k-mooney might be able to help with dpdk questions
18:55:32 artom We have some folks from mellanox who hang out here, but they're (I think) on Israel time
18:55:42 mriedem moshe is yeah
18:55:43 sean-k-mooney mriedem: gregwork sure let me read back
18:56:31 sean-k-mooney gregwork: so yes you have a coumpute node with ovs-dpdk. what is the question?
18:57:20 sean-k-mooney gregwork: ah you dont have hugepages requested for the vm
18:58:12 sdague mriedem: done. I was a little surprised you wanted some of those back to newton, but that's fine
18:58:35 gregwork sean-k-mooney: have you seen that error before ?
18:58:42 gregwork the one i posted in the pastebin
18:58:49 mriedem sdague: the bfv one was needed for another thing someone was reporting in newton but w/o better debug couldn't be determined
18:59:23 sdague mriedem: ok
18:59:23 gregwork sean-k-mooney: im using the mellanox rhosp11 plugin which has configured ofed on my nodes and we are pushing a nutron dpdk config
18:59:48 gregwork sean-k-mooney: the deploy works great, but when i spawn instances i get that error in qemu-kvm on the compute node
19:00:03 gregwork sean-k-mooney: networking doesnt appear to be working (most likely due to that error)
19:00:14 gregwork vm boots and all but is an island
19:00:15 sean-k-mooney gregwork: the one you posted is indicating that the guest memory is not backed by file descriptor backed memory that is mapped shared
19:00:26 gregwork sean-k-mooney: yar
19:01:26 sean-k-mooney vhost user requires file decriptor backed shared memeory that is preallocated and locked to work
19:01:51 sean-k-mooney in ocata that means the guest must use hugepage backed memory or it will boot but have no networking
19:02:17 gregwork what am i missing from a flavor pov? im reading about some keys i need to set
19:02:47 sean-k-mooney yes you need to add hw:mem_page_size=large
19:03:12 mriedem gregwork: https://docs.openstack.org/nova/pike/admin/huge-pages.html
19:04:17 dansmith efried: I assume you're good with this since you pushed the latest edit? https://review.openstack.org/#/c/470575
19:04:30 dansmith efried: no +1 so just wanted to make sure before I sent it to oblivio
19:04:33 dansmith *oblivion
19:04:45 efried dansmith oblivio away
19:04:56 dansmith efried: are you going to rebase the rest?
19:05:16 efried I could, if they need it. Do they need it?
19:05:21 sean-k-mooney mriedem: sfinucan wrote a section about vhost-user/dpdk also in the docs im trying to find that
19:05:22 dansmith I think so
19:05:27 efried ight.
19:05:31 dansmith efried: I think your edit(s) orphaned them
19:05:42 openstackgerrit Eric Fried proposed openstack/nova master: Send Allocations to spawn https://review.openstack.org/511879
19:05:49 efried dansmith On it. Meanwhile ^
19:06:14 dansmith efried: no, meanwhile lunch
19:06:21 mriedem sean-k-mooney: probably in the networking guide

Earlier   Later