Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-22
23:06:28 sean-k-mooney redhat has an openstack distobution that build on top of the triplo project
23:06:43 sean-k-mooney you would be surprised
23:07:03 sean-k-mooney some do if they have the devepolment staff to mainatin it
23:07:25 sean-k-mooney but its more common to see large companies contibuie and addopt the comuntiy installer
23:07:37 sean-k-mooney *installers
23:08:41 lucidguy Just sounds like a pain to have to troubleshoot another layer of software.
23:10:28 sean-k-mooney well when that software is built by a group of operators that have really clouds in produciton you would be suprised at how helpful it can be
23:10:53 sean-k-mooney that said i agree if its a complex installer it can be a pain to debug
23:11:24 sean-k-mooney so i tend to prefer the simpler ones like devstack and kolla-ansible
23:11:37 sean-k-mooney just never run devstack in production...
23:14:59 lucidguy ugg stupid instance stuck in deleting state.. and virsh on hv not responding FML
23:15:44 sean-k-mooney that sounds like the host kernel might have hung
23:16:18 lucidguy this is the box I disabled nesting
23:16:19 sean-k-mooney by the way i hope if you are runing 1TB VM you are using Hugepages for the memory
23:16:36 lucidguy Whatever the default option is.
23:16:44 sean-k-mooney so no you are not
23:17:12 sean-k-mooney the reason i mentioned that is it can take a long time for qemu/kvm to release all the mapped memory when you kill it
23:17:16 lucidguy I was just reading those options in nova.conf
23:17:40 sean-k-mooney so if you are using the decault 4k memory then it might take a while to fully delete the vm
23:18:19 lucidguy Would I not see a load on the system anywhere?
23:19:37 lucidguy I don't even see a qemu process running
23:21:05 sean-k-mooney you would see that the memory has not been freed with free -m
23:21:24 sean-k-mooney but in that case if virsh is not working i would check to see if libvirt is still running
23:21:31 lucidguy Its free
23:21:50 lucidguy libvirtError: failed to connect to monitor socket: No such process
23:22:16 sean-k-mooney so that mean libvirt tried to connecct to qemu but it failed because it was not running
23:22:23 lucidguy oooo its gone. So it did take forever.
23:22:44 lucidguy Now you're going to make me research how Hugepages works. Thanks alot. heh
23:23:05 sean-k-mooney hehe ya they are non trival
23:23:24 sean-k-mooney but basically it invole preallcoating meory so that it can be handed out in larger chunks
23:23:35 sean-k-mooney normaly either 2MB or 1G instead of 4k
23:23:56 sean-k-mooney that signicatly impoves memory performance but it has other implciations
23:24:32 lucidguy Won't the mem be used less efficiently?
23:24:34 sean-k-mooney the main onces beign non memory over subsription and no live migation suppor until train
23:24:41 sean-k-mooney no
23:25:09 sean-k-mooney by using larger page allocation you actully optimeis adress translation
23:25:49 sean-k-mooney the translation lookaside buffer can only cache a limite number of page translations. if each page is bigger you can cache the adress to more memory
23:26:24 sean-k-mooney so it can give 30%+ performanc improment if you applciation is memory or latency sensitive
23:26:59 lucidguy Tried launching a new intance on that HV without nexting.. instance comes up paused and I can see that same error in libvirt .. :(
23:27:17 lucidguy Was worth a try.
23:27:24 sean-k-mooney ya sorry it did not work
23:28:23 lucidguy Was so excited about that. Installing a 5.0 kernel on a 16.04 box in production I don't think is wise
23:28:52 sean-k-mooney i run 5.4 on my home server based on ubuntu 18.04
23:29:14 sean-k-mooney but its really something you need to evaluate for your own usecase
23:29:27 lucidguy Did you basically upgrade using this approach? https://www.tecmint.com/upgrade-kernel-in-ubuntu/
23:30:20 sean-k-mooney no i was lazy and used ukuu the ubuntu kernel update utility
23:30:24 sean-k-mooney it automates that
23:30:37 lucidguy ohh, theres a cli approach.
23:30:37 sean-k-mooney but i am running the 5.4 mainline kernel form that ppa
23:31:07 sean-k-mooney https://www.omgubuntu.co.uk/2017/02/ukuu-easy-way-to-install-mainline-kernel-ubuntu
23:32:01 sean-k-mooney again you proably shoudl not do that in prodcutin but i basically un the latest upstream lts kernel which is 5.4 a the moment
23:32:17 sean-k-mooney 5.0 was the previous upstream long life kernel i think
23:33:14 sean-k-mooney oh the previous long term support was 4.19
23:33:18 lucidguy Things to think about. Thanks again. Have to run. Wife is hating me. heh
23:33:31 sean-k-mooney https://www.kernel.org/category/releases.html
23:33:37 sean-k-mooney no worries o/
23:34:50 lucidguy I guess I should start by updating my local repo and giving 4.19 a try. Sadly I'm not optimistic of the outcome.
23:35:11 lucidguy have a good one
23:55:21 efried jroll: we had talked vaguely about you being involved in the development of the CI, so I thought to involve you in that conversation :)
#openstack-nova - 2020-01-23
08:31:30 openstackgerrit Merged openstack/nova master: Func test for failed and aborted live migration https://review.opendev.org/699065
08:31:36 openstackgerrit Merged openstack/nova master: Remove unused code https://review.opendev.org/703408
09:27:23 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove remaining Python 2.7-only dependencies https://review.opendev.org/698895
09:33:54 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove 'nova.image.api' module https://review.opendev.org/702451
09:33:54 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Update API reference guide https://review.opendev.org/703796
09:33:55 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: nova-net: Remove unused nova-network objects https://review.opendev.org/697156
11:21:15 openstackgerrit Merged openstack/nova master: Switch to hacking 2.x https://review.opendev.org/695733
11:21:20 openstackgerrit Merged openstack/nova master: requirements: Limit hacking to one minor version https://review.opendev.org/703405
11:32:52 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unused nova-network objects https://review.opendev.org/697156
11:32:53 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unused parameters https://review.opendev.org/703974
11:32:53 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unnecessary exception handling https://review.opendev.org/703973
11:34:09 stephenfin dansmith: I'm doing possibly bad stuff with objects in https://review.opendev.org/697156. I _think_ it's okay but would appreciate your input when you're around
11:34:34 stephenfin dansmith: Also want to figure out how to move objects between projects (nova -> os-vif) since that's seems like an obvious thing to do in that case
11:34:42 stephenfin sean-k-mooney: Maybe relevant, idk ^
11:43:47 sean-k-mooney we should not move the network object to os-vif
11:43:53 sean-k-mooney we already have one
11:44:43 sean-k-mooney the os vif objects are constucted from info often taken form the nova ones but they are not the same objects
11:57:19 stephenfin sean-k-mooney: Damn. Okay, maybe you can figure out what's going on. Fancy pulling down the earlier version of that change along with this patch for o.vo https://review.opendev.org/#/c/703970/ ?
11:58:05 gibi kashyap: rng patch looks good to me, thanks for the update
11:58:20 lucidguy sean-k-mooney: The only option I see regarding hugepages is reserved_huge_pages. Is that what you were talking about?
11:58:24 stephenfin If you install that version of the o.vo in the venv and run 'tox -e py36 -- -n nova/tests/unit/objects/test_objects.py::TestObjectVersions.test_obj_make_compatible', it'll fail saying it can't find the 'Network' object
11:58:42 sean-k-mooney stephenfin: i know dansmith found that the namespace stuff does not seam to be working properly
11:59:29 stephenfin did he now?
11:59:38 stephenfin any notes or bugs on that?
11:59:55 sean-k-mooney lucidguy: no, hugepages is a afeature that is enabled in the flavor/image by setting hw:mem_page_size=large
12:00:38 sean-k-mooney reserved_huge_pages is intended to allow you say that some of the hugepages on the host are reserved for host use and cannot be used by vms
12:01:00 sean-k-mooney stephenfin: it cam up on one of his patch let me see if it merged or not
12:01:02 lucidguy sean-k-mooney: Understood
12:01:25 sean-k-mooney stephenfin: https://review.opendev.org/#/c/702049/
12:01:33 sean-k-mooney the ephemeral object patch
12:02:45 sean-k-mooney stephenfin: line 1175 https://review.opendev.org/#/c/702049/4/nova/tests/unit/objects/test_objects.py
12:03:26 sean-k-mooney lucidguy: i think these are the docs on hugepage that you want https://docs.openstack.org/nova/latest/admin/huge-pages.html
12:03:47 sean-k-mooney lucidguy: we have summit presintaion on this too that stephenfin did
12:04:58 lucidguy I was just looking at that link
12:05:02 lucidguy Cool
12:07:28 kashyap gibi: Hi, thanks for looking!
12:09:05 kashyap gibi: Want me to respin removing the stray word "this"?
12:09:37 gibi kashyap: if you respin quickly then I can re +2 it. but I'm fine merging the patch as is
12:09:38 sean-k-mooney stephenfin: i think you need to update https://github.com/openstack/nova/blob/master/nova/tests/unit/objects/test_objects.py#L1225
12:09:58 stephenfin sean-k-mooney: incoming :)
12:10:25 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Make the security group API a module https://review.opendev.org/696746

Earlier   Later