Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-08-26
16:09:13 rmart04 Yes
16:09:23 sean-k-mooney ok so in that case you should use hugepages
16:09:48 sean-k-mooney using explcit small pages could help but there hugepages will improve performance
16:09:53 openstackgerrit Balazs Gibizer proposed openstack/nova master: Prevent starting services with older than N-1 computes https://review.opendev.org/738482
16:10:06 sean-k-mooney you are already using pinning so you cannot live migrete
16:10:15 sean-k-mooney adding hugepages wont make that worse in this case
16:10:30 rmart04 ok JohnGarbut also suggested this to me recently, i'll look into it and see how i get on. Does the host need any explicit configuration or is it just flavor metadata specs?
16:10:53 sean-k-mooney rmart04: the host needs to have memroy preaccloated as hugepages
16:11:02 sean-k-mooney you can do tha that on the kernel commandline
16:11:53 sean-k-mooney you have 768 GB and are running a 750GB guest right
16:12:06 sean-k-mooney so you would allocate 750GB of 1GB hugepages ideally
16:12:23 sean-k-mooney then set hw:mem_page_size=large in the flavor
16:12:58 sean-k-mooney default_hugepagesz=1G hugepagesz=1G hugepages=750
16:13:10 sean-k-mooney that is what you need to add to the kernel command line in grub
16:13:47 rmart04 ah OK amazing, thank you
16:13:47 sean-k-mooney you have only 1 numa node on the host currently right?
16:13:53 rmart04 No two
16:14:14 sean-k-mooney and the vm has how many numa nodes? 2
16:14:22 rmart04 yes
16:14:27 sean-k-mooney ok then that makes sense
16:14:37 sean-k-mooney otherwise i was wondering how the vm booted :)
16:15:03 rmart04 I managed to get over that hurdle :P
16:15:17 sean-k-mooney rmart04: you should see >20% incresse form hugepages in most workloads
16:15:27 sean-k-mooney more if its memory or io bound
16:15:35 sean-k-mooney less if it compute bound
16:15:41 sean-k-mooney or disk bound
16:17:50 rmart04 ok great, thanks for all your help
16:18:38 sean-k-mooney by the way the reason i ask about oversubsrcitip is hugepage cant be swapped out ot disk but they also will never be reap as a result of OOM
16:18:56 sean-k-mooney so it shoudl solve your issue although there is a very small change that the qemu process can still be selected
16:20:00 sean-k-mooney qemu will be using relitvly little non hugepage memeory in this config so it will have a low memery pressur score vs other process and memeory will be balance evenly across both host numa nodes for the vm
16:20:53 yoctozepto morning nova folks - today's question is when nova compute host name can get different from nova hypervisor host name and how
16:21:43 sean-k-mooney yoctozepto: it will be different for ironic
16:22:00 yoctozepto sean-k-mooney: true, let's focus on nova+kvm :-)
16:22:16 sean-k-mooney out side of that it shoudl be the same unless you set teh HOST config element or you add a fqdn to you /etc/hostname
16:22:17 yoctozepto actually evaluating noonedeadpunk's https://review.opendev.org/728629 (for masakari)
16:23:03 stephenfin yoctozepto: You can manually override '[DEFAULT] host'
16:23:09 sean-k-mooney yoctozepto: this is how its set https://github.com/openstack/nova/blob/master/nova/conf/netconf.py#L50-L70
16:23:19 stephenfin it defaults to socket.gethostname()
16:23:24 stephenfin what sean-k-mooney just linked to
16:23:33 sean-k-mooney yep
16:23:42 sean-k-mooney so they should be the same unless you change that config option
16:24:04 sean-k-mooney if you use socket.fqdn() in your code it may not agreee
16:24:08 sean-k-mooney i fixed this in cyborg
16:24:14 yoctozepto then I would be changing the thing for nova-compute but not for the hypervisor, right?
16:24:32 sean-k-mooney yes
16:24:46 sean-k-mooney if you set [default]/host to something
16:25:03 yoctozepto ok, so where does the name for the hypervisor spawn and how's it kept linked? :D
16:25:03 sean-k-mooney hyperviour_hostname and host will not be the same for libvirt
16:25:08 sean-k-mooney or other virt dirvers
16:25:54 sean-k-mooney host is the compute service host and is set by that config and hypervisor_hostname is set by the virt driver
16:26:04 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Don't inherit from 'ProviderUsageBaseTestCase' https://review.opendev.org/748271
16:26:13 sean-k-mooney the libvirt driver get it form libvirt
16:26:25 sean-k-mooney and it calls socket.gethostname internally in c
16:26:29 stephenfin bauzas: https://review.opendev.org/748271
16:26:52 sean-k-mooney socket.gethostname is a fucntion provided by libc and pythons socket module jsut call the libc version
16:27:19 sean-k-mooney socket.gethostname() reads /etc/hostname if it exits on linux
16:27:23 yoctozepto sean-k-mooney: ah, so it's consulted each time with libvirt? I guess that's why I had a hard time figuring this one out
16:27:32 sean-k-mooney yes
16:27:37 yoctozepto yeah, the rest makes sense
16:27:40 stephenfin sean-k-mooney: https://review.opendev.org/#/c/748250/
16:27:49 yoctozepto I do wonder how noonedeadpunk (and some others) ended up desynchronizing them
16:28:05 bauzas stephenfin: I don't get your comment, you're saying you can't just use the InstanceMixin ?
16:28:08 noonedeadpunk I have situnation where socket.gethostname and socket.getfqdn are different things
16:28:13 bauzas stephenfin: directly I mean ?
16:28:31 yoctozepto noonedeadpunk: hmm, ah
16:28:40 yoctozepto but then sean-k-mooney was only about gethostname
16:28:52 yoctozepto you mean libvirt is actually doing getfqdn?
16:28:52 bauzas stephenfin: of course, you'd have to specify the latest microversion but is that terrible to do ?
16:29:01 stephenfin bauzas: Not really. I'd have to copy-paste all of this https://github.com/openstack/nova/blob/master/nova/tests/functional/integrated_helpers.py#L1107-L1132
16:29:10 sean-k-mooney yoctozepto: no its getting the hostname
16:29:23 stephenfin bauzas: Which seems worse than having functions available that I'm not using
16:29:51 noonedeadpunk sean-k-mooney: but I think that libvirt checks for the socket.getfqdn...
16:29:52 sean-k-mooney stephenfin: lol ok
16:29:54 stephenfin '_IntegratedTestBase' is basically what you're asking for but with all that boilerplate inline
16:29:59 yoctozepto then there is a conflict between your testimonies, noonedeadpunk and sean-k-mooney :-)
16:30:00 sean-k-mooney noonedeadpunk: it does not
16:30:05 yoctozepto investigation ongoing!
16:30:07 yoctozepto :D
16:30:12 sean-k-mooney noonedeadpunk: unless they changed it wich would be a breaking change
16:30:20 stephenfin there are some differences around policy but I'm resolving those in separate patches
16:30:57 noonedeadpunk sean-k-mooney: http://paste.openstack.org/show/797181/
16:31:03 yoctozepto https://github.com/libvirt/libvirt/blob/7ba4838a18afb609d900190c57b0cbfb842ccbdf/src/util/virutil.c#L470
16:31:08 noonedeadpunk I can judge only by this...
16:31:25 stephenfin melwitt: want to unbork cloning on Windows? https://review.opendev.org/#/c/748250/
16:31:43 yoctozepto it seems they are training to canonicalize on master
16:31:51 yoctozepto that's how they ended up with an fqdn
16:31:55 bauzas stephenfin: technically, you'd need to do things like https://github.com/openstack/nova/blob/master/nova/tests/functional/regressions/test_bug_1849409.py
16:32:09 sean-k-mooney yoctozepto: noonedeadpunk we cant change this behavior
16:32:12 yoctozepto so yeah, they b0rked
16:32:14 sean-k-mooney its used in plamcnet
16:32:14 bauzas stephenfin: but that should be it
16:32:27 yoctozepto sean-k-mooney: what's plamcnet? ;d
16:32:42 noonedeadpunk sean-k-mooney: yeah, that's fine) just sharing feedback about fqdn vs gethostname - fighting this 3 years I guess...
16:32:49 sean-k-mooney the placement service
16:32:56 yoctozepto noonedeadpunk: but now we know
16:32:58 yoctozepto sean-k-mooney: ack
16:33:23 elod lyarwood: no need to squash, it's good as it is. will review now
16:33:31 sean-k-mooney noonedeadpunk: it is technially allowed to return an fqdn if you put an fqdn in /etc/hostname which you should not do but some people do
16:33:33 stephenfin bauzas: Yes, but those are nearly identical. The only difference is the use of the CinderFixture and fake image service (essentially GlanceFixture)
16:33:34 yoctozepto ok, this was a very worthwhile chat, thanks sean-k-mooney, stephenfin and noonedeadpunk

Earlier   Later