Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-06
22:59:43 sean-k-mooney cfriesen: im sure you know this also but just requesting guest numa nodes was never ment to give any form of cpu affinity either. from an api perspcetive it does not but for libvir it pince the cores of that numa node to float over a host numa node
23:00:44 sean-k-mooney cfriesen: similay requsting hugepage was not ment to create a numa topology for the guest and also restice the floating cores to a host numa node
23:01:17 sean-k-mooney cfriesen: all of those implit behaviors are internal implentation behaviors of the libvirt driver
23:01:50 cfriesen sean-k-mooney: the hugepage one is a qemu limitation though since you can only point at a single backing file for memory
23:01:57 cfriesen (on a given guest numa node)
23:03:04 sean-k-mooney cfriesen: yes and no you numa node number is optionaly and numactl will be used to determin the optimal numa node to use of you dont specify one
23:04:02 cfriesen sean-k-mooney: how do you tell qemu to use hugepages without specifying a backing file?
23:05:05 sean-k-mooney cfriesen: you do specify a backing file but the backing file itesf does not have numa affintiy
23:06:02 sean-k-mooney cfriesen: when you create a hugepage backing file under /dev/hugepages/libvrt/qemu the numa node is not part of the path
23:06:45 sean-k-mooney cfriesen: checkout https://libvirt.org/formatdomain.html#elementsMemoryBacking the note on the nodeset being optional
23:10:03 sean-k-mooney cfriesen: tacking hugepages per numa node made the resouce track code simpeler which is why it was doen per numa node originally
23:17:30 cfriesen sean-k-mooney: ah, right. I was thinking that mapping hugetlbfs was numa-affined, but it's not necessarily.
23:18:04 sean-k-mooney cfriesen: you might be able to do somthing via the mount options but by defualt no
23:19:19 sean-k-mooney efried: actully speaking of memory and numa we should really be tracking 4k pages e.g. non hugepage memory per numa node too
23:19:46 sean-k-mooney the kernel out of memory killer is triggered per numa node not gloably
23:20:06 sean-k-mooney so the reserved memory config option does not work properly on a numa system
23:20:53 sean-k-mooney vms can still be killed if a numa node has no memoy left and the kernel need to allocate some on that node but the host othersise has free ram
23:21:33 sean-k-mooney our just make all your vms use hugepages that works too
23:25:06 cfriesen that's what we do by default (2M pages)
23:26:36 sean-k-mooney cfriesen: ya we spent 6 months trying to track down what was causeing OOM to kill our vms in the the intel nfv ci and it turned out to be this. hugepages fixed it
23:27:31 sean-k-mooney the anoying thing was it took about 2-3 weeks to start showing up as it need the memory to become fragmented enough to force it
23:42:38 openstackgerrit Patricia Domingues proposed openstack/nova master: load up the volume drivers by checking architecture https://review.openstack.org/541393
23:44:22 openstackgerrit Patricia Domingues proposed openstack/nova master: load up the volume drivers by checking architecture https://review.openstack.org/541393
23:58:46 openstackgerrit Matt Riedemann proposed openstack/nova master: Add a prelude release note for the 17.0.0 Queens GA https://review.openstack.org/541495
#openstack-nova - 2018-02-07
00:22:28 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Fix pike GA prelude release note https://review.openstack.org/541498
00:44:23 openstackgerrit Merged openstack/nova master: docs: Add booting from an encrypted volume https://review.openstack.org/540506
01:06:42 openstackgerrit Arvind proposed openstack/nova-specs master: BP Support traits in Glance https://review.openstack.org/541502
01:51:06 openstackgerrit Arvind proposed openstack/nova-specs master: Support traits in Glance https://review.openstack.org/541507
02:19:16 openstackgerrit Merged openstack/nova master: libvirt: fix native luks encryption failure to find volume_id https://review.openstack.org/539739
02:19:28 mriedem woo finally
02:19:32 openstackgerrit Merged openstack/nova master: Remove instance.keypairs migration code https://review.openstack.org/517158
02:19:44 openstackgerrit Merged openstack/nova master: Fix the format file name https://review.openstack.org/523010
02:19:54 openstackgerrit Merged openstack/nova master: Add regression test for bug 1735407 https://review.openstack.org/526095
02:19:55 openstack bug 1735407 in OpenStack Compute (nova) "[Nova] Evacuation doesn't respect anti-affinity rules" [Medium,In progress] https://launchpad.net/bugs/1735407 - Assigned to Balazs Gibizer (balazs-gibizer)
02:20:57 openstackgerrit Nakanishi Tomotaka proposed openstack/nova master: Test Compute API in multiple cells https://review.openstack.org/534116
02:39:36 openstackgerrit chenpengzi proposed openstack/nova master: update docstring param discreption https://review.openstack.org/541519
02:56:38 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: doc: merge numa.rst to cpu-topologies.rst https://review.openstack.org/540585
03:01:34 openstackgerrit chenpengzi proposed openstack/nova master: update docstring param description https://review.openstack.org/541519
03:45:37 openstackgerrit Arvind proposed openstack/nova-specs master: Support traits in Glance https://review.openstack.org/541535
03:46:36 openstackgerrit Merged openstack/nova master: Add late server group policy check to rebuild https://review.openstack.org/525242
03:46:47 openstackgerrit Merged openstack/nova master: doc: mark the max microversions for queens https://review.openstack.org/539978
03:48:00 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: Not use thread alloc policy for emulator thread https://review.openstack.org/538700
03:51:09 openstackgerrit Arvind proposed openstack/nova-specs master: Support traits in Glance https://review.openstack.org/541507
05:07:00 openstackgerrit Merged openstack/nova master: Make sure that we have usable input for graphical console https://review.openstack.org/538003
05:07:36 openstackgerrit Merged openstack/nova master: Fix wrong link for "Manage Flavors" in CPU topologies doc https://review.openstack.org/541116
05:17:24 openstackgerrit Arvind Nadendla proposed openstack/nova-specs master: Support traits in Glance https://review.openstack.org/541507
05:29:01 openstackgerrit Merged openstack/nova master: Reset the _RC_CACHE between tests https://review.openstack.org/540404
05:29:20 openstackgerrit Merged openstack/nova master: Clarify CONF.scheduler.max_attempts https://review.openstack.org/541031
05:29:30 openstackgerrit Merged openstack/nova master: [placement] Add sending global request ID in put (2) https://review.openstack.org/531266
06:25:36 openstackgerrit Zhenyu Zheng proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware https://review.openstack.org/507486
06:30:13 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/541561
07:38:43 openstackgerrit Andreas Jaeger proposed openstack/os-vif master: zuul: Enable functional tests in gate https://review.openstack.org/530961
07:56:50 bauzas good morning folks
08:01:46 hrw Graphical console stuff got merged! yay and hurray!
08:10:28 openstack Launchpad bug 1741319 in OpenStack nova-compute charm "arm64: Migration pre-check error: CPU doesn't have compatibility." [Medium,Incomplete]
08:10:28 bauzas hrw: you might be interested in https://bugs.launchpad.net/nova/+bug/1741319
08:10:42 bauzas hrw: just let me know if that's a valid bug so I can triage it
08:11:35 hrw bauzas: there is no cpu_model = host-model on aarch64
08:12:07 hrw bauzas: so bug is valid and not solvable
08:12:22 bauzas hrw: that's what I understood
08:12:32 bauzas since we compare models for live migrating
08:14:28 hrw added note
08:15:34 bauzas hrw: so, for the moment, live-migration isn't supported for ARM64, I guess then
08:15:49 hrw bauzas: not quite.
08:15:49 bauzas if so, we should amend the doc
08:16:06 hrw bauzas: you may migrate but between same cpus
08:16:36 hrw will ask our testers to check it (as my setup is all-in-one)
08:17:11 bauzas hrw: the reporter says they have exact CPUs
08:17:15 bauzas exact same*
08:17:33 bauzas hrw: but the point is, nova is checking something that ARM64 libvirt doesn't pass, that's ity
08:17:45 hrw ok
08:18:07 bauzas so that probably needs some kind of discussion about the possible solutions
08:18:25 bauzas we could conditionally don't care of cpu models if arm64
08:18:32 bauzas or we could ask libvirt to provide a single one
08:18:49 bauzas I tend to consider the latter as the best one
08:19:07 hrw the problem is that even kernel does not provide cpu model
08:19:18 bauzas I know
08:19:21 hrw /proc/cpuinfo is useless
08:19:32 bauzas it's all about having the same XML
08:19:37 hrw DMI data is useless too
08:19:38 bauzas hence the interface
08:20:15 hrw bauzas: you can not set 'random' values as cpu model. that's why libvirt does not set that at all
08:22:25 bauzas I see
08:22:57 bauzas hrw: anyway, I leave the bug one
08:22:59 bauzas open*
08:23:00 hrw sure
08:23:13 bauzas that's worth considering the options
08:23:29 bauzas hrw: but we also have a support matrix page
08:23:46 bauzas hrw: and I feel we should explain that nova live-migration isn't supported for aarch64
08:23:56 bauzas 'not supported' doesn't mean that you can't do
08:24:05 bauzas just that you have to workaround that
08:24:31 bauzas hrw: know the support matrix ?
08:24:53 hrw bauzas: heard about it but not checked
08:25:03 bauzas ok, giving you the link
08:25:51 bauzas hrw: https://docs.openstack.org/nova/latest/user/support-matrix.html and https://docs.openstack.org/nova/latest/user/feature-classification.html
08:26:03 hrw thx
08:26:38 hrw looks like we need to fill whole new column ;D
08:28:48 bauzas hrw: you got my idea
08:29:00 openstackgerrit Thomas Goirand proposed openstack/python-novaclient master: Fix crashing console-log https://review.openstack.org/541610
08:35:34 bauzas hrw: volunteer for documenting it ? if so, ping me when your change is up :)

Earlier   Later