Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-03-23
12:59:40 sean-k-mooney but we have not been able to convice the nfv dfg to do that yet
13:00:20 sean-k-mooney https://github.com/redhat-performance/tuned/blob/master/profiles/cpu-partitioning/cpu-partitioning-variables.conf
13:00:50 sean-k-mooney you can use tuned to configure the cgoups for you including isolating the cores form irqs
13:01:18 sean-k-mooney thats what we use by default for non realtime hosts now
13:03:44 gibi I see
13:04:55 sean-k-mooney in the cgroup way you can prevent the kernel schduling vms to the core indepently of balancing the vms on cores
13:05:03 sean-k-mooney isolcpus does both
13:05:28 sean-k-mooney so you can use the tuned/cgrop method with floating vms too not jsut pinned
13:06:22 sean-k-mooney but isolcpu does a few things that you cant quite do form userspace fully or at least with tuned at this point so they still use and support itn for nfv usecases
13:06:30 gibi I see. I think we only support pinned vms by default
13:06:48 sean-k-mooney that both complicates and simplcies your life :)
13:06:53 gibi yeah
13:06:55 gibi soo telco
13:07:01 gibi :)
13:07:04 sean-k-mooney complciates becasue numa and simplfies because only numa
13:07:28 gibi yeah
13:07:44 sean-k-mooney if you dont have to deal with peopel mixing numa and non numa vms its nice
13:08:25 gibi yes we sort of force numa with pinned cpus and huge pages
13:19:59 openstackgerrit Merged openstack/nova master: Dynamically archive FK related records in archive_deleted_rows https://review.opendev.org/c/openstack/nova/+/773834
13:29:58 bauzas folks, if you need to discuss with kashyap, he's off IRC as he has an Internet issue with his ISP
13:31:08 sean-k-mooney i did have one thing that came up last night
13:32:06 sean-k-mooney stars here http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2021-03-22.log.html#t2021-03-22T19:34:19
13:32:49 sean-k-mooney basically qemu change the cpu flags that are set in a specific cpu model and if you dont use version machine types that will break live migration
13:33:29 sean-k-mooney we may or may not be able to work around that in nova via the migration xml
13:34:25 sean-k-mooney but the end effect is that eypc-ibrs when the vm was booted on the source host nolgere results in the same cpu feature enabel if you boot a vm on the same host or a different one with the same libvirt xml
13:34:40 sean-k-mooney wehn you are using unversion machine types
13:34:40 bauzas sean-k-mooney: kashyap dunno when he will be back
13:35:08 sean-k-mooney its not supper urgent at least for me
13:35:58 sean-k-mooney but rouk has already confrim that setting an explit machine type in nova or using kashyap recent patch to allow removing feature flags both cannot fix the issue
13:36:12 sean-k-mooney since neither ake effect on live migration
13:36:53 sean-k-mooney it wont break use downstream since we use versioned machine types but it will break anyone that uses an unversioned machine type after a qemu update
13:37:22 sean-k-mooney that include our downstream customer if they set an unversion machine type in the nova config or in the glance image
14:30:47 lyarwood gibi / stephenfin ; https://review.opendev.org/c/openstack/nova/+/733627 - could either of you look at this sometime this week, wsgi fix when launched by mod_wsgi
14:33:23 gibi lyarwood: ack
14:33:27 lyarwood thanks
14:39:14 stephenfin done
14:43:45 gibi stephenfin won
14:54:49 bauzas gibi: taking the semaphore for bug triaging
14:54:57 gibi bauzas: given
14:55:07 bauzas I see very old open bugs that aren't triaged
14:55:19 bauzas like https://bugs.launchpad.net/nova/+bug/1463631
14:55:19 openstack Launchpad bug 1463631 in grenade "60_nova/resources.sh:106:ping_check_public fails intermittently" [Undecided,Confirmed]
14:55:28 bauzas any reason to leave them in such state ?
15:13:59 bauzas lyarwood: hmm, very interesting corner case https://bugs.launchpad.net/nova/+bug/1920886
15:14:00 openstack Launchpad bug 1920886 in OpenStack Compute (nova) "ImageNotFound error occurs after live migration" [Undecided,New]
15:22:37 gibi bauzas: https://bugs.launchpad.net/nova/+bug/1463631 has a not too old comment from lyarwood that he saw it again
15:22:38 openstack Launchpad bug 1463631 in grenade "60_nova/resources.sh:106:ping_check_public fails intermittently" [Undecided,Confirmed]
15:24:07 lyarwood sorry was flooded with pings downstream
15:25:33 lyarwood my god, a well written bug report
15:25:42 lyarwood I think I might faint
15:35:05 lyarwood gibi / bauzas ; re the grenade bug, yeah I couldn't make any sense of that, we could drop nova if you don't think we can help
15:38:03 gibi lyarwood: I've just run a logstash query http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Couldn't%20ping%20server%5C%22 and in the last 7 days we had 5 hits so this is active
15:39:19 kashyap bauzas: Thanks! My network has been super flaky; just reading the scrollback
15:39:25 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Reject requests to commit intermediary snapshot of an inactive instance https://review.opendev.org/c/openstack/nova/+/781138
15:39:36 kashyap sean-k-mooney: Hi, looking at the chat log
15:40:21 kashyap Assuming I'm still connected here, I'm getting a "We’re having trouble finding that site."
15:45:31 bauzas lyarwood: ack, will put the grenade bug to be invalid for nova
15:46:33 rouk sean-k-mooney: so yeah, updating cpu_map nor machine type worked.
15:46:54 rouk im just building images with qemu 3.1 i guess now as the last ditch fix.
15:58:43 openstackgerrit Ruby Loo proposed openstack/nova master: Allow plus sign in flavor ids https://review.opendev.org/c/openstack/nova/+/782545
16:09:31 openstackgerrit Ruby Loo proposed openstack/nova master: Allow plus sign in flavor ids https://review.opendev.org/c/openstack/nova/+/782545
16:56:13 openstackgerrit Merged openstack/nova master: Initialize global data separately and run_once in WSGI app init https://review.opendev.org/c/openstack/nova/+/733627
18:31:59 rouk sean-k-mooney: nova patch wise, see a simpler way to make it work?
18:34:04 sean-k-mooney the only thing i can think of is
18:34:10 sean-k-mooney to update teh migrate xml
18:34:36 sean-k-mooney to example exactil the feature that are in use on the source node
18:34:42 sean-k-mooney and skip the cpu compare
18:34:47 sean-k-mooney with a workaround flag
18:35:03 sean-k-mooney but i have not looked at writing that
18:36:26 sean-k-mooney i ping kashyap about this a little eairler to see if he had any ideas but you tried most of the ones i tought would be strait forward
18:36:46 rouk that sounds like more work than removing a commit from qemu
18:36:47 sean-k-mooney since they didnt work nothing that is quick an simple
18:36:55 sean-k-mooney ya it is
18:37:26 rouk ill get it cooking then.
19:53:25 melwitt gmann: do you think such a change like https://review.opendev.org/c/openstack/nova/+/782545 represents an api change that needs a new microversion?
19:56:12 gmann melwitt: yeah, it change 400 to 200 which is interop issue. microversion is needed for such changes
19:56:34 melwitt gmann: ack, thanks
20:09:19 gmann melwitt: this is same as allowing more char in keypair name, - https://blueprints.launchpad.net/nova/+spec/allow-special-characters-in-keypair-name
20:19:25 melwitt gmann: a-ha, thanks for the example!
22:46:51 openstackgerrit Merged openstack/nova stable/ussuri: Add config parameter 'live_migration_scheme' to live migration with tls guide https://review.opendev.org/c/openstack/nova/+/782126
#openstack-nova - 2021-03-24
03:40:37 openstackgerrit Rico Lin proposed openstack/nova master: [Arm64] Makesure read bytes from jason files https://review.opendev.org/c/openstack/nova/+/782634
05:38:29 openstackgerrit Josephine Seifert proposed openstack/nova stable/train: Add config parameter 'live_migration_scheme' to live migration with tls guide https://review.opendev.org/c/openstack/nova/+/782430
07:00:45 openstackgerrit Rico Lin proposed openstack/nova master: [Arm64] Makesure read bytes from jason files https://review.opendev.org/c/openstack/nova/+/782634
07:13:07 openstackgerrit Rico Lin proposed openstack/nova master: [Arm64][libvirt] Fix read bytes from jason files https://review.opendev.org/c/openstack/nova/+/782634
07:14:12 ricolin stephenfin, I created a bug for the arm64 bionic issue here https://bugs.launchpad.net/nova/+bug/1921073
07:14:13 openstack Launchpad bug 1921073 in OpenStack Compute (nova) "[arm64][libvirt] firmware metadata files not found for arm64 on ubuntu 18.04" [Undecided,New]
07:15:14 ricolin Also launchpad and patch for focal arm64 libvirt issue I mentioned in https://review.opendev.org/c/openstack/nova/+/781210
07:15:15 ricolin https://review.opendev.org/c/openstack/nova/+/782634
07:19:35 ricolin stephenfin, gibi bauzas kashyap takashin lyarwood ^^^
08:32:53 hrw morning
08:33:09 hrw does anyone uses nova on Debian 'buster'?
08:34:43 hrw I am trying to get it working again in Kolla. so far updated libvirt and libvirt-python to 7.0.0, installed 2020.05 qemu-efi-aarch64 firmware. This gave me nova passing further.
08:38:30 ricolin hrw, do you facing the same issue as I'm on Ubuntu 'bionic'? https://bugs.launchpad.net/nova/+bug/1921073
08:38:32 openstack Launchpad bug 1921073 in OpenStack Compute (nova) "[arm64][libvirt] firmware metadata files not found for arm64 on ubuntu 18.04" [Undecided,New]
08:40:24 hrw ricolin: do yourself a favor. move to 20.04
08:40:42 hrw ricolin: I solved that problem with qemu-efi-aarch64 from buster-backports
08:45:49 ricolin hrw, already moved and it works with 20.04 (https://review.opendev.org/c/openstack/devstack/+/708317)
08:47:44 hrw https://paste.debian.net/1190776/ is where I am now
08:49:23 ricolin hrw, for that case, I think this fix works https://review.opendev.org/c/openstack/nova/+/782634
08:55:11 hrw hm. need to install with nova in developer mode then

Earlier   Later