| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-03-23 | |||
| 13:32:49 | sean-k-mooney | basically qemu change the cpu flags that are set in a specific cpu model and if you dont use version machine types that will break live migration | |
| 13:33:29 | sean-k-mooney | we may or may not be able to work around that in nova via the migration xml | |
| 13:34:25 | sean-k-mooney | but the end effect is that eypc-ibrs when the vm was booted on the source host nolgere results in the same cpu feature enabel if you boot a vm on the same host or a different one with the same libvirt xml | |
| 13:34:40 | sean-k-mooney | wehn you are using unversion machine types | |
| 13:34:40 | bauzas | sean-k-mooney: kashyap dunno when he will be back | |
| 13:35:08 | sean-k-mooney | its not supper urgent at least for me | |
| 13:35:58 | sean-k-mooney | but rouk has already confrim that setting an explit machine type in nova or using kashyap recent patch to allow removing feature flags both cannot fix the issue | |
| 13:36:12 | sean-k-mooney | since neither ake effect on live migration | |
| 13:36:53 | sean-k-mooney | it wont break use downstream since we use versioned machine types but it will break anyone that uses an unversioned machine type after a qemu update | |
| 13:37:22 | sean-k-mooney | that include our downstream customer if they set an unversion machine type in the nova config or in the glance image | |
| 14:30:47 | lyarwood | gibi / stephenfin ; https://review.opendev.org/c/openstack/nova/+/733627 - could either of you look at this sometime this week, wsgi fix when launched by mod_wsgi | |
| 14:33:23 | gibi | lyarwood: ack | |
| 14:33:27 | lyarwood | thanks | |
| 14:39:14 | stephenfin | done | |
| 14:43:45 | gibi | stephenfin won | |
| 14:54:49 | bauzas | gibi: taking the semaphore for bug triaging | |
| 14:54:57 | gibi | bauzas: given | |
| 14:55:07 | bauzas | I see very old open bugs that aren't triaged | |
| 14:55:19 | bauzas | like https://bugs.launchpad.net/nova/+bug/1463631 | |
| 14:55:19 | openstack | Launchpad bug 1463631 in grenade "60_nova/resources.sh:106:ping_check_public fails intermittently" [Undecided,Confirmed] | |
| 14:55:28 | bauzas | any reason to leave them in such state ? | |
| 15:13:59 | bauzas | lyarwood: hmm, very interesting corner case https://bugs.launchpad.net/nova/+bug/1920886 | |
| 15:14:00 | openstack | Launchpad bug 1920886 in OpenStack Compute (nova) "ImageNotFound error occurs after live migration" [Undecided,New] | |
| 15:22:37 | gibi | bauzas: https://bugs.launchpad.net/nova/+bug/1463631 has a not too old comment from lyarwood that he saw it again | |
| 15:22:38 | openstack | Launchpad bug 1463631 in grenade "60_nova/resources.sh:106:ping_check_public fails intermittently" [Undecided,Confirmed] | |
| 15:24:07 | lyarwood | sorry was flooded with pings downstream | |
| 15:25:33 | lyarwood | my god, a well written bug report | |
| 15:25:42 | lyarwood | I think I might faint | |
| 15:35:05 | lyarwood | gibi / bauzas ; re the grenade bug, yeah I couldn't make any sense of that, we could drop nova if you don't think we can help | |
| 15:38:03 | gibi | lyarwood: I've just run a logstash query http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Couldn't%20ping%20server%5C%22 and in the last 7 days we had 5 hits so this is active | |
| 15:39:19 | kashyap | bauzas: Thanks! My network has been super flaky; just reading the scrollback | |
| 15:39:25 | openstackgerrit | Lee Yarwood proposed openstack/nova master: compute: Reject requests to commit intermediary snapshot of an inactive instance https://review.opendev.org/c/openstack/nova/+/781138 | |
| 15:39:36 | kashyap | sean-k-mooney: Hi, looking at the chat log | |
| 15:40:21 | kashyap | Assuming I'm still connected here, I'm getting a "We’re having trouble finding that site." | |
| 15:45:31 | bauzas | lyarwood: ack, will put the grenade bug to be invalid for nova | |
| 15:46:33 | rouk | sean-k-mooney: so yeah, updating cpu_map nor machine type worked. | |
| 15:46:54 | rouk | im just building images with qemu 3.1 i guess now as the last ditch fix. | |
| 15:58:43 | openstackgerrit | Ruby Loo proposed openstack/nova master: Allow plus sign in flavor ids https://review.opendev.org/c/openstack/nova/+/782545 | |
| 16:09:31 | openstackgerrit | Ruby Loo proposed openstack/nova master: Allow plus sign in flavor ids https://review.opendev.org/c/openstack/nova/+/782545 | |
| 16:56:13 | openstackgerrit | Merged openstack/nova master: Initialize global data separately and run_once in WSGI app init https://review.opendev.org/c/openstack/nova/+/733627 | |
| 18:31:59 | rouk | sean-k-mooney: nova patch wise, see a simpler way to make it work? | |
| 18:34:04 | sean-k-mooney | the only thing i can think of is | |
| 18:34:10 | sean-k-mooney | to update teh migrate xml | |
| 18:34:36 | sean-k-mooney | to example exactil the feature that are in use on the source node | |
| 18:34:42 | sean-k-mooney | and skip the cpu compare | |
| 18:34:47 | sean-k-mooney | with a workaround flag | |
| 18:35:03 | sean-k-mooney | but i have not looked at writing that | |
| 18:36:26 | sean-k-mooney | i ping kashyap about this a little eairler to see if he had any ideas but you tried most of the ones i tought would be strait forward | |
| 18:36:46 | rouk | that sounds like more work than removing a commit from qemu | |
| 18:36:47 | sean-k-mooney | since they didnt work nothing that is quick an simple | |
| 18:36:55 | sean-k-mooney | ya it is | |
| 18:37:26 | rouk | ill get it cooking then. | |
| 19:53:25 | melwitt | gmann: do you think such a change like https://review.opendev.org/c/openstack/nova/+/782545 represents an api change that needs a new microversion? | |
| 19:56:12 | gmann | melwitt: yeah, it change 400 to 200 which is interop issue. microversion is needed for such changes | |
| 19:56:34 | melwitt | gmann: ack, thanks | |
| 20:09:19 | gmann | melwitt: this is same as allowing more char in keypair name, - https://blueprints.launchpad.net/nova/+spec/allow-special-characters-in-keypair-name | |
| 20:19:25 | melwitt | gmann: a-ha, thanks for the example! | |
| 22:46:51 | openstackgerrit | Merged openstack/nova stable/ussuri: Add config parameter 'live_migration_scheme' to live migration with tls guide https://review.opendev.org/c/openstack/nova/+/782126 | |
| #openstack-nova - 2021-03-24 | |||
| 03:40:37 | openstackgerrit | Rico Lin proposed openstack/nova master: [Arm64] Makesure read bytes from jason files https://review.opendev.org/c/openstack/nova/+/782634 | |
| 05:38:29 | openstackgerrit | Josephine Seifert proposed openstack/nova stable/train: Add config parameter 'live_migration_scheme' to live migration with tls guide https://review.opendev.org/c/openstack/nova/+/782430 | |
| 07:00:45 | openstackgerrit | Rico Lin proposed openstack/nova master: [Arm64] Makesure read bytes from jason files https://review.opendev.org/c/openstack/nova/+/782634 | |
| 07:13:07 | openstackgerrit | Rico Lin proposed openstack/nova master: [Arm64][libvirt] Fix read bytes from jason files https://review.opendev.org/c/openstack/nova/+/782634 | |
| 07:14:12 | ricolin | stephenfin, I created a bug for the arm64 bionic issue here https://bugs.launchpad.net/nova/+bug/1921073 | |
| 07:14:13 | openstack | Launchpad bug 1921073 in OpenStack Compute (nova) "[arm64][libvirt] firmware metadata files not found for arm64 on ubuntu 18.04" [Undecided,New] | |
| 07:15:14 | ricolin | Also launchpad and patch for focal arm64 libvirt issue I mentioned in https://review.opendev.org/c/openstack/nova/+/781210 | |
| 07:15:15 | ricolin | https://review.opendev.org/c/openstack/nova/+/782634 | |
| 07:19:35 | ricolin | stephenfin, gibi bauzas kashyap takashin lyarwood ^^^ | |
| 08:32:53 | hrw | morning | |
| 08:33:09 | hrw | does anyone uses nova on Debian 'buster'? | |
| 08:34:43 | hrw | I am trying to get it working again in Kolla. so far updated libvirt and libvirt-python to 7.0.0, installed 2020.05 qemu-efi-aarch64 firmware. This gave me nova passing further. | |
| 08:38:30 | ricolin | hrw, do you facing the same issue as I'm on Ubuntu 'bionic'? https://bugs.launchpad.net/nova/+bug/1921073 | |
| 08:38:32 | openstack | Launchpad bug 1921073 in OpenStack Compute (nova) "[arm64][libvirt] firmware metadata files not found for arm64 on ubuntu 18.04" [Undecided,New] | |
| 08:40:24 | hrw | ricolin: do yourself a favor. move to 20.04 | |
| 08:40:42 | hrw | ricolin: I solved that problem with qemu-efi-aarch64 from buster-backports | |
| 08:45:49 | ricolin | hrw, already moved and it works with 20.04 (https://review.opendev.org/c/openstack/devstack/+/708317) | |
| 08:47:44 | hrw | https://paste.debian.net/1190776/ is where I am now | |
| 08:49:23 | ricolin | hrw, for that case, I think this fix works https://review.opendev.org/c/openstack/nova/+/782634 | |
| 08:55:11 | hrw | hm. need to install with nova in developer mode then | |
| 08:57:41 | kashyap | sean-k-mooney: My internet connection last evening was super flaky here (they were, and still are, reworking roads here). | |
| 08:58:12 | hrw | ricolin: thanks. | |
| 08:58:20 | hrw | redeploying now | |
| 08:58:21 | kashyap | ricolin: Is this the one that was breaking the AArch64 CI? | |
| 08:59:11 | ricolin | hrw, I'm now looking at issue with delete volume backups in https://zuul.opendev.org/t/openstack/build/22b514f0bb2041d9b744cc0b37df8bf9 | |
| 08:59:25 | ricolin | kashyap, yes | |
| 09:00:34 | hrw | ricolin: step by step. I first want to get something running. | |
| 09:01:02 | hrw | already have 3 patches in kolla queue to get to current state ;D | |
| 09:01:41 | kashyap | ricolin: The "rb" --> read and binary, yeah? | |
| 09:02:04 | hrw | mgoddard: yes | |
| 09:02:08 | hrw | kashyap: yes | |
| 09:02:36 | ricolin | hrw, will check your kolla patches, maybe I'm facing same issues you ware faced! | |
| 09:02:47 | ricolin | kashyap, yes as hrw said:) | |
| 09:03:19 | mgoddard | hrw: me? | |
| 09:03:21 | ricolin | read and make all output from file in binary mode | |
| 09:03:28 | hrw | mgoddard: sorry | |
| 09:03:40 | kashyap | hrw: ricolin: Nice debugging. It's the Python-3 only thing, I guess? | |
| 09:04:37 | hrw | kashyap: I would say that it is old open() behaviour but maybe I am too long in py3 | |
| 09:05:12 | hrw | anyway it is probably usual bytes != chars thing | |
| 09:26:51 | bauzas | stephenfin: dansmith: lyarwood: I'm a bit afraid of loosing the RPC API bump merge to be in Xena, could you please look at this one ? | |
| 09:27:03 | bauzas | context : https://review.opendev.org/c/openstack/nova/+/761452 | |
| 09:37:09 | hrw | ricolin: https://review.opendev.org/c/openstack/nova/+/782634 got me closer ;D | |