Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-05-31
17:22:38 sean-k-mooney but wanted to get input form other before i do
17:22:46 sean-k-mooney i.e. is this somethign we want to do
17:22:52 sean-k-mooney before i spend time on it
17:23:40 bauzas hah
17:23:45 bauzas no worries
17:24:02 bauzas I understand your point and we can surely discuss this the other week
17:24:16 bauzas sean-k-mooney: just add your topic in the agenda in order to not forget it
17:25:36 sean-k-mooney sure will do
17:58:36 gibi away
17:58:44 gibi ehh
18:02:41 gibi bauzas: I can take the triage baton next week over a beer :)
18:36:22 opendevreview sean mooney proposed openstack/nova master: update nova to use black for code formating. https://review.opendev.org/c/openstack/nova/+/844120
18:42:43 sean-k-mooney i have run unit and functional test on that locally and they appear to pass so im pretty confident that there has been no effective code change in ^
18:43:25 sean-k-mooney black, pep8 and fast8 also pass and so does pre-commit
18:48:07 gibi sean-k-mooney: thanks
18:48:45 sean-k-mooney lol just realised it updated leet files 1337
18:49:23 sean-k-mooney in my somewhat biased opipion the code is more readable. im also not a black fangirl by any means its jut looks nicer to me
18:58:32 artom Are "Cinder multi-backend feature disabled" a known thing on stable/victoria?
19:45:14 sean-k-mooney am ping me tomorrow but im not sure what that means
19:45:36 sean-k-mooney i assume that is a tempest config option
20:06:02 gmann artom: it is enabled, it is enabled tempest-slow job and in cinder-tempest-lvm-multibackend which pass in victoria
#openstack-nova - 2022-06-01
00:05:12 opendevreview Artom Lifshitz proposed openstack/nova stable/victoria: Add a regression test for bug 1939545 https://review.opendev.org/c/openstack/nova/+/843948
00:05:13 opendevreview Artom Lifshitz proposed openstack/nova stable/victoria: compute: Ensure updates to bdms during pre_live_migration are saved https://review.opendev.org/c/openstack/nova/+/843949
00:19:02 opendevreview melanie witt proposed openstack/nova stable/train: Define new functional test tox env for placement gate to run https://review.opendev.org/c/openstack/nova/+/840777
05:35:57 opendevreview Steve Baker proposed openstack/nova master: Align ironic driver with libvirt secure boot enable https://review.opendev.org/c/openstack/nova/+/844243
08:44:02 opendevreview Rico Lin proposed openstack/nova master: libvirt: Ignore LibvirtConfigObject kwargs https://review.opendev.org/c/openstack/nova/+/830644
08:44:03 opendevreview Rico Lin proposed openstack/nova master: libvirt: Add vIOMMU device to guest https://review.opendev.org/c/openstack/nova/+/830646
08:44:03 opendevreview Rico Lin proposed openstack/nova master: libvirt: Remove unnecessary TODO https://review.opendev.org/c/openstack/nova/+/830645
09:22:55 opendevreview Balazs Gibizer proposed openstack/nova stable/train: Extend the reproducer for 1953359 and 1952915 https://review.opendev.org/c/openstack/nova/+/839354
09:22:56 opendevreview Balazs Gibizer proposed openstack/nova stable/train: [rt] Apply migration context for incoming migrations https://review.opendev.org/c/openstack/nova/+/839355
09:26:46 opendevreview Balazs Gibizer proposed openstack/nova master: Reject AZ changes during aggregate add / remove host https://review.opendev.org/c/openstack/nova/+/821423
09:32:28 opendevreview Balazs Gibizer proposed openstack/osc-placement master: Support microversion 1.39 https://review.opendev.org/c/openstack/osc-placement/+/828545
09:38:27 slaweq gibi hi, since few days we are seeing same failure in our fedora based scenario periodic job, see https://zuul.openstack.org/build/4a7f284f32eb436da6b5ef59d46e615d for example
09:38:37 slaweq it seems like some nova and/or glance issue for me
09:38:56 slaweq did You maybe saw already something like that? or should I open new LP for it?
09:39:55 gibi slaweq: it does not immediately ring a bell but give me some time to look into it... I will get back to you
09:40:09 slaweq @gibi sure, thx a lot
09:47:42 sean-k-mooney slaweq: its proably related to runnign on python 3.10
09:48:02 sean-k-mooney slaweq: we dont have any tempest based testing on 3.10 currently for nova
09:48:14 sean-k-mooney if you are only seing it on fedora
09:50:24 slaweq sean-k-mooney: yes, we are seeing it only on fedora currently
09:51:38 sean-k-mooney do you need to run that job on fedora for a particalar reason by the way? im wondering if ubuntu 22.04 woudl see the same failure
09:54:05 slaweq sean-k-mooney (@sean-k-mooney:matrix.org) we just want to have fedora based periodic job, we run many Ubuntu jobs in check/gate queues already
09:54:25 slaweq maybe we could/should move it to c9s now
09:54:30 slaweq but for now it's fedora
09:57:26 sean-k-mooney slaweq: right i was going to suggest ye move to ubuntu 22.04 or centos 9 stream
09:57:43 sean-k-mooney i strongly dislike haveing fedora based testing in the gate
09:58:00 sean-k-mooney im supportive of c9s testing
09:59:21 sean-k-mooney im looking at the logs currently and waitign for the filters error logs to render
10:02:10 sean-k-mooney actully there are no error level logs in nova
10:02:28 sean-k-mooney so there were no excptions
10:02:36 sean-k-mooney i do see DEBUG neutronclient.v2_0.client [-] Error message: {"NeutronError": {"type": "PortNotFound", "message": "Port 1a6c72ae-3e1a-45d6-a70e-1f9964b70756 could not be found.", "detail": ""}}
10:02:46 sean-k-mooney but that proably not related to the failure
10:03:04 sean-k-mooney im only looking at the novac compute currently
10:04:09 gibi slaweq: I see that the instance being snapshotted is crashed
10:04:11 gibi https://zuul.openstack.org/build/4a7f284f32eb436da6b5ef59d46e615d/log/controller/logs/screen-n-cpu.txt#19235
10:04:26 gibi Instance instance-0000002e disappeared while taking snapshot of it: [Error Code 42] Domain not found: no domain with matching uuid
10:04:44 gibi https://zuul.openstack.org/build/4a7f284f32eb436da6b5ef59d46e615d/log/controller/logs/libvirt/libvirt/qemu/instance-0000002e_log.txt
10:04:50 gibi 2022-06-01 03:13:33.853+0000: shutting down, reason=crashed
10:05:02 gibi but I don't see any reaonse why it is crashed
10:05:09 sean-k-mooney its paused but i have nto gotten to where it crahsed yet
10:06:03 sean-k-mooney Instance instance-0000002e disappeared while taking snapshot of it: [Error Code 42] Domain not found: no domain with matching uuid '84e8668d-6263-4a85-98f5-15d1a7e38f0d' (instance-0000002e)
10:07:01 gibi it seems that the syslog journal was rotated as it only has later logs in it
10:16:18 gibi slaweq: as far as I see this is job failing since 2022-05-07 with the same issue
10:17:45 gibi but older runs has no logs any more
10:18:01 gibi so potentially it is failing a lot longer than 05-07
10:26:09 gibi slaweq: I found the rotated yournal
10:26:13 gibi qemu is segfaulted
10:26:14 gibi https://paste.opendev.org/show/bIxQstqasZeJiuyDRHxD/
10:27:23 gibi better paste https://paste.opendev.org/show/byrrh32nwTYvGXtw52ax/
10:29:17 sean-k-mooney nice find
10:29:25 sean-k-mooney so likely a fedora bug
10:29:34 gibi probably worth taking up with the downstream virt team
10:29:43 gibi as my knowledge ends here
10:30:17 sean-k-mooney ya perhaps although they dont really care about the tcg backend for qemu
10:30:25 sean-k-mooney so if this does not happen with kvm
10:30:32 sean-k-mooney they may or may not help
10:32:20 sean-k-mooney hum there is nothing useful in the qemu instance log
10:33:08 sean-k-mooney it looks like a pretty normal vm
10:33:12 sean-k-mooney -machine pc-i440fx-6.1,accel=tcg,usb=off,dump-guest-core=off,memory-backend=pc.ram
10:33:24 sean-k-mooney so still pc machine type too
10:50:09 opendevreview Merged openstack/nova master: Fix typos in help messages https://review.opendev.org/c/openstack/nova/+/843843
10:50:17 opendevreview Merged openstack/nova master: Fix typos https://review.opendev.org/c/openstack/nova/+/843127
10:50:27 opendevreview Merged openstack/nova stable/train: Define new functional test tox env for placement gate to run https://review.opendev.org/c/openstack/nova/+/840777
11:28:15 opendevreview Balazs Gibizer proposed openstack/osc-placement master: Drop py36 and py37 support https://review.opendev.org/c/openstack/osc-placement/+/844281
11:29:09 opendevreview Balazs Gibizer proposed openstack/osc-placement master: Support microversion 1.39 https://review.opendev.org/c/openstack/osc-placement/+/828545
11:36:40 slaweq thx sean-k-mooney (@sean-k-mooney:matrix.org) and gibi for checking tha
11:36:42 slaweq *that
11:42:17 sean-k-mooney slaweq: assuming this is just the qemu bug then you shoudl be able to swap the nodeset to c9s and that hopefully will resolve the problem unless that but has already made it into c9s
11:46:38 gibi elodilles, melwitt: I see a new blocker on stable/ussuri https://zuul.opendev.org/t/openstack/build/034890df09f94bcba63b97f1586e9a48/log/job-output.txt#43311
11:46:57 gibi added to the tracking etherpad https://etherpad.opendev.org/p/nova-stable-branch-ci L26
11:47:28 gibi gmann: ^^ fyi maybe you have a view on this too
11:49:45 elodilles gibi: there's already a patch for that: https://review.opendev.org/c/openstack/openstacksdk/+/843978
11:51:35 gibi thanks, then now that is a blocker for ussuri as the job fails 100%
11:51:54 gibi due to master upper-constraint was bumped recently
11:52:38 elodilles yepp
11:54:16 gibi elodilles: thanks for the info
12:01:02 opendevreview Balazs Gibizer proposed openstack/nova stable/wallaby: func: Increase rpc_response_timeout in TestMultiCellMigrate tests https://review.opendev.org/c/openstack/nova/+/844200
12:03:40 gibi elodilles, melwitt: I saw this issue ^^ in wallaby (newer branches are already fixed)
12:09:05 opendevreview Balazs Gibizer proposed openstack/osc-placement master: Add Python3 zed unit tests https://review.opendev.org/c/openstack/osc-placement/+/835369

Earlier   Later