Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-04-17
16:07:09 opendevreview Stephen Finucane proposed openstack/nova master: db: Remove unnecessary 'insert()' argument https://review.opendev.org/c/openstack/nova/+/880664
16:07:09 opendevreview Stephen Finucane proposed openstack/nova master: db: Don't rely on branched connections https://review.opendev.org/c/openstack/nova/+/880663
16:07:10 opendevreview Stephen Finucane proposed openstack/nova master: fixup! db: Remove unnecessary 'insert()' argument https://review.opendev.org/c/openstack/nova/+/880666
16:07:10 opendevreview Stephen Finucane proposed openstack/nova master: tests: Pass parameters to sqlalchemy.text() as bindparams https://review.opendev.org/c/openstack/nova/+/880665
16:07:11 opendevreview Stephen Finucane proposed openstack/nova master: db: Avoid relying on autobegin https://review.opendev.org/c/openstack/nova/+/880668
16:07:11 opendevreview Stephen Finucane proposed openstack/nova master: tests: Add missing args to sqlalchemy.Table https://review.opendev.org/c/openstack/nova/+/880667
16:07:12 opendevreview Stephen Finucane proposed openstack/nova master: tests: Remove test for DB special characters https://review.opendev.org/c/openstack/nova/+/880669
16:08:31 opendevreview Stephen Finucane proposed openstack/nova master: tests: Pass parameters to sqlalchemy.text() as bindparams https://review.opendev.org/c/openstack/nova/+/880665
16:08:31 opendevreview Stephen Finucane proposed openstack/nova master: db: Remove unnecessary 'insert()' argument https://review.opendev.org/c/openstack/nova/+/880664
16:08:32 opendevreview Stephen Finucane proposed openstack/nova master: db: Avoid relying on autobegin https://review.opendev.org/c/openstack/nova/+/880668
16:08:32 opendevreview Stephen Finucane proposed openstack/nova master: tests: Add missing args to sqlalchemy.Table https://review.opendev.org/c/openstack/nova/+/880667
16:08:33 opendevreview Stephen Finucane proposed openstack/nova master: tests: Remove test for DB special characters https://review.opendev.org/c/openstack/nova/+/880669
16:41:10 opendevreview Merged openstack/placement master: Update python testing as per zed cycle testing runtime https://review.opendev.org/c/openstack/placement/+/841690
17:11:53 opendevreview Dan Smith proposed openstack/nova master: Stop ignoring missing compute nodes in claims https://review.opendev.org/c/openstack/nova/+/880633
17:11:53 opendevreview Dan Smith proposed openstack/nova master: Remove silent failure to find a node on rebuild https://review.opendev.org/c/openstack/nova/+/880632
19:30:10 dansmith gibi: sean-k-mooney: bauzas: re our conversation this morning, I've seen this on plenty of volume tests lately as well: https://04e1a81b9004b13f1732-58d59663991bfd1d0dc3ba3167f83a98.ssl.cf1.rackcdn.com/880632/2/check/nova-ceph-multistore/f6ebaaf/testr_results.html
19:30:23 dansmith i.e. several kernel crashes and, obviously, a failure to detach later
19:31:00 dansmith it's this kind of thing that makes me highly suspect of cirros
22:15:56 sean-k-mooney dansmith: if its the old cirror image form the 5.2 release
22:16:07 sean-k-mooney actully thts a differnt crash then i have seen in the past
#openstack-nova - 2023-04-18
04:39:02 frickler 5,15 kernel should be new cirros. that failure very much looks like memory corruption to me
06:38:18 opendevreview Merged openstack/placement master: Bugtracker link update https://review.opendev.org/c/openstack/placement/+/876768
07:50:07 gibi dansmith: I agree that kernel crash points to cirros. Obviously if we see such crashes whathever happens after it (ie detach failure) is probably a consequence of the crash.
07:50:23 gibi dansmith: in the original detach failure case there was no kernel crash, so this is definitely different
08:01:56 kashyap gibi: On that device-detach patch from the QEMU upstream, do we have a way to test it upstream? -- https://www.mail-archive.com/qemu-devel@nongnu.org/msg952944.html
08:02:25 kashyap gibi: I can create an RPM build of QEMU with that patch for Fedora ... if we there's a Fedora job that can test it
08:02:46 kashyap But probably a Ubuntu package is preferred, I guess
08:11:46 gibi I don't think we have a Fedora based job upstream :/
08:12:51 gibi kashyap: but if you can build the fix to Fedora then you can at least try my libvirt only reproduction to see if that is fixed
08:13:48 kashyap gibi: Oh, right indeed. I'll give it a go this week and update you on the bug
08:13:57 gibi cool,thanks
10:40:03 sean-k-mooney gibi: bauzas i now want to add a new hw_vif_model specifically igb
10:40:05 sean-k-mooney https://www.qemu.org/docs/master/system/devices/igb.html
10:40:38 sean-k-mooney qemu added sriov virtualsiation support to it last year
10:41:05 gibi wut? that feels super useful for testing if it does what I think it does
10:41:07 sean-k-mooney unfortunetly it will take a while before our cloud providers have this
10:41:18 sean-k-mooney gibi: yep thats why they added it
10:41:33 sean-k-mooney it would allow us to take a neutron port and then create VFs from it
10:41:43 gibi then I'm +100
10:42:18 sean-k-mooney support was added in libvirt 9.3.0
10:42:37 sean-k-mooney that may be qemu 9.3.0 libvirts docs kind of suck
10:42:46 sean-k-mooney at makeing it cleare if they mean the qemu or libvirt version
10:44:42 sean-k-mooney ill finish the filter stuff firts and you know actully try and do it locally before proposing anything but if it work ill bring this up in a team meeting
10:44:51 sean-k-mooney just not today
10:45:42 sean-k-mooney on a side note this is the alpine image i created https://fileshare.seanmooney.info/
10:46:18 sean-k-mooney im going to update the nova job patch to skip trying to build it and just pull that and see if it works after i finish the docs chagnes on teh schduler patch
11:16:21 zigo How do I migrate away a VM with status SUSPENDED ?!?
11:26:54 sean-k-mooney zigo: i suspended means suspended to disk. i think a cold migration might work but i would have to check the api
11:28:51 sean-k-mooney live migration wont because its not actully running
11:29:11 sean-k-mooney looking at the micate api it does not say but resize says You can only resize a server when its status is ACTIVE or SHUTOFF.
11:29:33 sean-k-mooney zigo: so my guess is without resuming the instnace its not possible
11:45:07 sean-k-mooney bauzas: you have reviewd this in the past can you take alook at this backprot again https://review.opendev.org/c/openstack/nova/+/790447
11:59:10 zigo sean-k-mooney: Thanks.
12:37:35 bauzas sean-k-mooney: me clicks
13:32:37 dansmith gibi: sean-k-mooney: yep, it could be totally different problems. However, we're at the point of suspecting the guest is not playing ball, and sometimes we just see no detach and other times a kernel crash
13:32:51 dansmith sometimes I see only one trace in the klog when that happens, other times several like this
13:33:20 dansmith so I'm just saying that I think expecting cirros to behave like a real guest is already suspect (in my mind)
13:44:48 gibi yeah, I agree
13:51:26 sean-k-mooney so i have the alpine image hosted at https://fileshare.seanmooney.info/alpine.qcow2
13:51:42 sean-k-mooney ill see if i can just sub that in later today into a job and see what breaks
13:54:01 dansmith most of the guest crashes were in kmalloc things,
13:55:28 dansmith which makes me wonder if there's more interaction with the host there
13:58:02 dansmith actually they all go through load_module() in the kernel and are failing there
14:13:32 bauzas dansmith: sorry I didn't had time to look at how many failures were having this kind of crash
15:45:19 bauzas reminder : nova meeting in 15 mins
16:00:26 dansmith bauzas: reminder, nova meeting 24 seconds ago
16:00:33 opendevmeet The meeting name has been set to 'nova'
16:00:33 opendevmeet Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:33 opendevmeet Meeting started Tue Apr 18 16:00:33 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:33 bauzas #startmeeting nova
16:00:40 bauzas dansmith: you beated me on the clock
16:00:51 bauzas walcome falks
16:01:03 elodilles :D
16:01:04 elodilles o/
16:01:05 bauzas #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
16:01:06 auniyal o/
16:01:40 bauzas let's try to do a quick one
16:01:56 bauzas so let's start
16:02:00 bauzas #topic Bugs (stuck/critical)
16:02:04 bauzas #info No Critical bug
16:02:28 bauzas (welp, somehow)
16:02:32 gibi o/
16:02:34 bauzas #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 13 new untriaged bugs (-5 since the last meeting)
16:02:41 bauzas kudos to gibi
16:02:52 gibi triaged couple of one https://etherpad.opendev.org/p/nova-bug-triage-20230411
16:02:52 bauzas gibi: any bug you wanna raise today ?
16:02:58 bauzas thanks
16:03:02 gibi nothing to raise today
16:04:01 bauzas very quickly looked at them, thanks for fishing them
16:04:11 bauzas #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster
16:04:23 bauzas so, next in the roster is melwitt
16:04:31 bauzas not sure she's around
16:04:43 bauzas I'll ask her later if she can help
16:04:49 bauzas if not, I'll take the baton
16:05:00 bauzas #info bug baton is being passed to melwitt/bauzas
16:05:16 bauzas #topic Gate status
16:05:21 bauzas #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:05:26 bauzas #link https://etherpad.opendev.org/p/nova-ci-failures
16:05:37 bauzas do people want to discuss about the recent failures we got ?
16:05:47 bauzas I didn't had time to look at logsearch yet

Earlier   Later