Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-03-22
14:48:12 artom Anyone able to run functional tests on ussuri?
14:48:28 artom Trying to figure out if it's something local to me, or more widespread
14:48:48 artom Seems to be hanging/timing out on:
14:48:49 artom functional installdeps: -chttps://releases.openstack.org/constraints/upper/ussuri, -r/home/artom/src/nova/requirements.txt, -r/home/artom/src/nova/test-requirements.txt, openstack-placement>=1.0.0
14:50:14 sean-k-mooney i can try it one sec
14:50:17 artom Seems to be doing... something?
14:50:19 artom Looping on https://paste.opendev.org/show/b45jbgPA429f5iKFJSEq/
14:52:44 sean-k-mooney looks like we are missing a fixture
14:52:48 sean-k-mooney from that trace
14:52:58 sean-k-mooney we shoudl not eb doing ioctl;calls in general
14:53:47 sean-k-mooney like that implies we are doing file io or network configuration
14:56:40 sean-k-mooney its running fine for me
14:56:46 sean-k-mooney were you having a failing test?
14:56:50 sean-k-mooney or just would not install
14:57:48 sean-k-mooney i did locally change the psycopg2 to psycopg2-binary in my test-requirements.txt but that is just because i dont have or want postgress installed on my laptop
14:58:09 sean-k-mooney so i dont have the headers to build psycopg2 form source
15:01:04 artom So there is a backport in progress
15:01:12 artom *repo
15:01:30 artom But... it's not running any tests (yet), it's on installdeps...
15:02:26 sean-k-mooney got the gerrit linke i can try that explictly if you want
15:03:54 artom sean-k-mooney, only local for now
15:04:11 artom Backporting https://review.opendev.org/c/openstack/nova/+/796907/2/nova/tests/functional/libvirt/test_pci_sriov_servers.py#73 to ussuri
15:08:07 sean-k-mooney i had one failure
15:08:18 sean-k-mooney FileNotFoundError: [Errno 2] No such file or directory: 'openssl'
15:08:35 sean-k-mooney which is likely just down to the fact im runing this on nixos
15:09:57 artom Seems to be the same problem with a pristine ussuri...
15:10:10 artom I should try on Ubuntu I guess?
15:10:22 artom Although func tests should be platform-independant
15:15:04 sean-k-mooney running them on macos last night not as much as you woudl think
15:15:16 sean-k-mooney we have a bunch that fail because they detach its not linux
15:15:55 sean-k-mooney maybe pass -r
15:16:01 sean-k-mooney or delete the .tox dir
15:16:17 sean-k-mooney incase you have some leftover issue form a previous run
15:17:02 artom Yep, tried with -r, same
15:17:32 sean-k-mooney odd what distro are you currently using
15:17:44 sean-k-mooney i can try on ubunu if you like i also have a centos 9 vm
15:18:18 artom F35
15:19:21 bauzas reminder : nova meeting in 41 mins here at #openstack-nova
15:19:53 bauzas fwiw, DST is not impacting our meeting, as we use UTC
15:20:10 clarkb artom: sean-k-mooney: pip installs taking forever likely indicates a dependency resolver problem
15:20:35 clarkb we've seen that happen when the solver can't find a valid answer. However constraints tends to fix that and you supply constraints so maybe not that
15:20:39 sean-k-mooney clarkb: i dont think it was the resolver
15:20:48 sean-k-mooney clarkb: i think artom is gettign stack traces
15:21:01 artom sean-k-mooney, no, just spinning in the void
15:21:07 sean-k-mooney oh have you added -v
15:21:09 artom The paste was a `strace -p` output
15:21:12 sean-k-mooney so you can see what actully happening
15:23:28 artom sean-k-mooney, *facepalm* lemme try that
15:35:47 sean-k-mooney artom: f35 has a much newer gcc libffi and kernel then ussuri was developed with by the way so that strace was refering to ffi presumable as part of compiling some of the c python modules so there might be issues with tyrign to install ussuri on f35 to run the func tests
15:42:24 zigo I just noticed that if a host is over its CPU ratio (because it has been reduced), then live-migrations are silently failing (only the scheduler gives a clue). Is this known? Is this considered a bug? Should I file the bug?
15:43:02 zigo The workaround is obviously to temporary up the CPU overcommit ratio temporarily, but that's still kind of annoying to do.
15:43:19 sean-k-mooney zigo: yes its a know issue
15:43:38 sean-k-mooney it has to do with how placement currently validates allocation candiates
15:43:50 sean-k-mooney if its the issue i think it is
15:45:37 zigo Thanks.
15:48:13 sean-k-mooney if i rememebr correctly it also effect evacuate
15:51:05 bauzas last reminder : nova meeting in 9 mins
16:00:16 bauzas #startmeeting nova
16:00:17 opendevmeet The meeting name has been set to 'nova'
16:00:17 opendevmeet Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:17 opendevmeet Meeting started Tue Mar 22 16:00:16 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:21 bauzas hey ho
16:00:24 elodilles o/
16:00:26 chateaulav \o
16:00:33 bauzas #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
16:00:45 gmann o/
16:00:50 dansmith o/
16:01:24 artom ~o~
16:01:30 bauzas ok, let's start
16:01:35 bauzas #topic Bugs (stuck/critical)
16:01:40 bauzas #info No Critical bug
16:01:44 bauzas #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 28 new untriaged bugs (+0 since the last meeting)
16:01:48 bauzas #help Nova bug triage help is appreciated https://wiki.openstack.org/wiki/Nova/BugTriage
16:01:53 bauzas #link https://storyboard.openstack.org/#!/project/openstack/placement 26 open stories (0 since the last meeting) in Storyboard for Placement
16:02:02 bauzas any bug in particular to discuss ?
16:02:17 bauzas I triaged a few of them but I need to create some env for verifying some others
16:03:07 bauzas ok, looks not
16:03:10 bauzas next,
16:03:15 bauzas #topic Gate status
16:03:18 bauzas #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:03:23 bauzas #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status
16:03:26 bauzas #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:03:33 bauzas I haven't seen any new problem
16:04:14 gmann one update for centos9 stream volume detach failure
16:04:35 gmann it is fixed now as SSH-able series is merged #link https://review.opendev.org/q/(topic:bug/1960346+OR+topic:wait_until_sshable_pingable)+status:merged
16:04:48 gmann I have made centos9-stream as voting job in tempest gate
16:04:53 bauzas \o/
16:05:05 dansmith gmann: really, that makes it all pass reliably?
16:05:16 gmann and proposed to be voting in devstack side too #link https://review.opendev.org/c/openstack/devstack/+/834546
16:05:22 gmann dansmith: for now yes:)
16:05:26 dansmith cool
16:05:49 dansmith fips job in glance was still failing this morning I think, but I will look and see if it ran against that or not
16:05:52 gmann and we will monitor it carefully now as we made it voting. n-v jobs always gets ignored somehow
16:05:57 dansmith yeah cool
16:06:13 artom So I wonder, would there be anything else to understand at the guest:host interaction level to understand why Ubuntu doesn't need to wait for SSHABLE?
16:06:43 dansmith artom: I'm super curious as well, as this seems like an odd thing to have changed with just newer libvirt/qemu, although certainly possible
16:06:54 dansmith we'll see if more weirdness comes out of running it in the full firehose
16:07:03 gmann dansmith: yeah, you can try with recheck. this patch fixed the last test #link https://review.opendev.org/c/openstack/tempest/+/831608
16:07:04 bauzas agreed, it's weird but ok
16:07:07 dansmith as I was seeing other problems (on stream 8 mind you) when we were running it voting

Earlier   Later