| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-01-17 | |||
| 19:07:24 | sean-k-mooney | we shoud not be using os-testr anywhere anymore | |
| 19:07:35 | sean-k-mooney | everthing should be using stestr | |
| 19:07:41 | gmann | that is installed successfully in gate I think that is using constraint? | |
| 19:07:59 | gmann | https://zuul.opendev.org/t/openstack/build/0fc9dc8ecbe748498c941c6f21cbf057/log/job-output.txt#4367 | |
| 19:08:23 | dansmith | yeah I dunno why I can't install it | |
| 19:08:31 | sean-k-mooney | its in uc ya https://opendev.org/openstack/requirements/src/branch/master/openstack_requirements/tests/files/upper-constraints.txt#L359 | |
| 19:08:37 | dansmith | unless it's a mirror sync thing? | |
| 19:08:57 | sean-k-mooney | https://opendev.org/openstack/devstack/src/branch/master/stack.sh#L803 | |
| 19:09:04 | sean-k-mooney | comment ou tthat line | |
| 19:09:26 | gmann | not this week but last week i stack successfully | |
| 19:09:53 | clarkb | note contraints don't pick what is installed. Only what version to install if something is to be installed | |
| 19:09:56 | dansmith | sean-k-mooney: heh, yeah, I could but.. | |
| 19:10:27 | sean-k-mooney | clarkb: yep i know just pointing out that it would be constrained if it was installed for other projects | |
| 19:10:39 | dansmith | I purged pbr and os-testr locally, which might have gotten past it | |
| 19:10:59 | sean-k-mooney | pip_install -U os-testr | |
| 19:11:04 | sean-k-mooney | so that is unconstiaed in devstack | |
| 19:11:18 | sean-k-mooney | unless pip_install in devstack add uc by default | |
| 19:12:23 | gmann | https://zuul.opendev.org/t/openstack/build/0fc9dc8ecbe748498c941c6f21cbf057/log/job-output.txt#4367 | |
| 19:12:37 | dansmith | sean-k-mooney: it does | |
| 19:12:47 | dansmith | I could install it myself without uc | |
| 19:12:58 | dansmith | but just purging those two packages locally seems to have worked | |
| 19:13:06 | dansmith | so maybe some not-so-correct version deps | |
| 19:13:28 | sean-k-mooney | what os are you using out of interest | |
| 19:13:29 | gmann | maybe | |
| 19:13:51 | dansmith | sean-k-mooney: focal | |
| 19:13:55 | sean-k-mooney | you said you had not stacked this year yet is it 20.04 | |
| 19:14:08 | sean-k-mooney | ok that should also work | |
| 19:14:21 | sean-k-mooney | well for the next 4-6 weeks | |
| 19:14:54 | gmann | focal should all work for latest master/constraints | |
| 19:15:03 | sean-k-mooney | yep | |
| 19:15:20 | sean-k-mooney | i just ment that after RC1 and we reopen master for bobcat | |
| 19:15:30 | sean-k-mooney | then it wont be in the testing runtime | |
| 19:15:35 | sean-k-mooney | nor wil python 3.8 | |
| 19:15:45 | sean-k-mooney | so it will proably be fine but it could start breaking | |
| 20:02:57 | opendevreview | Sofia Enriquez proposed openstack/nova master: Check NFS protocol https://review.opendev.org/c/openstack/nova/+/854030 | |
| 20:05:22 | opendevreview | Dan Smith proposed openstack/nova master: WIP: Detect host renames and abort startup https://review.opendev.org/c/openstack/nova/+/863920 | |
| #openstack-nova - 2023-01-18 | |||
| 10:01:01 | ierdem | Hi folks, is there any way to boot a signed image from volume? I am testing image validation, I can create VM by using signed images on ephemeral disks but when I try boot from volume, it throws an excepiton (https://paste.openstack.org/show/blZen5ID7OIbi47TN8ib/). I am currently working on OpenStack Ussuri, and image backend is Ceph | |
| 10:10:11 | bauzas | gibi: so, there are many ways tricking stestr scheduling | |
| 10:10:43 | bauzas | gibi: but given we directly call tempest which eventually calls stestr, the quickiest way to change the test scheduling is about renaming the testname | |
| 10:11:09 | bauzas | https://stestr.readthedocs.io/en/latest/MANUAL.html#test-scheduling | |
| 10:11:26 | bauzas | "By default stestr schedules the tests by first checking if there is any historical timing data on any tests. It then sorts the tests by that timing data loops over the tests in order and adds one to each worker that it will launch. For tests without timing data, the same is done, except the tests are in alphabetical order instead of based on timing data. If a group regex is used the same algorithm is used with groups instead of | |
| 10:11:26 | bauzas | individual tests." | |
| 10:20:12 | bauzas | gibi: https://review.opendev.org/c/openstack/tempest/+/870913 | |
| 10:20:44 | bauzas | gibi: could I modify https://review.opendev.org/c/openstack/nova/+/869900/ to be Depending on ^ ? | |
| 10:24:25 | gibi | lets keep https://review.opendev.org/c/openstack/nova/+/869900/ alone. as I want to land that regardless of our troubleshooting here. As I think some of the other problems might go away after we remove the excessive logging by that. | |
| 10:25:32 | gibi | about https://review.opendev.org/c/openstack/tempest/+/870913 why we are trying to move the test to the front? I thought we wanted to moved it later or even disable it temporarily to see if other tests are triggering the same OOM behavior and hence trying to establish a pattern causing the OOM | |
| 10:32:25 | bauzas | gibi: OK, then I'll add a DNM on nova, np | |
| 10:32:45 | bauzas | gibi: good question, I was wanting to see whether it was due to this test or not | |
| 10:33:03 | bauzas | if we call it first, and if this is due to this test, it would be killed earlier, right? | |
| 10:39:53 | opendevreview | Sylvain Bauza proposed openstack/nova master: DNM: Testing the killed test https://review.opendev.org/c/openstack/nova/+/870924 | |
| 10:44:42 | kashyap | gibi: Can you have a quick look at this workaround patch when you can (for a change all CI have passed): https://review.opendev.org/c/openstack/nova/+/870794 | |
| 10:44:47 | kashyap | (When you get a minute, that is) | |
| 10:48:15 | tobias-urdin | gibi: any possibility that we can backport this https://review.opendev.org/c/openstack/nova/+/838976 and parent reproducer patch? we are currently patching that in production as we're on newer libvirt with older nova release (xena right now, probably be yoga later this year) | |
| 11:06:50 | ierdem | Hi everyone, is there any way to boot a signed image from volume? I am testing image validation, I can create VM by using signed images on ephemeral disks but when I try boot from volume, it throws an excepiton (https://paste.openstack.org/show/blZen5ID7OIbi47TN8ib/). I am currently working on OpenStack Ussuri, and image backend is Ceph | |
| 11:15:49 | gokhanisi | hello folks, after rebooting my compute host, I can't attach my cinder volumes to instances. Nova throws "unable to lock /var/lib/nova/mnt/dgf/volume-xx for metadata change: No locks available" Full logs are in https://paste.openstack.org/show/beicZ71J17WeNwLjghKc/ What can be reason of this problem ? I am on victoria. On this compute node I am using also gpu passthrough. | |
| 11:23:15 | opendevreview | Sylvain Bauza proposed openstack/nova master: DNM: Testing the killed test https://review.opendev.org/c/openstack/nova/+/870924 | |
| 11:24:03 | gibi | tobias-urdin: regarding https://review.opendev.org/c/openstack/nova/+/838976 I think this is technically backportable but bauzas should know more about it as it is vgpu related | |
| 11:24:46 | bauzas | gibi: tobias-urdin: I already proposed the backports down to wallaby | |
| 11:25:40 | sean-k-mooney | tobias-urdin: there is a backport already | |
| 11:25:54 | sean-k-mooney | tobias-urdin: https://review.opendev.org/c/openstack/nova/+/866156 is the xena cherry pick | |
| 11:26:49 | sean-k-mooney | tobias-urdin: we needed it for wallaby for our downstream product so all the patches are up for review but we have already merged them downstream at the end of the year | |
| 11:35:48 | gibi | ahh I missed the backports as the topic was not set on them | |
| 11:39:10 | tobias-urdin | oh great, thanks! | |
| 11:39:55 | gibi | bauzas: regardin OOM I can do a parallel experiement moving the test to the latest to see if others before it trigger the OOM or not | |
| 11:40:04 | gibi | s/latest/last/ | |
| 11:40:06 | bauzas | gibi: sure, do it | |
| 11:40:10 | gibi | ack | |
| 11:40:39 | bauzas | gibi: I'm starting to look at the functest races | |
| 11:40:46 | bauzas | but I'm hungry | |
| 11:41:01 | gibi | ack. I start to get hungry too | |
| 11:41:03 | gibi | damn biology | |
| 11:47:49 | gibi | bauzas: this was the bug https://bugs.launchpad.net/nova/+bug/1946339 I referred to yesterday related to the funct test failures. It might or might not be related :/ | |
| 12:38:07 | opendevreview | melanie witt proposed openstack/nova master: imagebackend: Add support to libvirt_info for LUKS based encryption https://review.opendev.org/c/openstack/nova/+/826755 | |
| 12:38:08 | opendevreview | melanie witt proposed openstack/nova master: imagebackend: Cache the key manager when disk is encrypted https://review.opendev.org/c/openstack/nova/+/826756 | |
| 12:38:08 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | |
| 12:38:09 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Configure and teardown ephemeral encryption secrets https://review.opendev.org/c/openstack/nova/+/870931 | |
| 12:38:09 | opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | |
| 12:38:10 | opendevreview | melanie witt proposed openstack/nova master: Support resize with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/870933 | |
| 12:38:10 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to convert_image https://review.opendev.org/c/openstack/nova/+/870934 | |
| 12:38:11 | opendevreview | melanie witt proposed openstack/nova master: Add hw_ephemeral_encryption_secret_uuid image property https://review.opendev.org/c/openstack/nova/+/870935 | |
| 12:38:11 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936 | |
| 12:38:13 | opendevreview | melanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/870937 | |
| 12:38:13 | opendevreview | melanie witt proposed openstack/nova master: Add reset_encryption_fields() and save_all() to BlockDeviceMappingList https://review.opendev.org/c/openstack/nova/+/870938 | |
| 12:38:15 | opendevreview | melanie witt proposed openstack/nova master: Update driver BDMs with ephemeral encryption image properties https://review.opendev.org/c/openstack/nova/+/870939 | |
| 13:00:04 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw https://review.opendev.org/c/openstack/nova/+/862416 | |
| 13:03:13 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw https://review.opendev.org/c/openstack/nova/+/862416 | |
| 13:28:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: Test OOM killed test https://review.opendev.org/c/openstack/nova/+/870950 | |
| 13:29:25 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: Test OOM killed test https://review.opendev.org/c/openstack/nova/+/870950 | |
| 13:29:54 | gibi | bauzas: my trial is at https://review.opendev.org/c/openstack/tempest/+/870947 and https://review.opendev.org/c/openstack/nova/+/870950 | |
| 13:33:27 | opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: Test OOM killed test https://review.opendev.org/c/openstack/nova/+/870950 | |
| 13:35:23 | kashyap | gibi: Thanks for catching my sloppiness here (I actually was rephrasing it locally). Do my replies seem reasonable to you? - https://review.opendev.org/c/openstack/nova/+/870794 | |
| 13:37:16 | bauzas | gibi: hmm, TIL about dstat and memory_tracker usage from devstack | |
| 13:41:08 | sean-k-mooney | ya they run as a service in all the devstack jobs | |
| 13:41:19 | sean-k-mooney | we used to have peak_mem_tracker too or somethign like that | |
| 13:41:31 | sean-k-mooney | dstat more or less has all the info you want/need | |
| 13:54:12 | kashyap | gibi: When you get a sec, I'm wondering what else is missing in this unit-test diff to check the API is called only once - https://paste.opendev.org/show/b7CXHOkMeuuXuzQD6QtW/ | |
| 14:15:49 | gibi | bauzas: this fresh functional failure https://7ffaea22ff93fca2f0ea-bf433abff5f8b85f7f80257b72ac6f67.ssl.cf5.rackcdn.com/869900/7/gate/nova-tox-functional-py38/3b10d8a/testr_results.html is very similar to what we discuss with melwitt in the comments of https://bugs.launchpad.net/nova/+bug/1946339 | |
| 14:16:52 | gibi | so probably eventlets are escaping the end of test case execution whete they were born and interfering with later tests | |
| 14:17:22 | sean-k-mooney | parsing that statement... | |
| 14:18:09 | sean-k-mooney | that sound kind fo familar | |