| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-12-07 | |||
| 18:51:01 | sean-k-mooney | sorry i tought the -v on https://review.opendev.org/c/openstack/nova/+/866091/2 | |
| 18:51:07 | sean-k-mooney | was for the unit test failure | |
| 18:51:23 | sean-k-mooney | its not its form tempest-slow-py3 | |
| 18:51:58 | sean-k-mooney | melwitt: then yes i think that single mock is fine | |
| 18:52:03 | melwitt | oh, yeah. stable/train is never with it so its CI fails half the time :P | |
| 18:52:47 | sean-k-mooney | test_volume_swap failed in the slow job on the second patch | |
| 18:53:17 | sean-k-mooney | Details: volume 29450c9c-2303-46b1-a3c7-46c22abd2a90 failed to reach available status (current in-use) within the required time (196 s). | |
| 18:53:40 | sean-k-mooney | i dont think that is related to your patch | |
| 18:53:51 | sean-k-mooney | so i guess im +1 on both | |
| 18:54:11 | sean-k-mooney | +1 becasue we have not merged this on the newwer branches | |
| 18:54:28 | melwitt | yeah | |
| 18:54:31 | melwitt | ok, cool, thanks | |
| 18:57:40 | sean-k-mooney | soory it took so long to get to that point | |
| 18:57:55 | melwitt | that's ok :D | |
| 18:58:34 | sean-k-mooney | given its not 7 here its proably a sign that my brain has finshed for today so im going to follw its lead and go have food | |
| 18:58:40 | melwitt | I really wanted to know why it's failing too, like why 2.7 only | |
| 18:58:51 | sean-k-mooney | ya its odd | |
| 18:59:04 | sean-k-mooney | i wonder if it was just flaky | |
| 18:59:22 | sean-k-mooney | like woudl it alwasy fail | |
| 18:59:26 | melwitt | I ran it locally a bunch and it was very consistent | |
| 18:59:44 | sean-k-mooney | weird | |
| 18:59:47 | melwitt | I didn't run it in a long running loop but just while I was messing with it I ran it several times | |
| 19:00:41 | melwitt | if I left it running in a loop overnight, maybe it would pass at some point 😂 | |
| 19:06:42 | sean-k-mooney | thats a lower pass rate then is desireable in ci | |
| 19:06:55 | sean-k-mooney | so i think we are good with your change | |
| 19:06:56 | melwitt | a little | |
| #openstack-nova - 2022-12-08 | |||
| 06:46:21 | opendevreview | yangzhipeng proposed openstack/nova master: when evacuate is performing, and restart compute node, if get instance info early, the instance state is not latest. this will reset instance task to error incorrectly, so refresh instance when modify instance state. https://review.opendev.org/c/openstack/nova/+/866960 | |
| 06:53:50 | opendevreview | yzp proposed openstack/nova master: Remove all tag if instance has beed hard deleted. https://review.opendev.org/c/openstack/nova/+/865362 | |
| 07:54:21 | opendevreview | yzp proposed openstack/nova master: Refresh instance when init instance in rebuilding https://review.opendev.org/c/openstack/nova/+/866960 | |
| 07:58:33 | opendevreview | yzp proposed openstack/nova master: Remove all tag if instance has beed hard deleted. Signed-off-by: yangzhipeng |
|
| 07:58:33 | opendevreview | yzp proposed openstack/nova master: Remove all tag if instance has beed hard deleted. Signed-off-by: yangzhipeng |
|
| 08:00:22 | opendevreview | yzp proposed openstack/nova master: Remove all tag if instance has beed hard deleted. Signed-off-by: yangzhipeng |
|
| 08:03:18 | opendevreview | yzp proposed openstack/nova master: Remove all tag if instance has beed hard deleted. https://review.opendev.org/c/openstack/nova/+/865362 | |
| 08:04:56 | opendevreview | norman shen proposed openstack/nova master: Skip deleting instance info for same host migration https://review.opendev.org/c/openstack/nova/+/866521 | |
| 09:48:55 | opendevreview | yzp proposed openstack/nova master: Refresh instance when init instance in rebuilding https://review.opendev.org/c/openstack/nova/+/866960 | |
| 10:27:12 | opendevreview | yzp proposed openstack/nova master: Refresh instance when init instance. https://review.opendev.org/c/openstack/nova/+/866960 | |
| 13:24:38 | gibi | do we have a blocked nova gate? https://zuul.opendev.org/t/openstack/build/32c3c4a56eec4d7eac956a629abdee362 | |
| 13:24:54 | gibi | it looks we have constant failure in https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack/nova | |
| 13:25:04 | gibi | since yesterday | |
| 13:25:20 | gibi | also in https://zuul.opendev.org/t/openstack/builds?job_name=nova-grenade-multinode&project=openstack/nova | |
| 13:26:05 | gibi | "venv: failed with tempest is not allowed, use allowlist_externals to allow it" | |
| 13:47:36 | sean-k-mooney | that sound like we are not installing tempet in the venv properly | |
| 13:51:22 | gibi | yeah | |
| 13:51:35 | gibi | grenade fails a bit differently but maybe related too | |
| 13:52:54 | gibi | this seems related https://review.opendev.org/c/openstack/tempest/+/865314/1/tox.ini | |
| 13:53:16 | gibi | but merged couple weeks ago | |
| 13:53:23 | sean-k-mooney | ya so there was a mail about that | |
| 13:53:38 | sean-k-mooney | the allowlist_external=* is problematic | |
| 13:57:53 | sean-k-mooney | gibi: https://lists.openstack.org/pipermail/openstack-discuss/2022-November/031343.html | |
| 13:59:16 | sean-k-mooney | clarkb: ^ so that tempest fix may have broken grenade? | |
| 14:02:35 | sean-k-mooney | gibi: my guess is the tempest venv is not being recreated | |
| 14:03:00 | sean-k-mooney | so on zed it would have had allowlist_external=* and used global pip | |
| 14:03:31 | sean-k-mooney | as a result tempest was likely insalled outside of the venv? | |
| 14:03:45 | sean-k-mooney | and then with the master tox file it would have failed? | |
| 14:04:05 | gibi | it fails in nova-ceph-multistore too so it is not grenade specific | |
| 14:04:12 | sean-k-mooney | not sure if that actully what is happening as i tough we used master tempetst regradels of the branch but i would guess its soemthign like that | |
| 14:04:26 | sean-k-mooney | oh ok | |
| 14:04:46 | sean-k-mooney | so may somethign related to our jobs | |
| 14:05:38 | frickler | yes, that allowlist needs to include tempest | |
| 14:06:02 | sean-k-mooney | frickler: im not sure it should | |
| 14:06:09 | sean-k-mooney | tempest shoudl be installed in teh venv | |
| 14:06:21 | sean-k-mooney | we shoudl not be using tempest form teh host right? | |
| 14:06:41 | sean-k-mooney | we install tempest in a venv because it is often not the same version of openstack | |
| 14:06:54 | gibi | this is where we fail https://github.com/openstack/devstack/blob/master/lib/tempest#L731 | |
| 14:06:56 | sean-k-mooney | and we do it that way to avoid the possibel depency conflcit that woudl create | |
| 14:07:16 | sean-k-mooney | so tempest should __not__ be in the allowlist | |
| 14:07:41 | sean-k-mooney | ya so that shoudl be using tempst form the venv | |
| 14:08:09 | sean-k-mooney | shoudl it be tox -evenv-tempest https://github.com/openstack/devstack/blob/master/lib/tempest#L709 | |
| 14:08:14 | sean-k-mooney | instead of venv | |
| 14:08:47 | sean-k-mooney | tox -evenv-tempest seams to be what we use on the other lines | |
| 14:08:57 | gibi | hm, good point | |
| 14:09:02 | gibi | that seems like the wrong env | |
| 14:09:40 | gibi | I will push a patch | |
| 14:09:41 | frickler | oh, cool, so this found an actual bug, nice | |
| 14:11:33 | ralonsoh | sean-k-mooney, thanks! I saw this issue in out CI too | |
| 14:11:38 | ralonsoh | our* | |
| 14:14:41 | gibi | devstack fix is up https://review.opendev.org/c/openstack/devstack/+/866997 | |
| 14:28:38 | opendevreview | Konrad Gube proposed openstack/nova-specs master: Use extend volume completion action https://review.opendev.org/c/openstack/nova-specs/+/855490 | |
| 14:44:19 | opendevreview | Bence Romsics proposed openstack/nova master: doc: soft delete and shadow tables https://review.opendev.org/c/openstack/nova/+/867001 | |
| 15:06:12 | opendevreview | Bence Romsics proposed openstack/nova master: doc: soft delete and shadow tables https://review.opendev.org/c/openstack/nova/+/867001 | |
| 15:19:39 | elendrys | Hello here, is this channel open to nova related questions or is it dedicated to developers ? | |
| 15:45:00 | frickler | elendrys: this is a bit of a grey area, #openstack is what is meant for general questions, but the chance to get an answer there is low, so you might as well try here directly | |
| 15:47:14 | fungi | also the openstack-discuss@lists.openstack.org mailing list has a much higher chance of getting you an answer eventually | |
| 15:54:07 | elendrys | Ok thank you | |
| 15:54:35 | sean-k-mooney | elendrys: did you have a question in general | |
| 15:55:00 | sean-k-mooney | while the channel is not for supprot we will try an point you in the right direction in genreal | |
| 15:55:44 | elendrys | Yes | |
| 15:56:23 | elendrys | I have servers with cinder volumes, backed in PureStorage array over iscsi | |
| 15:57:21 | sean-k-mooney | ok so the vm is using host mount iscsi block devices passthroug to the vms | |
| 15:57:28 | elendrys | When i resize volumes attached to running servers, I can see in nova's log that it rescans the array to get the new size when cinder notifies | |
| 15:57:55 | elendrys | But in the vm, the size always have 1 notification late size | |
| 15:58:23 | elendrys | I mean, a 10Gb vol expended to 20 stills show as 10, but if I extend to 30, it shows 20 | |
| 15:58:49 | elendrys | I can't get why (probably libvirt) it is missing something | |
| 15:58:54 | sean-k-mooney | hum interesting. | |
| 15:59:13 | sean-k-mooney | am im not sure this is libvirt/qemu related | |
| 15:59:22 | elendrys | If I stop then start the server it shows 30 as intended | |
| 15:59:28 | sean-k-mooney | i woudl suppect this is more likely to be related to iscsid or simialr | |
| 15:59:59 | sean-k-mooney | elendrys: did you check on the host if the host kernel sees the correct volume size | |
| 16:00:30 | sean-k-mooney | you should be able to find the dev path form the vm xml and check with lsblk or a similar too to see if it sees the correct current size | |
| 16:01:06 | sean-k-mooney | if the hosts sees the correct size then the issue is likely at the qemu level | |