| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-12 | |||
| 10:02:16 | sean-k-mooney | not in the tempest one | |
| 10:02:19 | rpittau | ok | |
| 10:03:08 | sean-k-mooney | ya its form 2021-04-09 22:14:21.761580 | controller | 2021-04-09 22:14:21.761 | + inc/python:setup_package:445 : pip_install -e /opt/stack/new/ironic | |
| 10:03:26 | sean-k-mooney | that pulls in Collecting oslo.messaging===12.5.2 | |
| 10:04:10 | sean-k-mooney | rpittau: the issue here is the ironic job is only upgrading ironic and tempest | |
| 10:04:18 | sean-k-mooney | rpittau: it need to upgrade all installed services | |
| 10:04:33 | sean-k-mooney | because they all share the same version of oslo.messaging | |
| 10:04:48 | yoctozepto | rpittau, sean-k-mooney: that thing is fixing, e.g., ironic-grenade-dsvm job | |
| 10:04:50 | rpittau | sean-k-mooney: mmm ok I see, I think this was intended for some reason, need to double-check with dtantsur and TheJulia | |
| 10:05:04 | yoctozepto | (on train) | |
| 10:05:05 | sean-k-mooney | and the one form victorica that ironic is installing when its upgraded breaks ussuri | |
| 10:05:38 | yoctozepto | the failure was in smoke tests BEFORE upgrade | |
| 10:05:41 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/ussuri: Add regression test for bug #1908075 https://review.opendev.org/c/openstack/nova/+/783655 | |
| 10:05:44 | openstack | bug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Low,In progress] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood) | |
| 10:05:45 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/ussuri: api: Reject volume attach requests when an active bdm exists https://review.opendev.org/c/openstack/nova/+/783656 | |
| 10:05:54 | sean-k-mooney | yoctozepto: the smoke tests are disabled | |
| 10:06:01 | sean-k-mooney | yoctozepto: before and after upgrade in this job | |
| 10:06:28 | sean-k-mooney | yoctozepto: this job is failing because the victoria version of oslo messagin is been install when ironic is upgraded | |
| 10:06:37 | sean-k-mooney | yoctozepto: but the rest of openstack is not upgraded | |
| 10:06:48 | yoctozepto | what job are we talking about? :D I am only about ironic-grenade-dsvm on train | |
| 10:07:05 | sean-k-mooney | yoctozepto: ironic-grenade | |
| 10:07:14 | sean-k-mooney | no -dvsm | |
| 10:07:18 | yoctozepto | oh, the new one; then it's irrelevant as you said | |
| 10:07:22 | yoctozepto | must be other issue | |
| 10:07:32 | sean-k-mooney | yep the issue is as i said | |
| 10:07:50 | sean-k-mooney | the ironic plugin was the only thing run in the upgrade | |
| 10:08:18 | sean-k-mooney | so its currently trying to run victoria ironic with ussuri everything else but oslo move or removed a decorator | |
| 10:08:39 | sean-k-mooney | so the ussuri code cant run with the new verion of the oslo package form victoria | |
| 10:08:52 | sean-k-mooney | the fix is just to upgrade all the porjects | |
| 10:09:40 | sean-k-mooney | yoctozepto: https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt#47896-47897 see it just ran ironic | |
| 10:12:04 | yoctozepto | yes, I confirm what sean-k-mooney is saying | |
| 10:12:17 | yoctozepto | sorry for confusion, I misread the question | |
| 10:12:53 | rpittau | sean-k-mooney, yoctozepto, thanks, I will need to verify with other ironic cores | |
| 10:13:31 | sean-k-mooney | rpittau: unless ironic is installed in a venv its unsafe to upgrade it in isolation | |
| 10:13:44 | sean-k-mooney | rpittau: since we share depencies | |
| 10:13:44 | yoctozepto | ++ | |
| 10:13:57 | rpittau | ofc, makes total sense | |
| 10:13:58 | sean-k-mooney | rpittau: you could try to do that if you use the old UC | |
| 10:14:07 | yoctozepto | (psst, you can always base your gate on kolla, just saying ;-) ) | |
| 10:14:11 | rpittau | :) | |
| 10:14:34 | rpittau | I guess I have a topic for today meeting :) | |
| 10:14:37 | sean-k-mooney | yoctozepto: do you have full tempest/upgrade jobs in kolla yet? | |
| 10:15:11 | sean-k-mooney | i know it was a goal a long time ago just never followed up to see if ye completed it | |
| 10:16:51 | yoctozepto | sean-k-mooney: tempest not :-( but last cycle I made it able to track branches of other projects with zuul so that depends-on works etc. | |
| 10:17:00 | yoctozepto | I got bifrost to check against kolla-ansible | |
| 10:17:08 | yoctozepto | because they often broke us :D | |
| 10:17:43 | sean-k-mooney | yoctozepto: i still feel a little bad that i never had time to do phase 2 of the bifrost work | |
| 10:18:35 | sean-k-mooney | when i implemeted the inital work i had planned to decomose it into multiple contaier runnign the differen services instead of using systemd. | |
| 10:18:57 | sean-k-mooney | oh the other had it worked so :) | |
| 10:19:00 | yoctozepto | sean-k-mooney: https://github.com/markgoddard/bikolla might interest you then | |
| 10:19:40 | sean-k-mooney | haha why not just use kolla ironic support | |
| 10:19:57 | sean-k-mooney | e.g. just enable the minium service for ironic | |
| 10:20:11 | sean-k-mooney | i assume that is what that does | |
| 10:20:38 | sean-k-mooney | hum yes it looks like that is correct more or less https://github.com/markgoddard/bikolla/blob/master/etc/kolla/globals.yml | |
| 10:21:32 | yoctozepto | yes ;-) | |
| 10:21:41 | yoctozepto | just need to orchestrate the thing | |
| 10:22:11 | sean-k-mooney | im debating if im going to keep ironic or not when i redeploy my home cluster | |
| 10:22:32 | sean-k-mooney | part of me wants too but i use it rarely | |
| 10:22:47 | sean-k-mooney | and it seams to have issue with ubunutu | |
| 10:22:54 | sean-k-mooney | my centos image work fine | |
| 10:23:23 | sean-k-mooney | but i dont think the ipa image or cloud init fully work with ubunuts netplan conrigurtion | |
| 10:23:55 | sean-k-mooney | so i end up installing the os with a USB when i need anything other then centos anyway | |
| 10:34:10 | sean-k-mooney | stephenfin: im getting pinged for https://review.opendev.org/q/topic:%22bug%252F1910466%22+(status:open%20OR%20status:merged) downstream in bz. when you have time can you rereview those | |
| 10:34:16 | dtantsur | rpittau, sean-k-mooney, grenade should update all services.. breakage in infra jobs? | |
| 10:34:25 | stephenfin | sean-k-mooney: sure | |
| 10:34:31 | dtantsur | maybe we did something wrong when migrating to zuul v3? | |
| 10:34:52 | sean-k-mooney | dtantsur: perhaps only the plugins seam to be listed | |
| 10:37:33 | elod | just for the record I've looked into the last successful run of ironic-grenade on stable/victoria and it also upgraded to oslo.messaging===12.5.2 | |
| 10:37:56 | sean-k-mooney | elod: did it upgrade the other serivces? | |
| 10:38:32 | sean-k-mooney | its likely not oslo.messaging by the way | |
| 10:38:44 | sean-k-mooney | i think the reraise decorator comes form oslo.utils | |
| 10:38:48 | elod | sean-k-mooney: yes: Upgrade projects: keystone placement swift glance neutron nova ironic | |
| 10:39:20 | sean-k-mooney | https://github.com/openstack/oslo.utils/blob/master/oslo_utils/excutils.py#L145 | |
| 10:39:29 | sean-k-mooney | elod: ya so thats what has changed | |
| 10:39:37 | elod | sean-k-mooney: it seems | |
| 10:39:38 | sean-k-mooney | now its just ironic | |
| 10:39:46 | elod | yes | |
| 10:40:05 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: Add regression test for bug #1908075 https://review.opendev.org/c/openstack/nova/+/785847 | |
| 10:40:09 | openstack | bug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Low,In progress] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood) | |
| 10:40:10 | sean-k-mooney | do we use the victoria or ussuri version of grenade | |
| 10:40:10 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: api: Reject volume attach requests when an active bdm exists https://review.opendev.org/c/openstack/nova/+/785848 | |
| 10:40:28 | sean-k-mooney | e.g. is it the source or target version that does the upgrade | |
| 10:41:15 | elod | it should be victoria | |
| 10:41:26 | elod | in my understanding | |
| 10:41:46 | sean-k-mooney | ok then i should be looking at stable/victoria grenade so | |
| 10:49:55 | sean-k-mooney | i suspect the issue is with the grenaderc generation | |
| 10:50:05 | sean-k-mooney | UPGRADE_PROJECTS is empty by default | |
| 10:50:12 | sean-k-mooney | plugins are auto added to it | |
| 10:50:21 | sean-k-mooney | which is why ironic is there | |
| 10:51:06 | sean-k-mooney | there are added by register_project_for_upgrade | |
| 10:51:14 | sean-k-mooney | https://opendev.org/openstack/grenade/src/branch/master/inc/upgrade#L101-L111 | |
| 10:54:26 | sean-k-mooney | each of the in tree services should register them selves https://opendev.org/openstack/grenade/src/commit/15e02fec78cdcce00d302fc2c6c29afea91f6f14/projects/10_keystone/settings#L1 | |
| 10:54:55 | sean-k-mooney | but i am wondering if that setting file is correctly loaded | |
| 11:08:15 | yoctozepto | ok, so still no consensus on what has failed? | |
| 11:08:43 | yoctozepto | oh, I see the missing projects were confirmed | |
| 11:09:08 | yoctozepto | so then it's "only" a matter of tracking what introduced the breaking change in behaviour | |
| 11:11:23 | sean-k-mooney | yoctozepto: something cause the list of project to be changed ya im not seeing what that was at first glance | |
| 11:11:44 | yoctozepto | sean-k-mooney: ack | |
| 11:59:49 | openstackgerrit | Qiu Fossen proposed openstack/nova-specs master: Support live migrate vtpm server https://review.opendev.org/c/openstack/nova-specs/+/785860 | |
| 12:12:12 | openstackgerrit | Qiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data https://review.opendev.org/c/openstack/nova-specs/+/785563 | |