| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-28 | |||
| 21:29:53 | gouthamr | thanks tosky! | |
| 21:30:31 | clarkb | the fixes are all in the gate right now | |
| 21:30:40 | clarkb | we have to roll them forward from victoria to master due to grenade | |
| 21:32:11 | gouthamr | great, thank you clarkb (/me watches https://review.opendev.org/c/openstack/devstack/+/811399 and the train) | |
| 22:22:59 | gmann | gouthamr: as you are here, nova ceph jobs also broken which need fix in devstack-plugin-ceph. https://review.opendev.org/q/I7061f8d1491ff957452c9c777e40186a4e9c324e | |
| 22:23:16 | gmann | I am still testing it on victoria and wallaby | |
| 22:23:29 | gouthamr | ack, saw your change gmann | |
| 22:23:49 | gmann | gouthamr: ok, I might need your help to merge it after testing | |
| 22:23:55 | gouthamr | thanks for the patch! yep :) i'm on it | |
| 22:24:01 | gmann | thanks | |
| 23:20:03 | opendevreview | Ghanshyam proposed openstack/nova stable/wallaby: DNM: Testing nova-grenade-multinode with neutron-trunk https://review.opendev.org/c/openstack/nova/+/811513 | |
| 23:22:48 | opendevreview | Ghanshyam proposed openstack/nova stable/xena: DNM: Testing nova-grenade-multinode with neutron-trunk https://review.opendev.org/c/openstack/nova/+/811491 | |
| #openstack-nova - 2021-09-29 | |||
| 00:51:35 | opendevreview | Ghanshyam proposed openstack/nova master: DNM testing grenade neutron-trunk fix https://review.opendev.org/c/openstack/nova/+/811118 | |
| 01:02:48 | opendevreview | Ghanshyam proposed openstack/nova stable/xena: DNM: Testing nova-grenade-multinode with neutron-trunk https://review.opendev.org/c/openstack/nova/+/811491 | |
| 01:03:18 | opendevreview | Ghanshyam proposed openstack/nova stable/wallaby: DNM: Testing nova-grenade-multinode with neutron-trunk https://review.opendev.org/c/openstack/nova/+/811513 | |
| 01:34:42 | opendevreview | Hang Yang proposed openstack/nova master: Support creating servers with RBAC SGs https://review.opendev.org/c/openstack/nova/+/811521 | |
| 01:48:58 | opendevreview | melanie witt proposed openstack/nova stable/train: address open redirect with 3 forward slashes https://review.opendev.org/c/openstack/nova/+/806629 | |
| 01:52:57 | opendevreview | Federico Ressi proposed openstack/nova master: Check Nova project changes with Tobiko scenario test cases https://review.opendev.org/c/openstack/nova/+/806853 | |
| 01:56:37 | opendevreview | Federico Ressi proposed openstack/nova master: Debug Nova APIs call failures https://review.opendev.org/c/openstack/nova/+/806683 | |
| 04:10:29 | opendevreview | Ghanshyam proposed openstack/nova stable/victoria: DNM: Testing nova-grenade-multinode with neutron-trunk https://review.opendev.org/c/openstack/nova/+/811540 | |
| 06:43:02 | bauzas | good morning Nova | |
| 08:16:00 | deke | Hi | |
| 08:16:48 | deke | Has there been any discussion about implementing a MAAS driver for nova to enable ironic-like functionality for users who have deployed openstack with juju on MAAS? | |
| 08:36:13 | bauzas | deke: I haven't heard anything about this | |
| 08:36:30 | bauzas | deke: tbh, it would be a laaaarge discussion, right? | |
| 08:36:38 | deke | yea it would | |
| 08:36:43 | deke | it was just a thought I had | |
| 08:37:00 | deke | if we are already deploying on top of a baremetal provisioning service | |
| 08:37:15 | deke | then why deploy another baremetal provisioning service | |
| 08:39:35 | bauzas | we are not saying "no" for a new virt driver | |
| 08:40:31 | bauzas | but for having a new virt driver upstream, that means we need to discuss how to use it and how we could verify it by the CI | |
| 08:40:55 | bauzas | that means large discussions during a lot of PTGs + making sure we at least have a third-party CI | |
| 08:42:49 | lyarwood | dansmith: https://review.opendev.org/c/openstack/grenade/+/811117 - looks like you're the only remaining active core on grenade, can you ack this when you get online to unblock Nova's gate? | |
| 09:03:28 | gibi | gmann: thanks for the patches! | |
| 09:05:12 | bauzas | gmann: what gibi said, thanks for having worked on them | |
| 09:06:08 | bauzas | so the xena and master changes for fixing the placement API endpoints are now merged, we only have the grenade issue left, right? | |
| 09:07:08 | lyarwood | Yup I believe so | |
| 09:07:12 | lyarwood | and I think we only need it in master | |
| 09:07:15 | gibi | yupp thats my view too | |
| 09:07:17 | lyarwood | I don't know why we've backported it tbh | |
| 09:07:53 | gibi | lyarwood: you mean why we are enabling trunk testing on wallaby and back? | |
| 09:08:26 | lyarwood | on stable/xena and backwards, I thought grenade used the current branch of itself and the previous branch for everything else for the initial deploy | |
| 09:08:47 | lyarwood | so master grenade deploys a stable/xena env and then upgrades that to master | |
| 09:10:08 | bauzas | gibi: I stupidely forgot to +1 the RC2 patches yesterday before leaving | |
| 09:10:14 | bauzas | :facepalm: | |
| 09:10:29 | bauzas | now we need elodilles_pto but as his nick says, he's on PTO :) | |
| 09:12:17 | gibi | lyarwood: if we do this https://review.opendev.org/c/openstack/devstack/+/811518/2/lib/tempest then we need to do this as well https://review.opendev.org/c/openstack/grenade/+/811542/1/.zuul.yaml on stable. But I agre that we don't have to enable trunk on stable, skip is OK to me too. | |
| 09:12:59 | lyarwood | yeah I agree with the comment on the stable/wallaby change that we shouldn't be adding tests to stable this late on | |
| 09:13:01 | gibi | bauzas: I think other release cores can approve the RC2 we don't necessary need to wait for elodilles_pto. | |
| 09:13:12 | gibi | anyhow he is back tomorrow so no worries | |
| 09:14:13 | gibi | lyarwood: ack, we can discuss this with gmann when he is up | |
| 09:20:45 | lyarwood | kk | |
| 10:56:11 | opendevreview | Lee Yarwood proposed openstack/nova master: nova-manage: Ensure mountpoint is passed when updating attachment https://review.opendev.org/c/openstack/nova/+/811713 | |
| 11:17:24 | opendevreview | Lee Yarwood proposed openstack/nova master: nova-manage: Always get BDMs using get_by_volume_and_instance https://review.opendev.org/c/openstack/nova/+/811716 | |
| 11:26:10 | lyarwood | https://review.opendev.org/c/openstack/nova/+/811118 - cool the check queue is passing with the grenade fix | |
| 11:27:28 | gibi | \o/ | |
| 12:00:50 | mdbooth | 👋 Are we confident that https://review.opendev.org/c/openstack/devstack/+/811399 fixed the devstack issue on Ubuntu? My devstack-using CI job still seems to be failing: | |
| 12:00:51 | mdbooth | https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cluster-api-provider-openstack/1009/pull-cluster-api-provider-openstack-e2e-test/1443169418262614016/artifacts/logs/cloud-final.log | |
| 12:02:32 | mdbooth | I can see the following in the devstack output: "ProxyPass "/placement" "unix:/var/run/uwsgi/placement-api.socket|uwsgi://uwsgi-uds-placement-api" retry=0", which looks like it contains the fix. However I'm still seeing the borked GETs to placement | |
| 12:04:33 | mdbooth | I'm far from discounting some other issue in the environment, btw, just looking for confirmation or otherwise that we've seen this fix the issue. | |
| 12:05:09 | gibi | mdbooth: you logs shows the original error I can confirm as placement logs ""GET /placemen//resource_providers?in_" | |
| 12:05:21 | mdbooth | gibi: Right | |
| 12:06:07 | gibi | and I do see the ProxyPass you mentioned, and that is the correct one | |
| 12:06:21 | gibi | are you sure that your env using the ProxyPass that is logged? | |
| 12:06:23 | mdbooth | gibi: There's a git pull; git show up the top of that output which suggests the working directory is at "7f16f6d4 Fix uwsgi config for trailing slashes" | |
| 12:07:28 | mdbooth | gibi: i.e. Can I confirm that the apache config actually contains that ProxyPass? | |
| 12:07:54 | gibi | gibi: or that apache2 was restarted / reload after the setting is applied | |
| 12:08:22 | gibi | hm, I see an apache2 reload later | |
| 12:08:54 | mdbooth | This is running in cloud-init on initial boot. *However* it is running from an image that already contains devstack bits, so it's entirely possible there's dirty state there. | |
| 12:09:28 | mdbooth | Let me try to confirm that. I hadn't considered that it might not actually be updating the apache config. | |
| 12:09:54 | gibi | hm, sorry I only see logs that suggest the reload like | |
| 12:09:54 | gibi | To activate the new configuration, you need to run: | |
| 12:09:54 | gibi | systemctl reload apache2 | |
| 12:10:20 | gibi | but I don't see the reload itself | |
| 12:11:12 | gibi | so If you can log into the system then try accessing placment if it fails then check the ProxyPass confing then reload apache and check that placement is accessible now | |
| 12:11:38 | gibi | (I did this manually with in my local devstack so I do belive the fix helps at least in devstack) | |
| 12:12:27 | mdbooth | Unfortunately this is running in a CI system I don't have access to :( I have to debug via modifying the job and re-executing 😬 | |
| 12:12:45 | mdbooth | I'll see what I can confirm. Thanks! | |
| 12:13:59 | gibi | mdbooth: no problem, let us know if this still fails for you | |
| 12:18:14 | mdbooth | Ok, I can confirm that the image the machine is created from already contains dirty devstack state | |
| 12:18:28 | mdbooth | Including the broken ProxyPass directives | |
| 12:19:08 | mdbooth | So unless we consider it a bug that devstack doesn't update that, I'm guessing this is my problem? | |
| 12:20:30 | gibi | mdbooth: so you have both wrong and bad ProxyPass lines in the config file? | |
| 12:20:41 | gibi | I mean wrong and good | |
| 12:20:41 | gibi | :D | |
| 12:21:11 | mdbooth | So *before* pulling and executing a patched devstack I already have bad config present | |
| 12:21:21 | mdbooth | And running the patched devstack doesn't fix it | |
| 12:21:58 | gibi | I do see multiple ProxyPass lines locally too, I guess those are from multiple unstack.sh / stack.sh runs in my case | |
| 12:22:04 | lyarwood | yeah that isn't a devstack bug | |
| 12:22:07 | lyarwood | ./clean.sh first | |
| 12:22:38 | gibi | hm, if clean.sh removes it then I agree it is not a bug (I'm lazy to always run clean.sh but I should) | |
| 12:23:03 | mdbooth | lyarwood: Or more likely summon the arcane wizards of CI and update the 'preinstalled' image! | |
| 12:23:22 | lyarwood | yeah or that if it's already baked into the image | |
| 12:23:24 | gibi | mdbooth: yeah, for CI I suggest to always start from a clean image :) | |
| 12:23:26 | lyarwood | that's odd tbh | |
| 12:24:05 | mdbooth | Apparently it saves a ton of time, although I haven't personally measured it. I'm about to measure it, though :) | |
| 12:24:44 | gibi | not too long ago dansmith added parallelism to devstack stack.sh run that helped a lot with runtime | |
| 12:27:51 | lyarwood | yeah that we use across most jobs now AFAIK | |
| 12:28:02 | mdbooth | lyarwood: Is there a flag? | |
| 12:29:16 | gibi | there should be | |