| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-11-16 | |||
| 14:03:48 | sean-k-mooney | first its used for auth as it has an expiry and can only be issues by someone in the same porject | |
| 14:04:06 | sean-k-mooney | second we use it to look up which instance to proxy the websocket conenction too | |
| 14:04:46 | sean-k-mooney | so the porxy checks if the token is still valid and then uses it to lookup what host/tcp port to proxy too | |
| 14:05:07 | SvenKieske | so it seems it's always enabled? at least here is no config toggle documented? https://docs.openstack.org/nova/latest/configuration/config.html#consoleauth | |
| 14:05:17 | sean-k-mooney | combin that with vencrypt and that will mean the only way to conenct to the console is via the proxy | |
| 14:06:06 | sean-k-mooney | SvenKieske: so console auth used to be a speerate service but it was roled into the other nova servies several years ago | |
| 14:06:21 | sean-k-mooney | the token_tll is all that is left | |
| 14:06:55 | sean-k-mooney | so yes will alwasy issue the token when you do openstack console url show <instance> | |
| 14:07:09 | sean-k-mooney | with a default expiry of 10 minutes | |
| 14:07:43 | sean-k-mooney | the current behavior of the proxy is to keep the active session alive after token expiry but we are addign a featue to terminate it this cycle | |
| 14:08:20 | sean-k-mooney | it will be opt in via a config option since most end user will wont the current beahiovor | |
| 14:19:33 | SvenKieske | okay, nice to know, thank you very much for the detailed explanations, as always :) | |
| 14:26:07 | opendevreview | Merged openstack/nova stable/zed: Add debug logging when Instance raises OrphanedObjectError https://review.opendev.org/c/openstack/nova/+/888161 | |
| 14:26:56 | opendevreview | John Garbutt proposed openstack/nova-specs master: Expose PCI device NUMA using PXB https://review.opendev.org/c/openstack/nova-specs/+/869416 | |
| 14:35:00 | SvenKieske | opened a bug for our deployment scenario, as we don't currently seem to lock down the connection between proxy and qemu: https://bugs.launchpad.net/kolla-ansible/+bug/2043709 | |
| 14:35:21 | sean-k-mooney | most installers dont | |
| 14:35:41 | sean-k-mooney | its not even done in tripleo and redhat added the feature in nova :P | |
| 14:35:51 | sean-k-mooney | but it is somethign we are planning to enable soon | |
| 14:36:17 | sean-k-mooney | the main blocker to using this by defualt in installer is the need to do tls cert generation and rotaion | |
| 14:37:33 | sean-k-mooney | we are currently in the process of movign our downstream product to use golang operator to deploy openstack on openshift | |
| 14:37:56 | sean-k-mooney | and we plan ot leverage the fact that we have access to certmanager in that env to handle that for us | |
| 14:40:52 | opendevreview | Pierre Riteau proposed openstack/nova master: Fix a couple of typos https://review.opendev.org/c/openstack/nova/+/892300 | |
| 15:12:13 | opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Proposes mdev live-migration support in libvirt https://review.opendev.org/c/openstack/nova-specs/+/900636 | |
| 16:12:02 | admin1 | i get this "Build of instance 6c7b960d-ede4-4218-99b0-2f9d55f84dab aborted: Volume d53a7298-685a-43cd-89a3-13b9d6207dcd did not finish being created even after we waited 187 seconds or 61 attempts. And its status is creating" -- this is due to a 20gb image which is not present in the hypervisors and they need to be downloaded from glance first to | |
| 16:12:03 | admin1 | use as base image .. which variables in nova.conf do I need to use to fix this ? | |
| 16:20:45 | opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Proposes mdev live-migration support in libvirt https://review.opendev.org/c/openstack/nova-specs/+/900636 | |
| 16:47:49 | opendevreview | Merged openstack/nova stable/yoga: Add debug logging when Instance raises OrphanedObjectError https://review.opendev.org/c/openstack/nova/+/888162 | |
| 17:11:27 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow enabling cpu_power_management with 0 dedicated CPUs https://review.opendev.org/c/openstack/nova/+/901188 | |
| 17:14:34 | opendevreview | John Garbutt proposed openstack/nova-specs master: Expose PCI device NUMA using PXB https://review.opendev.org/c/openstack/nova-specs/+/869416 | |
| 17:24:57 | Continuity__ | admin1: block_device_allocate_retries and block_device_allocate_retries_interval | |
| 17:25:12 | Continuity | in nova.conf will allow you to adjust those timeouts | |
| 17:39:55 | opendevreview | John Garbutt proposed openstack/nova-specs master: Add spec for PCI Groups https://review.opendev.org/c/openstack/nova-specs/+/899719 | |
| 17:43:58 | opendevreview | John Garbutt proposed openstack/nova-specs master: Add spec for PCI Groups https://review.opendev.org/c/openstack/nova-specs/+/899719 | |
| 17:44:06 | opendevreview | John Garbutt proposed openstack/nova-specs master: Add spec for PCI Groups https://review.opendev.org/c/openstack/nova-specs/+/899719 | |
| 17:45:17 | opendevreview | Jing Geng proposed openstack/nova-specs master: APIImpact https://review.opendev.org/c/openstack/nova-specs/+/901145 | |
| 18:03:33 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Debug log if host state not updated from compute https://review.opendev.org/c/openstack/nova/+/901196 | |
| 18:10:24 | admin1 | Continuity, thanks .. is block_device_allocate_retries: 600 and block_device_allocate_retries_interval: 5 an accpetable value ? | |
| 19:02:49 | opendevreview | Merged openstack/nova master: Fix python shebang https://review.opendev.org/c/openstack/nova/+/898594 | |
| 20:24:28 | Continuity | admin1: yeah that should be fine | |
| 20:24:35 | Continuity | it means it will wait 5 seconds and try 600 times | |
| 20:50:16 | opendevreview | Merged openstack/nova master: Drop remaining deprecated upgrade_levels option for nova-cert https://review.opendev.org/c/openstack/nova/+/898613 | |
| 20:53:09 | opendevreview | Merged openstack/nova stable/xena: Add debug logging when Instance raises OrphanedObjectError https://review.opendev.org/c/openstack/nova/+/888163 | |
| 23:19:50 | opendevreview | melanie witt proposed openstack/nova master: imagebackend: Add support to libvirt_info for LUKS based encryption https://review.opendev.org/c/openstack/nova/+/826755 | |
| 23:19:50 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Configure and teardown ephemeral encryption secrets https://review.opendev.org/c/openstack/nova/+/826754 | |
| 23:19:51 | opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | |
| 23:19:51 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to convert_image https://review.opendev.org/c/openstack/nova/+/870934 | |
| #openstack-nova - 2023-11-17 | |||
| 07:01:45 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix KeyError on assisted snapshot call https://review.opendev.org/c/openstack/nova/+/900783 | |
| 07:38:35 | tkajinam | https://zuul.opendev.org/t/openstack/builds?job_name=grenade-skip-level-always&project=openstack/nova | |
| 07:39:03 | tkajinam | I'm afraid grenade-skip-level-always might be broken. I've not yet looked into it though. | |
| 07:46:41 | frickler | ERROR cinder etcd3gw.exceptions.Etcd3Exception: Not Found | |
| 07:59:12 | tkajinam | frickler, ah, thanks for that info. I saw the same error in a few run so there may be something wrong with cinder | |
| 07:59:20 | tkajinam | it seems cinder is deployed with etcd3gw tooz driver. | |
| 08:03:01 | tkajinam | I vaguely guess the issue was caused by the new tooz release and https://review.opendev.org/c/openstack/tooz/+/891355 might be the cause | |
| 08:06:28 | tkajinam | hmm. do we really need that job in master ? I see the job test upgrade from zed to 2023.2 | |
| 08:06:48 | tkajinam | and does not look appropriate for master | |
| 08:08:36 | tkajinam | ignore above. I noticed I was looking at a wrong branch of grenade... | |
| 09:30:58 | bauzas | tkajinam: looking | |
| 09:31:35 | bauzas | yeah this is failing when upgrading cinder | |
| 09:32:53 | bauzas | we're on Caracal, so it's upgrading from Antelope | |
| 09:33:23 | bauzas | https://github.com/openstack/grenade/commit/159054a9274f984d8e53e5e5bd9e7ab0fb29708b | |
| 09:33:30 | bauzas | (C is a SLURP release) | |
| 09:36:17 | bauzas | heh, fun, we got a GMR report in https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_73a/900783/2/check/grenade-skip-level-always/73acbef/controller/logs/screen-c-vol.txt | |
| 09:39:31 | bauzas | 2023-11-17 06:30:28.118 | ++ /opt/stack/new/grenade/projects/70_cinder/resources.sh:verify:220 : openstack volume show cinder_grenade_vol2 -f shell -c status 2023-11-17 06:30:29.692 | Internal Server Error (HTTP 500) | |
| 09:39:41 | bauzas | yeah, something got wrong on the cinder side | |
| 09:40:27 | bauzas | and yup yup, good catch tkajinam | |
| 09:40:43 | bauzas | https://paste.opendev.org/show/br9ep5e0MjNReG9A9ta7/ | |
| 10:00:07 | frickler | bauzas: cf. https://review.opendev.org/c/openstack/devstack/+/901221 | |
| 10:00:51 | bauzas | ack thanks | |
| 10:01:51 | bauzas | I don't exactly get the patch that broke tooz yet | |
| 10:02:00 | bauzas | but it was merged on Oct 16 | |
| 10:02:11 | bauzas | IMHO we should rather revert it | |
| 10:03:03 | bauzas | https://review.opendev.org/c/openstack/tooz/+/891355 was merged on Sep 28 so it's not the culprit | |
| 10:09:55 | frickler | bauzas: tooz was released yesterday with that patch afaict, added into u-c with https://review.opendev.org/c/openstack/requirements/+/901131 | |
| 10:11:50 | frickler | maybe that default setting change should have triggered a major version bump. also maybe grenade should be patched to handle the etcd upgrade | |
| 10:12:46 | bauzas | frickler: oh right, tooz is a library | |
| 10:13:07 | bauzas | don't judge me, it's Friday morning and I'm already tired :) | |
| 10:13:33 | bauzas | and yeah, looks to me it's a behavioural change | |
| 10:13:34 | frickler | I don't judge anyone on Friday's ;) | |
| 10:15:10 | bauzas | "* Note that this probably means a new minor version of tooz, while the behaviour can be worked around in config, this is enough to break existing systems on upgrade." | |
| 10:15:31 | bauzas | indeed it breaks | |
| 10:17:50 | bauzas | frickler: I'm tempted to require a u-c reverty | |
| 10:17:56 | bauzas | elodilles: ^ | |
| 10:18:50 | bauzas | elodilles: fwiw, our grenade skip level job is broken due to the new tooz releasee https://review.opendev.org/c/openstack/releases/+/899966 | |
| 10:20:34 | elodilles | reading | |
| 10:24:34 | bauzas | I just sent the usual 'hold your rechecks' email to the wolrd | |
| 10:26:07 | bauzas | frickler: I feel something is wrong with the tooz patch | |
| 10:26:33 | bauzas | if we want to have SLURP upgrades, we somehow need to keep a backwards compatible behaviour for a second | |
| 10:27:17 | bauzas | or we need to say 'this is a major breaking change' and only once we no longer support older releases that we want to pull it | |
| 10:31:28 | elodilles | bauzas: thanks for the mail o/ (maybe [oslo] team should have been in the subject, too) | |
| 10:31:40 | bauzas | yeah | |
| 10:32:22 | bauzas | well, oslo is free to deliver major versions of their libraries that break compatibility | |
| 10:32:41 | bauzas | but then they need to signal it correctly for the release team appropriately | |
| 10:33:13 | bauzas | elodilles: I tagged with [tooz] I get that filter is maybe too much restricted ? | |
| 10:33:25 | bauzas | (fwiw, I don't do tag filtering on my mailbox) | |
| 10:33:46 | bauzas | (and I hope oslo contributors to not do it as well) | |
| 10:36:38 | elodilles | yes, they probably see the mail with [tooz] tag as well | |
| 10:40:54 | bauzas | elodilles: https://review.opendev.org/c/openstack/requirements/+/901079 | |
| 10:42:40 | elodilles | +1 | |
| 10:43:19 | elodilles | (from release team perspective i'm still happy we are not finding this one week before the final release o:)) | |
| 10:44:31 | bauzas | all hail to the grenade-skip-level lord | |