| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-01-13 | |||
| 17:18:49 | sean-k-mooney | since the interation of the patch where i added that raise i added a buch of addtion lockign to make it safe to share the lockgrarud between threads | |
| 17:19:37 | sean-k-mooney | so removing the raise is proably the correct thing to do and just make it a debug log or remove that if and the self._active varible entirly | |
| 17:21:00 | sean-k-mooney | that raise only exists to provide a singal to use that | |
| 17:21:03 | sean-k-mooney | thread1 = threading.Thread(target=worker, args=(1, lock_guard)) | |
| 17:21:13 | sean-k-mooney | thread2 = threading.Thread(target=worker, args=(2, lock_guard)) | |
| 17:21:25 | sean-k-mooney | is a programing logic error | |
| 17:22:01 | sean-k-mooney | with the extra lockign i have added that shoudl now be safe even if i dislike it so the extra protection is not required | |
| 17:22:07 | sean-k-mooney | any prefernce on how to fix it | |
| 17:23:05 | sean-k-mooney | we can just say the comments are enouch to discourge that useage and delete the check, we can add a debug log or i can remove the test. | |
| 17:23:06 | gibi | I don't know at the moment. Sorry. | |
| 17:23:19 | gibi | I need to get back to this | |
| 17:23:30 | sean-k-mooney | its fine ill create a quick patch to remvoe the raise and ill ask this quetion on the review | |
| 17:23:42 | sean-k-mooney | and you can leave feedback when you have time | |
| 17:24:02 | gibi | cool. Thanks! | |
| 17:24:59 | sean-k-mooney | was there a bug for this if not ill just do "Related-Bug: #2048837" to tie it to the orgianal bug fix | |
| 17:25:33 | sean-k-mooney | tkajinam: ^ | |
| 17:46:19 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Add capability to load loader and nvram from xml https://review.opendev.org/c/openstack/nova/+/969086 | |
| 18:17:04 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Add capability to load ssm feature from existing xml https://review.opendev.org/c/openstack/nova/+/969131 | |
| 22:59:25 | JayF | gibi: finally got back around to this post-holidays; no code change was needed and I have an explanation: https://review.opendev.org/c/openstack/nova/+/969321 | |
| #openstack-nova - 2026-01-14 | |||
| 05:23:29 | opendevreview | Merged openstack/nova master: Do not fork compute workers in native threading mode https://review.opendev.org/c/openstack/nova/+/965466 | |
| 07:17:41 | ralonsoh | gibi, hello! I've tested pyroute2 0.9.5 in a /requirements patch | |
| 07:17:43 | ralonsoh | https://review.opendev.org/c/openstack/requirements/+/973210 | |
| 07:18:17 | ralonsoh | do you mind if we bump to this version (latest) right now? If we detect any other issue with os-vif, we can revert it again | |
| 07:39:02 | ralonsoh | just for reference: https://zuul.opendev.org/t/openstack/buildset/1b150188643d435e8813c9d8be2ea184. I added nova-ceph-multistore and tempest-integrated-compute jobs | |
| 08:08:33 | opendevreview | Merged openstack/os-vif master: reno: Update master for unmaintained/2024.1 https://review.opendev.org/c/openstack/os-vif/+/965493 | |
| 10:21:16 | gokhan | We are currently running OpenStack Caracal and our security audit team has raised several critical concerns regarding VNC console access via nova-novncproxy. I would like to get your insights on the best practices or if there are any missing configurations on our side. | |
| 10:21:16 | gokhan | Hi everyone, | |
| 10:21:33 | gokhan | Lack of Session Timeout: Once a VNC connection is established, it remains open indefinitely (or for very long periods), which is seen as a major risk if a workstation is left unattended. | |
| 10:21:51 | gokhan | Missing Rate Limiting: There is no built-in mechanism to limit the number of console URLs a single user can request in a short period, potentially leading to resource exhaustion or brute-force attempts | |
| 10:22:10 | gokhan | URL Hijacking Risk: If a console URL is intercepted, it allows immediate access to the VM without further authentication. | |
| 10:23:35 | gokhan | we are planning to implement: Setting [console] enforce_session_timeout = True and adjusting [consoleauth] token_ttl to enforce a hard disconnect (e.g., after 30-60 minutes), Implementing rate limiting at the HAProxy level using stick-tables based on source IP for the /remote-consoles endpoint, Removing none from auth_schemes to enforce vencrypt only. | |
| 10:24:25 | gokhan | Does enforce_session_timeout in Caracal fully address the "infinite session" issue by terminating the WebSocket proxy connection on the server side once the TTL expires? | |
| 10:24:34 | gokhan | Are there any known side effects of setting a very low token_ttl (e.g., 30s) while enforce_session_timeout is active? | |
| 10:25:06 | gokhan | Regarding the password authentication spec, what is the current stability status for KVM/Libvirt in Caracal? https://specs.openstack.org/openstack/nova-specs/specs/wallaby/approved/nova-support-webvnc-with-password-authentication.html | |
| 10:25:45 | gokhan | I would appreciate any guidance or best practices from the you, thanks :) | |
| 10:30:10 | gibi | gokhan: about vnc timeout we have https://specs.openstack.org/openstack/nova-specs/specs/2024.1/implemented/enforce-remote-console-session-timeout.html | |
| 10:35:15 | gibi | > Does enforce_session_timeout in Caracal fully address the "infinite session" issue by terminating the WebSocket proxy connection on the server side once the TTL expires? | |
| 10:35:20 | gibi | I believe so | |
| 10:35:37 | gibi | > Are there any known side effects of setting a very low token_ttl (e.g., 30s) while enforce_session_timeout is active? | |
| 10:37:21 | gibi | as far as I understand that would mean that the VNC session would disconnect after 30secs, if that is OK for you as a user experience then it is fine from nova perspective to configure such short ttl | |
| 10:40:55 | gibi | regarding password auth, it was never implemented. I think the last attempt was https://review.opendev.org/c/openstack/nova/+/622336 | |
| 10:42:08 | gibi | it had an approved spec so I think the nova core team was OK with that feature proposal. So if you are interested it can be revived. | |
| 10:43:53 | gibi | if you are new to openstack then here are our contributor guide https://docs.openstack.org/nova/latest/contributor/index.html | |
| 10:48:37 | gibi | JayF: +2+A thanks for the explanation | |
| 10:56:58 | gokhan | It is very helpful to know that enforce_session_timeout is the server-side solution we were looking for. We now understand that token_ttl serves as both the link validity period and the maximum session duration when this feature is enabled. We will set a reasonable TTL (like 3600s) to balance security and user experience. | |
| 10:56:58 | gokhan | Thanks a lot for the clarification, Gibi! | |
| 10:56:58 | gokhan | Regarding the password authentication; it’s a bit disappointing that it wasn’t fully implemented, but knowing its current status helps us explain the situation to our security audit team. We will evaluate the possibility of developing and reviving this feature with our team. | |
| 10:58:46 | gibi | ralonsoh: re pyroute2 0.9.5 the job you run actaully shows the same error about the event loop https://zuul.opendev.org/t/openstack/build/9e54327cbaa94a4b8ecf9c5472934c97/log/controller/logs/screen-n-cpu.txt#48564-48581 | |
| 10:59:15 | gibi | gokhan: cool. Happy to help | |
| 11:00:51 | ralonsoh | gibi, ok I thought that was fixed | |
| 11:04:56 | gokhan | gibi, I have also created a bug : Live Migration Rollback Corrupts Block Device Mapping of Unrelated VM : https://bugs.launchpad.net/nova/+bug/2133501 do you have any insgihts about it ? we will send a patch for this. this worked for us but I want to be sure I don't ignore critical things > https://bugs.launchpad.net/nova/+bug/2133501/comments/10 | |
| 12:11:35 | sean-k-mooney | gokhan: passward auth was not impletned because it very insecure | |
| 12:12:03 | sean-k-mooney | gokhan: we felt adding ti was mroe harmful as it woudl lead operator to think that it was a sufficnet security mechanium to use in production | |
| 12:12:58 | sean-k-mooney | gokhan: vnc's password auth can have a maxium of i belive 7 ascii pritnabale charters as the password so the password space is very small | |
| 12:13:26 | sean-k-mooney | gokhan: we implented vencypt to do cert based auth between the proxy and the qemu instance instead | |
| 12:13:51 | sean-k-mooney | and then expect the guest passwoard auth to be used on the guest | |
| 12:15:11 | sean-k-mooney | gokhan:... any corss instece behiver is generaly treated as a possible secuirty bug so it shoudl not be filed publicly but its too late to change that now | |
| 12:18:10 | sean-k-mooney | hum i see that proably should have been set as public security rahter hten just public | |
| 12:18:20 | sean-k-mooney | as there are still security and multi tenant aspect to this | |
| 12:29:37 | opendevreview | Balazs Gibizer proposed openstack/nova master: Upgrade note for concurrency mode default change https://review.opendev.org/c/openstack/nova/+/969888 | |
| 12:39:15 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove eventlet from libvirt/test_driver https://review.opendev.org/c/openstack/nova/+/970070 | |
| 12:39:15 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove eventlet from CacheConcurrencyTestCase https://review.opendev.org/c/openstack/nova/+/970069 | |
| 12:39:16 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove eventlet from libvirt/volume/test_mount https://review.opendev.org/c/openstack/nova/+/970071 | |
| 12:42:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Remove eventlet from libvirt/volume/test_mount https://review.opendev.org/c/openstack/nova/+/970071 | |
| 12:55:35 | opendevreview | Takashi Natsume proposed openstack/nova-specs master: Create specs directory for 2026.2 Hibiscus https://review.opendev.org/c/openstack/nova-specs/+/973373 | |
| 12:57:30 | opendevreview | Merged openstack/nova master: [ironic] Use constants from Ironic, test w/ddt https://review.opendev.org/c/openstack/nova/+/969321 | |
| 12:58:39 | gokhan | Thanks for the warning and the detailed explanation, Sean. | |
| 12:58:40 | gokhan | I now understand the rationale behind not implementing the 7-8 character weak password auth. We will focus on hardening the session management side instead. Thanks again for your time and the heads-up! | |
| 12:58:40 | gokhan | Following your advice, we will rely on vEncrypt for proxy-to-instance security and utilize enforce_session_timeout (implemented in Caracal) to mitigate the risks raised by our security team. | |
| 12:58:40 | gokhan | I shared this publicly because I saw the "approved" status on the Wallaby spec and assumed it was a standard feature discussion rather than a zero-day vulnerability. Interestingly, even that spec mentions: "the VNC password is not a very strong security mechanism" which aligns perfectly with what you said. | |
| 12:59:04 | gokhan | sean-k-mooney, ^^ | |
| 13:01:27 | sean-k-mooney | no worries. when we were eveualting the password auth we had some converstation with the qemu and libvirt maintainer and it was there advice to not implement it in nova and pivot to the vencypt approch orginally | |
| 13:02:19 | sean-k-mooney | the enforce_session_timeout was also part of the dicsussion on general hardening of the capablities | |
| 13:04:30 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Use firmware auto-selection by libvirt https://review.opendev.org/c/openstack/nova/+/969132 | |
| 13:12:30 | gokhan | Thanks for the insights on vEncrypt and the 8-character limit of VNC passwords. sean-k-mooney. we have a specific "Session Hijacking" concern. When a legitimate user requests a console URL, logs into the Guest OS, and then someone else copies that exact URL (with the token) to another browser/machine, they find themselves already logged into the session without needing any credentials. | |
| 13:12:59 | gokhan | Is there a way to enforce "One-Time Use" for console tokens, so that once a WebSocket connection is established, the token becomes invalid for any subsequent connection attempts? | |
| 13:13:05 | sean-k-mooney | gokhan: so the mitigation for that is intened to be using ssl | |
| 13:13:30 | sean-k-mooney | i.e. so that the console token cannot eb intersepted over the wire | |
| 13:13:45 | sean-k-mooney | if your worred about client side hijacts in the browser then that diffent | |
| 13:13:54 | sean-k-mooney | and now we do not have the consept of one time use tokens | |
| 13:14:49 | sean-k-mooney | one time use tokes woudl break browser reresh or when you pop the console out into its own tab in horizong but it could eb something we consider supportign eventually | |
| 13:15:31 | sean-k-mooney | it would not be soemthign we could enabel by default IMO based on the end user ux regression it woudl bring | |
| 13:40:59 | opendevreview | sean mooney proposed openstack/os-vif master: Add TAP device pre-creation support for OVS/OVN https://review.opendev.org/c/openstack/os-vif/+/971231 | |
| 14:48:50 | tkajinam | stephenfin, hi do you mind voting +2+A to https://review.opendev.org/c/openstack/osc-placement/+/970058 again ? | |
| 14:49:13 | tkajinam | previous review scores were wiped when I fixed that depends-on link ... | |
| 14:59:49 | tkajinam | stephenfin, oh, gibi++ already approved it :-) | |
| 15:03:57 | opendevreview | Merged openstack/osc-placement master: Add Python 3.13 functional tests https://review.opendev.org/c/openstack/osc-placement/+/970058 | |
| 15:39:25 | gokhan | token if they originate from a different Source IP. | |
| 15:39:25 | gokhan | thanks sean-k-mooney I am worried about client side hijacks in the browser. Now I am considering a workaround at the Load Balancer (HAProxy) level to mitigate the risk of URL sharing/theft. I am trying to implement IP-to-Token Pinning" using HAProxy stick-tables. Catch the token parameter from the VNC URL.Store the Source IP of the first requester for that specific token in a stick-table. Reject (403 Forbidden) any subsequent requests for the same | |
| 16:07:51 | sean-k-mooney | gibi so for the os-vif init its implemnted here https://github.com/openstack/os-vif/blob/master/os_vif/__init__.py#L24-L49 which calls load on the plugin to regester the plugin config options but i dont see anythign that would cause the ovs conenction to be established | |
| 16:08:32 | sean-k-mooney | gibi: so that backgorund heatbeat to vos shodul not be established until the first time we call plug on a vif object that uses ovs | |
| 16:09:05 | sean-k-mooney | i.e. after the service is fully created so it shoudl not be a factor in the service creation workflow | |
| 16:10:58 | sean-k-mooney | i lazy load teh ovs db connection on first use here https://github.com/openstack/os-vif/blob/master/vif_plug_ovs/ovsdb/ovsdb_lib.py#L35-L44 | |
| 16:41:04 | gibi | sean-k-mooney: cool. thanks for checking it | |
| 17:10:30 | opendevreview | sean mooney proposed openstack/nova master: Support os-vif TAP pre-creation for OVS/OVN ports https://review.opendev.org/c/openstack/nova/+/973414 | |
| 17:11:55 | opendevreview | sean mooney proposed openstack/nova master: Support os-vif TAP pre-creation for OVS/OVN ports https://review.opendev.org/c/openstack/nova/+/973149 | |
| 17:14:41 | sean-k-mooney | ralonsoh: ^ should work with your chagnes now | |
| 17:15:04 | sean-k-mooney | ignore 973414 i abandoed that the change id got messed up | |
| 18:27:34 | ralonsoh | sean-k-mooney, I'll recheck the testing patch again | |
| 20:47:15 | opendevreview | sean mooney proposed openstack/nova master: FairLockGuard: Support cross-thread sharing and nesting https://review.opendev.org/c/openstack/nova/+/973438 | |