Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-05-29
16:58:33 sean-k-mooney gmaan: stephenfin has two small spec related to the api that you may have interest in https://review.opendev.org/c/openstack/nova-specs/+/940440 and https://review.opendev.org/c/openstack/nova-specs/+/947210
16:58:54 sean-k-mooney the latter came form a code review as part of the openapi schema work
17:00:23 gmaan sure, will check today
18:03:52 gmaan sean-k-mooney: replied for the evacuate policy change, that is something we should discuss more. especially if that can be used to know infra/service info.
18:03:57 gmaan gibi: when you have time, can you provide your opinion on evacuate policy default change (specially on Does it leak information about infra/services? part) https://review.opendev.org/c/openstack/nova-specs/+/937650/7/specs/2025.2/approved/policy-manager-role-default.rst#127
18:04:41 gmaan we do return 400 if evacuate is tried and compute is up but host info are returned in error message
18:11:53 sean-k-mooney gmaan: i dont think we would return a 400
18:12:23 sean-k-mooney for a normal user they will get a 403 or 401 but an admin i would expect to get a 409
18:12:27 gmaan 409?
18:12:36 sean-k-mooney if the host was up ya
18:12:41 sean-k-mooney it might be a 400
18:12:51 gmaan yeah, that is something I was thikning why we did not return 409 in this case
18:12:58 sean-k-mooney i.e. it may be one of the api we missed when we treid to use 409 better
18:13:07 gmaan yeah
18:14:39 sean-k-mooney gmaan: is the concern that tehy will knwo the mangment of the host is still up?
18:15:01 sean-k-mooney like if they try to evacuate and the nova-compute is up
18:15:10 gmaan sean-k-mooney: that one + current code return instance' host in the error message which we can change
18:15:18 sean-k-mooney then they could just cold/live migratiate instead right?
18:15:53 sean-k-mooney gmaan: oh right i didnt look at the respocne
18:16:24 gmaan they can migrate other way but it is that they will have a way to know compute service is up
18:17:08 gmaan which they know indirectly because instance is up :) that is why I am not 100% sure if this is infra info leak or not
18:17:12 sean-k-mooney well they can infer but doint instnace actions on the instnace
18:17:18 sean-k-mooney like server diagnostics
18:17:30 gmaan that is admin only
18:17:41 sean-k-mooney is it?
18:17:48 sean-k-mooney ok but its valid
18:17:58 gmaan I do not think manager will know host/info from any API
18:18:09 sean-k-mooney we need too decide how mcuh awareness of this info is ok for the manager role to have
18:18:26 gmaan yeah
18:18:30 sean-k-mooney right i dont think they should eitehr
18:18:56 gmaan that is evacuate is little tricky because it depends on service availability checks
18:19:08 sean-k-mooney yep
18:19:50 gmaan now, I do not have any strong opinion to open it for manager at least until someone have use case and ask us to do
18:19:52 sean-k-mooney by the way even if diagnostics was admin only console log show or hardreboot
18:20:11 sean-k-mooney can all indreictlly be used to prob if the instance is still managable
18:21:00 gmaan reboot is member-or-admin https://github.com/openstack/nova/blob/221a3e89e8988bc664298106ee691a4e41ca71f9/nova/policies/servers.py#L376
18:21:38 gmaan console log too https://github.com/openstack/nova/blob/221a3e89e8988bc664298106ee691a4e41ca71f9/nova/policies/console_output.py#L27
18:22:02 sean-k-mooney i expected https://docs.openstack.org/api-ref/compute/#show-console-output-os-getconsoleoutput-action to be reader
18:23:02 sean-k-mooney gmaan: im less concered with the service up vs leaking hostname/fqdn
18:23:23 sean-k-mooney we could defer the evacuate change for now or see what gibi and other thing before updating the spec
18:23:34 gmaan yeah, sounds good.
18:24:48 sean-k-mooney the calculus im doing is i dont know waht it gets you if you knwo nova-compute on a host is not up in general. the fqdn can tell you a lot about a host if your a bad actor
18:25:20 sean-k-mooney public cloud still may not want to expose that but since noone will have the manager role by default
18:25:33 gmaan yeah
18:25:34 sean-k-mooney its kind of a decison they can make
20:51:25 opendevreview Merged openstack/nova-specs master: Add remove-os-volumes_boot-api spec https://review.opendev.org/c/openstack/nova-specs/+/947210
21:13:09 opendevreview Merged openstack/nova master: libvirt: Use common naming convention for ephemeral disk labels https://review.opendev.org/c/openstack/nova/+/947541
23:52:59 opendevreview Merged openstack/nova master: api: Adjust validation helpers for a single-method future https://review.opendev.org/c/openstack/nova/+/936365
#openstack-nova - 2025-05-30
01:09:50 opendevreview Takashi Kajinami proposed openstack/nova master: Replace license classifier https://review.opendev.org/c/openstack/nova/+/951226
01:12:57 opendevreview sean mooney proposed openstack/nova-specs master: Add cpu-teiring with cpu_shares spec https://review.opendev.org/c/openstack/nova-specs/+/951222
01:13:31 opendevreview sean mooney proposed openstack/nova-specs master: Add cpu-teiring with cpu_shares spec https://review.opendev.org/c/openstack/nova-specs/+/951222
07:03:26 gibi gmaan: sean-k-mooney: I replied about evacuate in https://review.opendev.org/c/openstack/nova-specs/+/937650/comment/aaf66b8d_53e159c3/
08:58:27 zigo Under Trixie, with RabbitMQ 4.0.5, I'm getting:
08:58:27 zigo PRECONDITION_FAILED - invalid property 'non-durable' for queue 'conductor' in vhost '/'
08:58:38 zigo Is this a miss-feature somewhere ?!?
08:58:48 zigo I mean, in oslo.messaging ...
10:28:39 sean-k-mooney zigo: not that im aware of but i belive kolla hit some issues with rabbit 4.x and i belive they also use debain
10:30:26 sean-k-mooney zigo: all our testing for expoy and master is still useing 3.12.1
10:30:53 sean-k-mooney zigo: that what ubuntu 24.04 is shipping currently
10:34:51 sean-k-mooney zigo: im not sure if oslo.messaging offically supprot rabbit 4.X
10:35:35 sean-k-mooney you may be able to work around the issue by changign the cofnig default in service like chaging to quorm_queues
10:36:53 sean-k-mooney zigo: i dont know if the idea of having a debian testing image has ever been raised with infra or what the eta is for trixie. we currently use debain for spice testign in the nova-ovs-hybrid-plug job
10:37:11 sean-k-mooney i would be happy to see that move to trixe or sid in the future once its aviable
10:37:34 sean-k-mooney although the main blocker is likely to be python 3.13 support
10:39:08 frickler so far we would have to wait for trixie to be released to build an image with nodepool. with the zuul-launcher feature we might be able to start earlier, but I didn't have time yet to look into this
10:40:52 frickler also, yes, the version of oslo.messaging will likely be relevant. I think this should all be working in kolla, but maybe our CI coverage is incomplete
10:44:26 sean-k-mooney frickler: im wondering if the debian installer and kolla are using slightly diffent config options for the messaging transport
10:45:04 sean-k-mooney frickler: for example ha queue have been deprecated for a long time they may have finally killed them in 4?
10:46:44 frickler sean-k-mooney: yes, there was a lot of cleanup work to be done in the last cycle, but kolla does ship with rmq 4 for 2025.1
10:47:49 sean-k-mooney that at least promising as it implies there is a workable config that zigo could move too
10:56:00 zigo sean-k-mooney: I did move to quorum queues, no choice with RabbitMQ 4.0.5, as it doesn't support HA queues anymore. Though that's not the issue. The issue is nova attempting to set not-durable, which seems to be gone from Rabbit.
10:57:19 zigo I wonder how can I tell oslo.messaging not to do that.
10:57:30 sean-k-mooney zigo: have you treied setting the durable_queses config option to true
10:57:33 zigo I did grep not-durable in oslo.messaging and didn't find where it is.
10:57:54 sean-k-mooney oslo is ment to ignore it when using quorm_queues which requrie durablity
10:57:57 zigo sean-k-mooney: This creates more issues that it solves, I wouldn't recommend to anyone to do that.
10:58:30 sean-k-mooney https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/amqp.py#L33-L37
10:58:55 sean-k-mooney zigo: quourm queue are unconditonally durable in rabbit
10:59:05 zigo Oh ok.
10:59:15 sean-k-mooney https://www.rabbitmq.com/docs/quorum-queues#feature-matrix
10:59:19 zigo Let me try something then.
11:00:02 sean-k-mooney so oslo should be ignoreing that config option but it may be bugged or it may be fixed but in a newer verion
11:00:35 zigo I'll try setting durable_queue to true and see how it goes.
11:00:50 zigo Though as much as I can tell, this doesn't fix.
11:00:55 zigo Let me try.
11:01:49 zigo Same thing.
11:01:57 zigo I'm getting:
11:01:57 zigo Queue.declare: (406) PRECONDITION_FAILED - invalid property 'non-durable' for queue 'conductor' in vhost '/'
11:02:45 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: trying to figure out what broke the unit test https://review.opendev.org/c/openstack/nova/+/951393
11:06:36 opendevreview Balazs Gibizer proposed openstack/nova master: [hacking] N374 do not use time.sleep(0) to yield https://review.opendev.org/c/openstack/nova/+/950992
11:08:14 zigo sean-k-mooney: Any idea what's attempting to add non-durable and how one does that in Rabbit?
11:11:10 opendevreview Balazs Gibizer proposed openstack/nova master: Cache [pci]alias parsing https://review.opendev.org/c/openstack/nova/+/951151
11:44:13 sean-k-mooney zigo: not off the top of my head sorry
11:56:06 frickler zigo: did you set rabbit_quorum_queue and rabbit_transient_quorum_queue?
11:56:21 zigo I did.
11:56:29 zigo Both to true.
11:57:16 zigo frickler: If I comment this line:
11:57:16 zigo https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_rabbit.py#L447
11:57:16 zigo then daemons car start ...
11:57:46 frickler for reference here's the full template that kolla uses, but maybe some rmq config is also relevant https://opendev.org/openstack/kolla-ansible/src/branch/master/ansible/roles/aodh/templates/aodh.conf.j2#L63-L78

Earlier   Later