| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-12-11 | |||
| 16:46:30 | noonedeadpunk | we have adapted them quite some time ago as well, because of some bug in nova-manage - when a command with different password was just creating another cell. So it was breaking nova on password rotation process | |
| 16:46:41 | noonedeadpunk | (I bet I reported it) | |
| 16:47:34 | noonedeadpunk | as then you also need to properly time update of the cell with service restart.... | |
| 16:49:00 | noonedeadpunk | https://bugs.launchpad.net/nova/+bug/1923899 | |
| 16:49:17 | noonedeadpunk | it's still in review :D | |
| 16:49:44 | sean-k-mooney | you should not return the create cell command | |
| 16:49:55 | sean-k-mooney | that is not for updating the cell | |
| 16:50:03 | sean-k-mooney | it sfor creating a new one | |
| 16:50:37 | sean-k-mooney | if you want ot update use the update command https://docs.openstack.org/nova/latest/cli/nova-manage.html#cell-v2-update-cell | |
| 16:51:01 | sean-k-mooney | the name is for human uses only its not the uniuqe identifyer | |
| 16:52:10 | sean-k-mooney | this is somewhere between user error and a missign unique constratit | |
| 16:52:33 | sean-k-mooney | interally nova only uses the cell uuid for joins | |
| 16:52:59 | noonedeadpunk | I Guess it's kidna general expectation of name_or_uuid behavior as well... | |
| 16:53:16 | sean-k-mooney | right that not the documented behavior | |
| 16:53:24 | sean-k-mooney | and we dont support that for nova-mange normally | |
| 16:53:29 | noonedeadpunk | as it's kinda hard in automated tooling to reference by uuid, but yeah, I get the point | |
| 16:53:30 | sean-k-mooney | that a openstack client thing | |
| 16:53:48 | sean-k-mooney | not somethng that the api or project -manage command typeicly support | |
| 16:54:44 | sean-k-mooney | the propsed path is not actully a fix | |
| 16:54:58 | sean-k-mooney | its a nova status commadn to detech cells with the same name | |
| 16:55:19 | sean-k-mooney | it does tno prevnet it from happeign or actully adress te bug | |
| 16:55:40 | sean-k-mooney | well there are two https://review.opendev.org/c/openstack/nova/+/901810 | |
| 16:55:49 | sean-k-mooney | is the nova-status check | |
| 16:56:39 | noonedeadpunk | it could be helpful as well, but yeah... | |
| 16:56:40 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/876940 changed the unique constrating but that has upgrade impacts | |
| 16:59:16 | noonedeadpunk | how the heck notifications are even supposed to work in multi-cell envs.... | |
| 16:59:52 | noonedeadpunk | or well. you set yet another rabbitmq cluster for notifications ,and all services can report to it.... | |
| 16:59:57 | noonedeadpunk | I guess that's the way.... | |
| 17:00:08 | sean-k-mooney | well you are recommened to use a dedicated rabbit for notrication regardless of cells | |
| 17:00:52 | sean-k-mooney | so it does not really change that you eithe ruse one rabbit for noticaion for all cells or you need to have 1 per cell and listnet to muptle notificions busses in yoru applcations | |
| 17:01:00 | noonedeadpunk | and then they are not scaled with cells, simple as that I guerss | |
| 17:01:22 | noonedeadpunk | oh, right... or that | |
| 17:01:26 | sean-k-mooney | it depend i belive celipmenter supprots a list of notificaon transport urls | |
| 17:01:48 | sean-k-mooney | but its more commont to just use 1 dedicated rabbit for them | |
| 17:02:46 | sean-k-mooney | but again that not really difent then a normal deployment where the recommadnation has alwasy beend seperate rpc and notficaiton traffic to diffent rabbit instances | |
| 17:03:58 | noonedeadpunk | yes, true, I was just thinking of implications of cells, like firewalling, etc, where ceilometer should be also allowed to get connected to these cell-specific notification clusters or whatever... | |
| 17:04:30 | noonedeadpunk | it somehow feels to have an enomorous amount of various implications for deployments | |
| 17:21:40 | sean-k-mooney | no more so then condotor groups or shard in ironic | |
| 17:21:54 | sean-k-mooney | but cells are an architecural choice | |
| 17:22:44 | sean-k-mooney | cells are very diffent then AZs AZ are just names on an aggrete | |
| 17:23:19 | sean-k-mooney | cells is about how you architect yoru clodu to scale and deserves the same tought like l3 routed netowrk for neutron | |
| 17:30:14 | melwitt | sean-k-mooney: I don't think those are duplicates fwiw. https://bugs.launchpad.net/nova/+bug/2132020 is debatable as a bug bc instances in ERROR state do still consume resources (from a quota perspective) until they are deleted. I don't have a 100% opinion on that without thinking about it more | |
| 17:31:40 | noonedeadpunk | I am jsut frankly concerned that by the time when you might get up to a limit of rabbitmq/mariadb for nova, you will have a more bottleneck at other place | |
| 17:31:53 | noonedeadpunk | (like neutron db) | |
| 17:33:22 | JayF | I mean, yeah. Ironic has been dealing with problems shaped like that for years, with Nova computes being the thing we outscale | |
| 17:33:29 | JayF | and we figured it out (shards and conductor groups) | |
| 17:35:35 | noonedeadpunk | and cell_mappings in nova_api is pretty much used only by superconductor, right? So when cell mapping is not templated, but you need to update password, you pretty much caring about config change and restart of superconductor with cell mapping change? | |
| 17:39:10 | noonedeadpunk | and it feels like with rabbitmq quorum queues and using streams for fanouts, rabbit become more horizontally scalable then it used to be... but yeah... | |
| 17:43:12 | sean-k-mooney | melwitt: they are not entirly but i think they are related | |
| 17:44:12 | sean-k-mooney | melwitt: https://bugs.launchpad.net/nova/+bug/2132020 depend o nthe way the error happened. | |
| 17:44:29 | sean-k-mooney | basiclly if we have berried the isntance in cell 0 because of a no valid host | |
| 17:44:58 | sean-k-mooney | then we shoudl free any placment allcation on comptue nodes for it | |
| 17:45:08 | sean-k-mooney | the same way we do when you shelve offload | |
| 17:45:24 | sean-k-mooney | if the isntance is in error but insthace.host is pointing at a real host | |
| 17:45:28 | melwitt | I'm saying it's not 100% clear whether that's consistent bc currently any instance that has an instance record consumes quota | |
| 17:45:28 | sean-k-mooney | then we need to keep the allcation | |
| 17:45:38 | sean-k-mooney | because in thiory you coudl rebuild ro hard reboot the vm | |
| 17:46:03 | sean-k-mooney | we do the instnace count check sepreaty form the palcmenet allcation | |
| 17:46:26 | sean-k-mooney | i think the orgianll assution is the qutota violation is not for the instnace quota | |
| 17:48:40 | melwitt | yeah, I dunno. it is just not 100% clear to me what to do to make sure things are consistent | |
| 17:49:39 | sean-k-mooney | i dont know if you have had time to look at the regression test https://review.opendev.org/c/openstack/nova/+/969251 | |
| 17:49:52 | sean-k-mooney | im still trying to figure out if it is valid base don that for waht its worht | |
| 17:50:13 | sean-k-mooney | the two just seam realted to me and i wanted to make sure we were not tryint to solve the same thing in two diffent ways | |
| 17:50:45 | melwitt | no I only skimmed | |
| 17:51:09 | sean-k-mooney | the repoducer has not conviced me its an actula bug yet for what its worth | |
| 17:53:06 | sean-k-mooney | it looks like they actully might be using the logic form https://opendev.org/openstack/nova/src/branch/master/nova/tests/functional/regressions/test_bug_1806064.py#L92 | |
| 17:53:19 | sean-k-mooney | to simulate this so maybe it is the num isntance quota | |
| 17:53:45 | sean-k-mooney | to me if we exceed quota but are not shcduled to a host as a restl of it we shoudl not eb preventign other instances form using that space | |
| 17:54:04 | sean-k-mooney | but we obviously cant delete the allcaiton if there si any change of htis vm being recovered | |
| 17:54:25 | sean-k-mooney | i.e. with reset state or rebuidl or whatever | |
| 17:54:49 | melwitt | yeah | |
| 17:56:27 | melwitt | I see the point about not preventing other instances from building. I was thinking about it focused on just the instance but thinking about it focused on the compute hosts I see the issue | |
| 17:58:38 | melwitt | superficially thinking about it I think each fix does have to be in a different place (conductor vs compute/api) because the allocation creation and volume reservation happen at different places | |
| 17:59:53 | melwitt | if you exceed quota in compute/api and a reserved volume was created, you have to clean it up there and you won't have any allocations yet | |
| 18:00:42 | melwitt | I'm actually not sure how quota is getting exceeded in conductor, if it is, then that would be the "quota recheck" step after the compute/api quota check passed | |
| 18:01:32 | melwitt | which is valid of course. just also highlighting how each bug is about a different point in the server create path | |
| 18:06:29 | opendevreview | Jay Faulkner proposed openstack/nova master: [ironic] Use constants from Ironic, test w/ddt https://review.opendev.org/c/openstack/nova/+/969321 | |
| 18:19:12 | JayF | https://review.opendev.org/c/openstack/nova/+/969321 is a follow-up to the Ironic bugfix, I did some changes in Ironic's side so we could just copy over the file that becomes ironic_states.py in our driver, to help make it easier to keep states synced. Also updated the tests (as requested on the previous patch) to use ddt. | |
| 18:19:33 | JayF | No real urgency behind landing it, but I would like to get it in since I did the refactor on Ironic's side to make that constants file able to be copied around | |
| 18:55:25 | sean-k-mooney | melwitt: it can only execced in the conductor if 2 api request happen to diffent api isntance at almost the same time | |
| 18:55:35 | sean-k-mooney | so it rare but its possibel | |
| 18:56:05 | sean-k-mooney | well that or if an amdin change the quota while the request is in flight | |
| 18:56:21 | sean-k-mooney | i.e. after its pass the api | |
| 18:59:02 | melwitt | right | |
| 20:51:46 | opendevreview | Merged openstack/nova master: Drop direct dependency on iso8601 https://review.opendev.org/c/openstack/nova/+/943083 | |
| #openstack-nova - 2025-12-12 | |||
| 09:13:19 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: create generic check for mem encryption support by host https://review.opendev.org/c/openstack/nova/+/967969 | |
| 09:13:20 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: stop using _get_mem_encryption_config() for SEV checks https://review.opendev.org/c/openstack/nova/+/967970 | |
| 09:13:20 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: remove explicit SEV dependency from nova.scheduler.utils https://review.opendev.org/c/openstack/nova/+/967971 | |
| 09:13:21 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: make RP creation independent of specific encryption models https://review.opendev.org/c/openstack/nova/+/967972 | |
| 09:13:21 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: refactor _guest_configure_mem_encryption() for extensibility https://review.opendev.org/c/openstack/nova/+/967973 | |
| 09:13:22 | opendevreview | Taketani Ryo proposed openstack/nova master: mem-enc: adjust requirement checks for mem_encryption guests https://review.opendev.org/c/openstack/nova/+/967974 | |
| 10:32:16 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |
| 10:32:16 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron subnets https://review.opendev.org/c/openstack/nova/+/962190 | |
| 10:32:17 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron extensions https://review.opendev.org/c/openstack/nova/+/962270 | |
| 10:32:17 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron floating IPs https://review.opendev.org/c/openstack/nova/+/962604 | |
| 11:29:30 | stephenfin | sean-k-mooney: gmaan: Do you think you'd be able to get to the outstanding openapi patches this side of new year? I'd *really* like to be able to close that one off in 2025 | |
| 11:29:48 | stephenfin | as always, I am happy to trade reviews | |
| 15:04:57 | nicolairuckel | tkajinam: I took a look at your patch and you put a todo there for clearing the NVRAM in certain situations. Do you plan on implementing that in that patch too? | |
| 15:05:37 | nicolairuckel | I'm asking because if you'd implement that already, I can just focus on copying the NVRAM and if my patch gets merged after yours, there shouldn't be any problems. | |
| 16:20:46 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron networks https://review.opendev.org/c/openstack/nova/+/928022 | |