| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-07-16 | |||
| 16:21:15 | sean-k-mooney | that breaks fips compliance but it might eb a better option then a downstream code change if its somethign that was accpable form a secuirty perspective until the issue is fixed properly | |
| 16:21:21 | noonedeadpunk | I think I will try it, as all other options are kinda not perfect | |
| 16:21:50 | noonedeadpunk | I don't want tunneled migrations as we are have local storages here and there | |
| 16:22:09 | noonedeadpunk | and parallel and compress both not solving issue but making it more distant | |
| 16:22:25 | noonedeadpunk | so it's a question of how big and loaded VM should be | |
| 16:23:38 | sean-k-mooney | well as someone who prefers the simple life of migratin unlooed cirros iamges in my devstack small an light :) but customer want to run real workloads, how unreasonabel :) | |
| 16:23:42 | noonedeadpunk | It's also a question if there could be more side-effects from disabling aes | |
| 16:24:22 | sean-k-mooney | so there are some, if you dont use aes and use one of the other algroatims you may not be able to leverage aes-ni or similer exteions to offload the encyprtion to hardware | |
| 16:24:49 | noonedeadpunk | yeah, and we start talking if customer moved from VMWare with their 1Tb MSSQL with 200gb of ram... | |
| 16:24:52 | sean-k-mooney | the side effect of that would be a bottelneck on througput | |
| 16:25:21 | noonedeadpunk | yeah, but there's also like OVN on host likely using AES as well | |
| 16:25:32 | sean-k-mooney | i remember jay pipes talking about one of the customers mirantas had wehre they litrally ran 1 VM per compute node that used all the cpus/ram | |
| 16:25:38 | noonedeadpunk | or well, ovs | |
| 16:26:22 | sean-k-mooney | the litrally just wanted to have an api to create/delete vms effectivly and provide some levle of security via a vm absrction | |
| 16:26:29 | noonedeadpunk | I wonder how scheduling work for them if they need to evacuate the VM on hypervisor failure | |
| 16:26:47 | sean-k-mooney | hopefully they had hot spares | |
| 16:27:04 | sean-k-mooney | as in free hosts | |
| 16:27:18 | sean-k-mooney | so i beliv ethis dropin file is specific to qemu | |
| 16:27:34 | sean-k-mooney | i.e. its createing /etc/crypto-policies/local.d/gnutls-qemu.config | |
| 16:27:48 | sean-k-mooney | but it woudl apply to all tls coonection use by qemu fro anything it doing | |
| 16:28:07 | sean-k-mooney | but again im not an experit on this so that may be incorrect | |
| 16:28:14 | noonedeadpunk | well, this would be fine then | |
| 16:28:42 | noonedeadpunk | I was a bit more, if crypto-policies just do fileglob of /etc/crypto-policies/local.d/*.config | |
| 16:28:50 | noonedeadpunk | but will check on that | |
| 16:29:04 | sean-k-mooney | i think that is what the QEMU= at the start of it is doing not the file name | |
| 16:29:11 | sean-k-mooney | but again i have not looked into this deaply at all | |
| 16:30:04 | noonedeadpunk | actually, I'd argue that using CHACHA-POLY1305 should be faster then AES | |
| 16:32:17 | sean-k-mooney | it depend aes on sufficlty new enough intel chips will be done in dedicated hardware espiclly if you have access to qat acclerators | |
| 16:32:46 | sean-k-mooney | but it might be faster and you will really only knwo once your try it on your hardware | |
| 16:36:59 | noonedeadpunk | I tottally need to sleep over with all this info | |
| 17:19:22 | opendevreview | Merged openstack/nova master: FUP: Translate scatter-gather to futurist https://review.opendev.org/c/openstack/nova/+/953338 | |
| 17:19:33 | opendevreview | Merged openstack/nova master: Add Project Manager role context in unit tests https://review.opendev.org/c/openstack/nova/+/941056 | |
| #openstack-nova - 2025-07-17 | |||
| 08:50:05 | ralonsoh | Hi folks, qq about tempest | |
| 08:50:11 | ralonsoh | I've pushed a patch for devstack | |
| 08:50:19 | ralonsoh | https://review.opendev.org/c/openstack/devstack/+/955187 | |
| 08:51:04 | ralonsoh | And tempest-full-py3, devstack-platform-ubuntu-jammy or nova-ceph-multistore are failing in the same 2 tests | |
| 08:51:26 | ralonsoh | --> https://zuul.opendev.org/t/openstack/build/61d7370a71494a2192ee60cfc4eb6cc3/logs | |
| 09:25:29 | ralonsoh | gibi, ^ | |
| 09:29:27 | ralonsoh | Ehhhh sorry. Sorry for the noise: because of this patch (^ the one that triggered this log), we are not deploying the OVN agent | |
| 09:29:36 | ralonsoh | please, disregard my comments... | |
| 10:40:28 | stephenfin | gmaan: sean-k-mooney: If you have time today, could you look at https://review.opendev.org/c/openstack/nova/+/952284/ The patches above it are in merge conflict and will need a rebase, but it'd be good get everything below that in first | |
| 10:43:04 | opendevreview | Stephen Finucane proposed openstack/nova master: Migrate mypy configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953703 | |
| 10:43:05 | opendevreview | Stephen Finucane proposed openstack/nova master: Migrate codespell configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953704 | |
| 10:43:06 | opendevreview | Stephen Finucane proposed openstack/nova master: Migrate setup configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953705 | |
| 10:43:06 | opendevreview | Stephen Finucane proposed openstack/nova master: setup: Remove pbr's wsgi_scripts https://review.opendev.org/c/openstack/nova/+/902688 | |
| 10:44:59 | sean-k-mooney | stephenfin: sure | |
| 11:19:44 | gibi | ralonsoh: no worries :) | |
| 14:12:45 | opendevreview | Kamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event https://review.opendev.org/c/openstack/nova/+/949754 | |
| 15:21:36 | gmaan | stephenfin: ack, not sure if I can do today and I am on PTO tomorrow. But I will do early next week. not just this but the remaining series | |
| 15:22:19 | stephenfin | nw. doesn't look like they will merge anytime soon. The gate is super unstable :( | |
| 15:23:10 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.1: Add ServersViewBuilderTestV296 unit test class https://review.opendev.org/c/openstack/nova/+/955304 | |
| 15:23:11 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.1: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/955305 | |
| 16:17:59 | opendevreview | Merged openstack/nova master: Move ConductorManager to use spawn_on https://review.opendev.org/c/openstack/nova/+/948187 | |
| 16:29:25 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.2: Add ServersViewBuilderTestV296 unit test class https://review.opendev.org/c/openstack/nova/+/955309 | |
| 16:29:26 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.2: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/955310 | |
| #openstack-nova - 2025-07-18 | |||
| 04:24:07 | aravindh5murugesan | Hello folks. I'm running into a problem when I set CPU models to custom and the specifying the models. The default config that we receive from our vendor sets the cpu-model to broadwell-notsx-IBS something. But our CPU model is saphirerapids and the oldest we need to support is Icelake. But When I set it to Icelake or SaphireRapids, I used the same name as listed in virsh cpu-models, but nova complains that CPU | |
| 04:27:37 | aravindh5murugesan | can assist me here. 2025-07-17 20:44:36.072 ERROR oslo_service.service [req-fa1ce5c0-d687-4d09-ac10-7ca05eb98461 None None] Error starting thread.: nova.exception.InvalidCPUInfo: Configured CPU model: SapphireRapids-noTSX and CPU Flags [] ar not compatible with host CPU. Please correct your config and try again. Unacceptable CPU info: CPU doesn't have compatibility | |
| 04:28:08 | aravindh5murugesan | CPU model in hypervisor - Vendor ID: GenuineIntel Model name: Intel(R) Xeon(R) Silver 4416+ | |
| 09:09:51 | sean-k-mooney | aravindh5murugesan the problem you are having is that icelake has feature flags like tsk tha tintel disabled in microcode and does tno supprot on saphire rapids | |
| 09:11:11 | sean-k-mooney | *like the flags for tsx that intel ... | |
| 10:53:08 | opendevreview | Takashi Kajinami proposed openstack/nova master: Drop unused parameter type validation method https://review.opendev.org/c/openstack/nova/+/909189 | |
| 12:03:33 | aravindh0murugesan | sean-k-mooney: In that case, would it possible to disable those unsupported flags and somehow make this work? | |
| 12:20:19 | sean-k-mooney | yes there are 2 ways, libvirt provide some custom models for older cpu i.e. the *-notsx-IBS variants | |
| 12:20:36 | sean-k-mooney | but nova also has a config option that allows you to add/remove cpu flags | |
| 12:37:05 | aravindh0murugesan | I compared the icelake.xml and my lscpu flags and found these flags are missing in my host cpu - avx512vnni,mpx,pclmuldq,spec-ctrl,sse4.1,sse4.2,tsc-deadline, | |
| 12:37:18 | aravindh0murugesan | so I just do a - on these flags in nova configs? | |
| 12:38:45 | aravindh0murugesan | cpu_models = Cascadelake-Server-noTSX cpu-model-extra-flags = -avx512vnni,-mpx,-pclmuldq,-spec-ctrl,-sse4.1,-sse4.2,-tsc-deadline this does not work either | |
| 12:44:45 | aravindh0murugesan | sean-k-mooney: Editing the XML to say removed='yes' kinda worked. But I'm not sure if this is the suggested way to do things. | |
| 12:58:38 | aravindh0murugesan | My bad, changing this cpu-model-extra-flags to snake case fixed it. Thanks. | |
| 13:23:40 | sean-k-mooney | ack so you were able to get this wokring | |
| 13:25:27 | sean-k-mooney | and yes https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cpu_model_extra_flags is the cofnig opiton | |
| 13:26:13 | sean-k-mooney | by the wya i have alwasy know cpu_model_extra_flags as SLUG case and cpu-model-extra-flags as snake case | |
| 13:28:24 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.2: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/955310 | |
| 13:28:39 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.1: Add ServersViewBuilderTestV296 unit test class https://review.opendev.org/c/openstack/nova/+/955304 | |
| 13:28:40 | opendevreview | Wesley Hershberger proposed openstack/nova stable/2024.1: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/955305 | |
| 14:31:42 | dansmith | gibi: so, with my patch to actually fix s-g, I'm hitting the db poison fixture on NoDBTestCase tests.. I was thinking that I must be working around one of the mocks for that, but I haven't found it yet | |
| 14:32:26 | dansmith | I'm wondering if maybe before we were scheduling things to threads but not actually running them and these tests are actually missing db mocks that are now getting tickled | |
| 14:32:32 | dansmith | have you run into that anywhere else? | |
| 15:08:18 | gibi | I havent seen this type of issues yet with tasks using the futurists executors especially not in legacy mode spawning GreenThreads. | |
| 15:09:31 | gibi | I can take a look at the actual failure on Monday | |
| 15:10:02 | dansmith | gibi: wait, I think I'm just stupid | |
| 15:10:24 | dansmith | I was raising the exception the task raised in my result() but I think our contract with the sg caller is to return the Exception if it raised | |
| 15:11:29 | gibi | yes I think scatter gather expects the exception to be returned | |
| 15:11:55 | gibi | https://github.com/openstack/nova/blob/770a6f1eabc6cfb77d3ac5afd91e6e8ce9f8b9d9/nova/context.py#L425 | |
| 15:12:16 | gibi | https://github.com/openstack/nova/blob/770a6f1eabc6cfb77d3ac5afd91e6e8ce9f8b9d9/nova/context.py#L463 | |
| 15:12:34 | gibi | so yes the exception caught on the task thread and returned az the result value of the task | |
| 15:14:26 | dansmith | yup | |
| 15:23:32 | opendevreview | Merged openstack/nova master: Make nova.utils.pass_context private https://review.opendev.org/c/openstack/nova/+/948188 | |
| 15:29:47 | opendevreview | Dan Smith proposed openstack/nova master: Remove eventlet timer from multi_cell_list https://review.opendev.org/c/openstack/nova/+/954990 | |
| 15:29:47 | opendevreview | Dan Smith proposed openstack/nova master: WIP: Parallelize s-g generators https://review.opendev.org/c/openstack/nova/+/955091 | |
| 15:30:16 | dansmith | that ^ should fix the unit/fixture failure on the eventlet timer patch and get the WIP much closer to passing.. more work to be done there of course, but at least it's not 100% fail | |
| #openstack-nova - 2025-07-19 | |||
| 03:11:10 | aravindh1murugesan | sean-k-mooney[m]: We are doing a POC with lightbits nvme/TCP. I get about 1.5 million IOPS/s on hypervisors directly, but on the VM running in the same hypervisor, its hardly touching 400k. | |
| 03:12:54 | aravindh1murugesan | Lightbits suggestes adding iothreads to libvirt domain XML and dedicating some to the disk. But seems like a hack to edit domain XML, I would to know if there is a way to set this from Nova, or any plans to add scalable IO threads to the VM? | |
| 10:20:11 | sean-k-mooney[m] | aravindh1murugesan: there are plans to add iothread support, the desigin missed the spec freeze for this release so we will hopefully have them next release 2026.1 | |
| 10:21:31 | sean-k-mooney[m] | for now the best you can do is use a dedicated emulator thread and test with hw_disk_bus=scisi with hw_scsi_model=virtio | |
| 10:23:50 | sean-k-mooney[m] | aravindh1murugesan: im not sure how familar you are with nova but the libvirt xml is and internal implmenation detail and not something you are allowed to modify. nova will regenerate the xml when you reboot or move the vm so it not a viable option. lightbit should be aware of that so im not sure why they would recommend that | |
| #openstack-nova - 2025-07-21 | |||
| 07:11:48 | gibi | FYI Finally I got my ceph bug tracker account approved so I filed https://tracker.ceph.com/issues/72203 about the stack allocation error issue we saw in nova-ceph-multistore last week with v19.2.1 | |
| 11:09:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: [pci]Keep used dev in Placement regardless of dev_spec https://review.opendev.org/c/openstack/nova/+/954149 | |
| 12:01:14 | opendevreview | Kamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event https://review.opendev.org/c/openstack/nova/+/949754 | |
| 14:21:13 | opendevreview | Balazs Gibizer proposed openstack/nova master: [pci]Keep used dev in Placement regardless of dev_spec https://review.opendev.org/c/openstack/nova/+/954149 | |