| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-07-29 | |||
| 16:48:45 | sean-k-mooney | buting the isntnace to error and locking them i think is a step we coudl take to improve the situraion | |
| 16:49:16 | sean-k-mooney | but the ultimate fix would be to be able to resume the management of the migration and allow it to succeded if in fact did | |
| 16:49:51 | sean-k-mooney | i feel like an incremental approch is what we shoudl try to do but i dont know what others think | |
| 16:50:11 | sean-k-mooney | sp-bmilanov: is this somethign you woudl ahve time to work on or are you rasing it as a pain point that you think we shoudl prioritize | |
| 16:50:19 | gibi | last week we discussed that sp-bmilanov interested in the non-graceful restart case | |
| 16:50:33 | gibi | so we need something in the startup code path that prevents the original problem | |
| 16:51:36 | sp-bmilanov | gibi: yes, the ungraceful restart case is what we experienced ; sean-k-mooney pain point first, I am not sure if I can help implement something that you've triaged as so complex | |
| 16:51:37 | gibi | ie. the duplication of instances reported https://bugs.launchpad.net/nova/+bug/2092391 | |
| 16:52:43 | gibi | I believe we should be able to detect during compute startup that we have a duplication based on the fact that one of the domain is on a host that is not equal to the instance.host | |
| 16:53:12 | gibi | or based on that we have an unfinished migration | |
| 16:53:16 | gibi | in the DB | |
| 16:53:33 | sp-bmilanov | just an aside, I guess that's is not a good approach since you haven't brought it up, but why not try using a flag in the database to check if the migration is at at point where the instance is running at the destination | |
| 16:53:39 | sean-k-mooney | so the duplciation i think weill need us to do an rpc to the dest compute | |
| 16:53:59 | sean-k-mooney | and determin if we shoudl revert or run post-live-migration | |
| 16:54:02 | gibi | sp-bmilanov: I hope we have a migration state that signals that already | |
| 16:55:50 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Address issues with images APIs https://review.opendev.org/c/openstack/nova/+/956102 | |
| 16:56:37 | sean-k-mooney | sp-bmilanov: so resumeing the guest is not done by nova | |
| 16:56:40 | gibi | anyhow I think it make sense to review a spec and perpare a PTG discussion about it | |
| 16:56:43 | sean-k-mooney | its doen by libvirt | |
| 16:56:55 | sean-k-mooney | and the scoue comput enode will update the dbstate if its still runing | |
| 16:57:15 | sean-k-mooney | the problem currently is if you restart it the thread we have monitoring the libvirt job will be killed | |
| 16:57:25 | sean-k-mooney | and we wont do that update. | |
| 16:57:57 | sean-k-mooney | but on startup we can see fi the job is still runing and either resume waiting or make an rpc to see if it scuceeded. or check if the vm is running lcoally | |
| 16:58:14 | sean-k-mooney | if its runnign locally and there is no migration job active in libvirt it mean the migratoin failed | |
| 16:58:25 | sean-k-mooney | if the vm is not local it means either it crashed or it moved | |
| 16:58:38 | sean-k-mooney | so we coudl be more intelegent on startup | |
| 16:58:48 | sean-k-mooney | but someone need to spend tiem documenting this bevhioer | |
| 16:58:50 | sean-k-mooney | and writing it up | |
| 16:59:10 | sean-k-mooney | so yes i think a spec/ptg discussion woudl be good | |
| 16:59:11 | gibi | yepp I agree | |
| 17:00:58 | sp-bmilanov | s/checked from the DB/set from the dst and then checked from the src from the DB/ | |
| 17:01:32 | sean-k-mooney | well | |
| 17:01:45 | sean-k-mooney | the compute agent does not have direct db access for one | |
| 17:01:55 | sean-k-mooney | so just chekc the db state is already an rpc to teh conductor | |
| 17:02:06 | sean-k-mooney | but the responisbleity for updating the host of the vm is on the souce node | |
| 17:02:14 | sean-k-mooney | as part of post live migrate | |
| 17:02:33 | sean-k-mooney | so on start up if we need to check if tis on the dest node the source node need to call the dest to do that | |
| 17:02:52 | sean-k-mooney | i.e. asumeign the vm is not still running locally | |
| 17:03:39 | sean-k-mooney | we need to know if it crashed or if its actully active on the other host. and we can do that with only local knowlsage or by checkign the db. | |
| 17:04:50 | Uggla | We are on top of the hour. So I propose to close the meeting. ^ discusion can happen right after if needed. | |
| 17:05:06 | sp-bmilanov | thanks Uggla | |
| 17:05:20 | Uggla | Thanks all. | |
| 17:05:22 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-07-29-16.02.log.html | |
| 17:05:22 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-07-29-16.02.txt | |
| 17:05:22 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-07-29-16.02.html | |
| 17:05:22 | opendevmeet | Meeting ended Tue Jul 29 17:05:22 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 17:05:22 | Uggla | #endmeeting | |
| 17:05:42 | sp-bmilanov | sean-k-mooney right, it makes sense that the DB will be of no use if the one responsible for the book-keeping there is the source | |
| 17:06:13 | gibi | I needs to drop. Thanks folks | |
| 17:07:29 | sean-k-mooney | Uggla: 16905 tests in 84.9128 sec. with 48 thread/24 cores even at only 2.5GHz you can run the test suite pretty fast | |
| 17:07:42 | sean-k-mooney | but looking at my slow tests | |
| 17:07:44 | sean-k-mooney | nova.tests.unit.api.openstack.compute.test_schemas.SchemaTest.test_schemas 25.415 | |
| 17:07:59 | sean-k-mooney | 25 second for the schema tests seam a bit off | |
| 17:08:22 | sean-k-mooney | the other ones at the top of the list i think are ones gibi is speeding up | |
| 17:08:22 | sp-bmilanov | got to go as well, thanks sean-k-mooney, gibi! | |
| 17:09:35 | Uggla | yes the test schema seems surprisingly long | |
| 17:12:40 | sean-k-mooney | with gibi's speedup serise applied i get this https://paste.opendev.org/show/bN1SLJLLAhPmqGFIRNID/ | |
| 17:13:01 | sean-k-mooney | so its liek 9 tiems slower the the nearest other test | |
| 17:19:59 | sean-k-mooney | https://paste.opendev.org/show/brPnnHtdLgbbaEFxJbfi/ 229.4782 sec. is not too bad for all the funcitonal tests on this host but its longer then i would perfer | |
| 17:54:09 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for tenant network APIs https://review.opendev.org/c/openstack/nova/+/956088 | |
| 17:54:09 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server usage audit log APIs https://review.opendev.org/c/openstack/nova/+/953209 | |
| 17:54:10 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for simple tenant usage APIs https://review.opendev.org/c/openstack/nova/+/956096 | |
| 18:29:23 | opendevreview | Merged openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | |
| 19:12:33 | opendevreview | Merged openstack/nova master: api: Add response body schemas for images APIs https://review.opendev.org/c/openstack/nova/+/952284 | |
| 20:26:35 | opendevreview | Merged openstack/nova master: Replace eventlet.event.Event with threading.Event https://review.opendev.org/c/openstack/nova/+/949754 | |
| #openstack-nova - 2025-07-30 | |||
| 01:03:17 | opendevreview | melanie witt proposed openstack/nova master: Add hw:tpm_secret_security extra spec validation https://review.opendev.org/c/openstack/nova/+/940197 | |
| 01:03:17 | opendevreview | melanie witt proposed openstack/nova master: Add hw_tpm_secret_security image property https://review.opendev.org/c/openstack/nova/+/940196 | |
| 01:03:18 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `user` secret security https://review.opendev.org/c/openstack/nova/+/942502 | |
| 01:03:18 | opendevreview | melanie witt proposed openstack/nova master: TPM: migrate legacy instances to new security policy https://review.opendev.org/c/openstack/nova/+/942501 | |
| 01:03:19 | opendevreview | melanie witt proposed openstack/nova master: Add vtpm_secret_(uuid|value) to LiveMigrateData https://review.opendev.org/c/openstack/nova/+/952628 | |
| 01:03:19 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `host` secret security https://review.opendev.org/c/openstack/nova/+/941795 | |
| 01:03:20 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `host` secret security https://review.opendev.org/c/openstack/nova/+/941483 | |
| 01:03:21 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `deployment` secret security https://review.opendev.org/c/openstack/nova/+/925771 | |
| 01:03:21 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `deployment` secret security https://review.opendev.org/c/openstack/nova/+/942021 | |
| 01:03:23 | opendevreview | melanie witt proposed openstack/nova master: TPM: update instance request_spec with secret security https://review.opendev.org/c/openstack/nova/+/952630 | |
| 01:03:23 | opendevreview | melanie witt proposed openstack/nova master: TPM: test live migration between hosts with different security https://review.opendev.org/c/openstack/nova/+/952629 | |
| 01:03:25 | opendevreview | melanie witt proposed openstack/nova master: TPM: confirm secret security via hard reboot https://review.opendev.org/c/openstack/nova/+/955847 | |
| 01:12:59 | opendevreview | melanie witt proposed openstack/nova master: TPM: migrate legacy instances to new security policy https://review.opendev.org/c/openstack/nova/+/942501 | |
| 01:13:00 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `host` secret security https://review.opendev.org/c/openstack/nova/+/941795 | |
| 01:13:00 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `user` secret security https://review.opendev.org/c/openstack/nova/+/942502 | |
| 01:13:01 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `host` secret security https://review.opendev.org/c/openstack/nova/+/941483 | |
| 01:13:01 | opendevreview | melanie witt proposed openstack/nova master: Add vtpm_secret_(uuid|value) to LiveMigrateData https://review.opendev.org/c/openstack/nova/+/952628 | |
| 01:13:02 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `deployment` secret security https://review.opendev.org/c/openstack/nova/+/925771 | |
| 01:13:02 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `deployment` secret security https://review.opendev.org/c/openstack/nova/+/942021 | |
| 01:13:03 | opendevreview | melanie witt proposed openstack/nova master: TPM: update instance request_spec with secret security https://review.opendev.org/c/openstack/nova/+/952630 | |
| 01:13:03 | opendevreview | melanie witt proposed openstack/nova master: TPM: test live migration between hosts with different security https://review.opendev.org/c/openstack/nova/+/952629 | |
| 01:13:05 | opendevreview | melanie witt proposed openstack/nova master: TPM: confirm secret security via hard reboot https://review.opendev.org/c/openstack/nova/+/955847 | |
| 01:17:26 | sean-k-mooney | melwitt: im actully going to go to sleep now instead of saying i will and not but do you have tempest test for that and or a devstack config you can share. i dont think i have any locally with barbican but the rest is pretty simple. | |
| 01:17:33 | sean-k-mooney | oh you have whitebox https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/955969 | |
| 01:18:47 | melwitt | sean-k-mooney: yes whitebox so far but I'm thinking of writing up some plain tempest tests also. thanks :) | |
| 01:19:19 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/c0bce36dbd29472bb743b4e883d0dfc0/log/controller/logs/local_conf.txt i can prbably steal the bit i will need form ther | |
| 01:19:39 | sean-k-mooney | ack ya whitebox is nice but not strictly needed for this | |
| 01:20:08 | sean-k-mooney | its useful for checkign the libvirt secrete flags in the deployment one | |
| 01:20:25 | sean-k-mooney | but the base functionalty can be done in core tempest | |
| 01:21:16 | melwitt | yeah the main configs you need are swtpm_enabled = True and supported_tpm_secret_security = <list>. there is also default_tpm_secret_security = <string> that you specify if you want | |
| 01:21:42 | sean-k-mooney | that and deploying barbican :) | |
| 01:21:55 | melwitt | also with barbican, the thing to know is if you have enforce_new_defaults = False then you need the 'creator' role to create secrets | |
| 01:21:58 | sean-k-mooney | which is the main thing missing form my config i think btu the rest is useful | |
| 01:22:18 | melwitt | if you have enforce_new_defaults = True then being a project member is good enough for creating secrets | |
| 01:22:55 | sean-k-mooney | melwitt: so i need to deploy my 3 node devstack with watcher ot test some other stuff but ill add barbican to it when i do and then probly next week ill checkout your tpm serise and try and use it | |