| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-11-04 | |||
| 15:05:49 | dansmith | you can if you control the order of the importing in the entry point | |
| 15:05:51 | gibi | when the second monkey patch call tries to change the backend due to differning default | |
| 15:06:30 | gibi | dansmith: that is what we do, we added the monkey patch as early as possible in the import list | |
| 15:06:37 | gibi | that is our control | |
| 15:07:44 | dansmith | I'm not sure if you're not understanding what I'm saying, or don't think it's possible, or think it's a bad idea.. either way, I'm not arguing...just commenting about my opinion | |
| 15:07:52 | gibi | import wsgi from api first execute __init__ so we need to be in __init__ if __init__ has other imports like oslo.log that creates a lock | |
| 15:08:22 | gibi | I guess I'm missing what would you like to see in case of api/__init__ | |
| 15:08:44 | gibi | s/import wsgi from api/from api import wsgi/ | |
| 15:10:05 | gibi | I think it is not possible to move the monkey_patch to api/wsgi from api/__init__ | |
| 15:10:32 | gibi | as api/__init__ has other imports | |
| 15:11:56 | sean-k-mooney | you could monkey patch before the other import | |
| 15:12:08 | dansmith | this ^ | |
| 15:12:08 | sean-k-mooney | you might need to add a noqa or similar comment | |
| 15:12:11 | dansmith | but I don't want to argue | |
| 15:12:17 | sean-k-mooney | but we can if we need too | |
| 15:12:35 | gibi | but I monkey patch before all the imports | |
| 15:12:36 | gibi | :) | |
| 15:13:05 | sean-k-mooney | i have not been folloing the converstaion by they way | |
| 15:13:21 | gibi | the first line of nova/api/openstack/__init__.py is the monkey patch | |
| 15:13:47 | gibi | I think dansmith would like that to be moved somewhere else | |
| 15:13:51 | sean-k-mooney | ah yes i remoember dicussign the #noqa comment for that | |
| 15:14:38 | gibi | at the moment I'm not sure where to move our nova.api.openstack.compute.wsgi:init_application to be able to move the monkey_patch | |
| 15:14:41 | sean-k-mooney | so ya https://github.com/openstack/nova/blob/master/nova/api/__init__.py and https://github.com/openstack/nova/blob/master/nova/__init__.py are empty today | |
| 15:15:24 | sean-k-mooney | well the entry point ot that is https://github.com/openstack/nova/blob/master/nova/wsgi/osapi_compute.py | |
| 15:15:55 | sean-k-mooney | across openstack we are ment ot have <package>.wsgi.<api endpoint) | |
| 15:16:23 | sean-k-mooney | so you could monkey patch here https://github.com/openstack/nova/blob/master/nova/wsgi/__init__.py but im not sure it is required | |
| 15:16:25 | gibi | hm but our setup.cfg ponts to nova-api-wsgi = nova.api.openstack.compute.wsgi:init_application | |
| 15:16:49 | sean-k-mooney | that in wsgi script | |
| 15:16:52 | sean-k-mooney | but that is being deleted | |
| 15:17:05 | sean-k-mooney | it will generate the same script effectivly | |
| 15:17:42 | sean-k-mooney | we agreed at the ptg to finally delete https://github.com/openstack/nova/blob/master/setup.cfg#L87-L89 | |
| 15:17:51 | gibi | but until that setup.cfg is a thing I need to make sure that works. So today I cannot move the monkey patch to nova/wsgi because that will not be executed if the setup.cfg defined entry point is used | |
| 15:18:35 | gibi | I agree that once the setup.cfg does not have such entry point defined and only the nova/wsgi is used then I can move the monkey_patch | |
| 15:18:37 | sean-k-mooney | right but lest just find and merge stephesn patch | |
| 15:18:44 | sean-k-mooney | then you can move it if needed | |
| 15:18:50 | gibi | OK | |
| 15:19:14 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/902688 | |
| 15:19:40 | sean-k-mooney | he updated the docs too | |
| 15:20:22 | sean-k-mooney | so its a slightly longer review then i was plannign but ill re review that today/tomorrow | |
| 15:20:53 | gibi | so that patch suggest the deployers do the right thing and use nova/wsgi | |
| 15:21:17 | sean-k-mooney | yep which is how devstack has worked for 2 releases now | |
| 15:25:33 | gibi | I need to respin my defaulting series anyhow so I can rebase on top of stephenfin and move the monkey patch accordingly | |
| 15:37:29 | stephenfin | sean-k-mooney: if you have time, would you be able to continue and grab the other 3 patches on top of that one? https://review.opendev.org/c/openstack/nova/+/953703 | |
| 16:00:05 | sean-k-mooney | my 1:1 just finished so sure. | |
| 16:10:09 | gibi | stephenfin: could you quickly fix the versioning issue in https://review.opendev.org/c/openstack/nova/+/902688/9#message-72481479904c69ce6f3adbcdfb26143c6f85756a then we can land that | |
| 16:13:33 | sean-k-mooney | gibi: ah the versionchanged line | |
| 16:13:39 | sean-k-mooney | ya that shoudl eb 33.0.0 | |
| 16:14:14 | opendevreview | Stephen Finucane proposed openstack/nova master: setup: Remove pbr's wsgi_scripts https://review.opendev.org/c/openstack/nova/+/902688 | |
| 16:14:15 | opendevreview | Stephen Finucane proposed openstack/nova master: Migrate mypy configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953703 | |
| 16:14:15 | opendevreview | Stephen Finucane proposed openstack/nova master: Migrate codespell configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953704 | |
| 16:14:16 | opendevreview | Stephen Finucane proposed openstack/nova master: Migrate setup configuration to pyproject.toml https://review.opendev.org/c/openstack/nova/+/953705 | |
| 16:14:16 | opendevreview | Stephen Finucane proposed openstack/nova master: pre-commit: Bump versions https://review.opendev.org/c/openstack/nova/+/966089 | |
| 16:14:22 | stephenfin | gibi: sean-k-mooney: done | |
| 16:14:53 | sean-k-mooney | stephenfin: did you see https://review.opendev.org/c/openstack/nova/+/953704/4/pyproject.toml | |
| 16:15:03 | sean-k-mooney | im fien with not changing it just noded it was false before | |
| 16:15:06 | gibi | stephenfin: thanks | |
| 16:15:24 | sean-k-mooney | as in im fine with the patch as is just making sure you saw it | |
| 16:18:20 | stephenfin | sean-k-mooney: replied on both | |
| 16:20:06 | sean-k-mooney | thanks | |
| 16:29:37 | Zhan[m] | hi friends, I'm trying to work on some improvements for pre-copy live migrations - to reduce VMs' network connectivity loss time. I put my ideas and research in https://bugs.launchpad.net/nova/+bug/2128665 and would love some feedback before I start coding and making changes. does anyone mind taking a look? I can create a blueprint if it's needed. thanks!! | |
| 16:45:04 | sean-k-mooney | Zhan[m]: you are effectivly askign to trigger post-live-migration very slightly earlier | |
| 16:45:22 | sean-k-mooney | when pre-copy triggers a pause of the guest. | |
| 16:45:39 | sean-k-mooney | that only happens if you have set force-complete as the timeout action or if you manually do that | |
| 16:46:06 | sean-k-mooney | we do this for post-copy migration already when it enter the post copy phase | |
| 16:46:32 | sean-k-mooney | im not sure how safe ti woudl be to do that in this case vs the current migration completion event | |
| 16:46:54 | sean-k-mooney | Zhan[m]: this likely need more then a bug ie. perhaps a spec | |
| 16:47:26 | sean-k-mooney | or atleast more condierdation the a quick review of your propal to think though the implciiton for upgrades and the diffenent failure modes | |
| 16:49:15 | Zhan[m] | understood, I will go ahead and create a spec. wanna get some early feedback to see if this idea make sense or not. | |
| 16:49:48 | Zhan[m] | we are not using post-copy (I know that we'll avoid this with post-copy) so hence this is just for pre-copy | |
| 16:49:50 | sean-k-mooney | we might be able to use a slightly earlier event | |
| 16:50:40 | Zhan[m] | from my understanding, the paused event should be the earliest? (basically when the process is paused for switch-over) | |
| 16:51:24 | sean-k-mooney | yes but we need to think about the recover cases if there si a failure after pause | |
| 16:51:39 | sean-k-mooney | we currently do it in complete because we know we cant rollback at that point | |
| 16:52:20 | sean-k-mooney | Zhan[m]: just to confirm you are currenlty using pre-copy not post-copy | |
| 16:52:27 | sean-k-mooney | is there a reason you are not using post-copy | |
| 16:55:56 | Zhan[m] | > if there si a failure after pause... (full message at <https://matrix.org/oftc/media/v1/media/download/AQcXzaU3yYgbieAMs5LdQxhfkMQjWLQ8n98jM_lIB8gwWH9ApwZ_vdawDM9LNz0brOl_UevcPCkv3f-67_yYcUZCeal-T3mAAG1hdHJpeC5vcmcvRFZtSG9LWGxZaFNCZ0hrYnpXbVh0WnNi>) | |
| 16:56:15 | Zhan[m] | * - if there si a failure after pause... (full message at <https://matrix.org/oftc/media/v1/media/download/AYPX5olphEwNM-oVQEAeW1n4bjBMOCL7JcBa2--xwCdXhT0nA_sPuR_jnKT0C0oDEmSyzPWDZpPKsZ1wc-FdO-dCeal-VAbgAG1hdHJpeC5vcmcvRUtYVUxDZVBiekRLck1wZ0hpSW1ZVWNn>) | |
| 16:56:51 | Zhan[m] | * - if there si a failure after pause... (full message at <https://matrix.org/oftc/media/v1/media/download/AfGmfWXp6q289_-pAg_3r2CNmHk8GLA3AAnfqOAWmBViUGQMwTDwe8p12QyJHYlvI7HoFg0OieVLTgc2FNFdS1dCeal-XNrgAG1hdHJpeC5vcmcvZkprTUVOeWlRR0htWmlYc2hLZVhYZ0ha>) | |
| 16:58:00 | Zhan[m] | * - if there si a failure after pause... (full message at <https://matrix.org/oftc/media/v1/media/download/AQbdtBpPhkIEZJLqRgxshCfnuFsmdpJyW0XBuScJnchhklfJJ0L11j5bwq4dTpkoh88wJMokcThxBRTCLw1ECLJCeal-bZagAG1hdHJpeC5vcmcvQXNCTUJqVWtqUWF0cmJKRG1XRURxbmF1>) | |
| 16:58:37 | Zhan[m] | I think this change can help regardless, maybe in some environments, pre-copy migrations are preferred. | |
| 17:05:50 | Zhan[m] | We've also seen some crazy wait time (20+ seconds) between when the VM is paused and when libvirt says the migration is done (the completion event), so that's where our motivation comes from. | |
| 17:48:08 | gibi | I saw working live migration with native threaded compute \o/ | |
| 17:48:11 | gibi | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_071/openstack/07142d96394445fc9d4d5ad58b963223/testr_results.html | |
| 18:12:30 | sean-k-mooney | gibi: nice | |
| 18:14:10 | sean-k-mooney | gibi: do you know what is causing the volume issues | |
| 18:14:17 | gibi | not yet | |
| 18:14:41 | sean-k-mooney | well its less broken then it could have been | |
| 18:14:54 | gibi | I'm still not convinced that the lifecycle events are actually working between libvirt and nova in the gate. Locally it is working for me so I need to look more tomorrow | |
| 18:14:57 | sean-k-mooney | i assum this is lvm? | |
| 18:15:57 | sean-k-mooney | ok ya it is cidner lvm | |
| 18:16:57 | gibi | yeah it is not ceph I'm sure | |
| 18:17:08 | gibi | but I drop now. Continue from here tomorrow | |
| 18:17:26 | sean-k-mooney | gibi: looksl ilke there are are a buch of multipahd error bug i cant tell at a glance if that brick being brick or actully imporant | |
| 18:19:18 | sean-k-mooney | gibi: when we stop monkeypatching nova-comptue | |
| 18:19:28 | sean-k-mooney | we ar ealso unmonkeypatchign os-brick | |
| 18:19:40 | sean-k-mooney | i hope that is not related | |
| 18:20:16 | sean-k-mooney | i do see that some of the iscsiadm commadns failed os hopefuly it a unrealted ci faliure and a recheck will jsut work | |
| 18:20:42 | sean-k-mooney | most of the time the iscsiadm dicovery is workign so it coudl be load/timing related | |
| 18:49:28 | sean-k-mooney | Zhan[m]: multi line matix message are hard to read on the irc side as we only get the first line and link to the rest of the message | |
| 18:49:52 | sean-k-mooney | Zhan[m]: but no i htink the issue you are hitting is sepcific to pre-copy migration | |