| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-02-21 | |||
| 16:37:56 | bauzas | the fact is, I wrote Opinion | |
| 16:38:04 | bauzas | unless people have concerns with what I wrote, I'm done. | |
| 16:39:07 | bauzas | looks not | |
| 16:39:11 | bauzas | then I assume we're done. | |
| 16:40:16 | dansmith | ++ | |
| 16:40:18 | bauzas | thanks all | |
| 16:40:23 | bauzas | #endmeeting | |
| 16:40:23 | opendevmeet | Meeting ended Tue Feb 21 16:40:23 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:40:23 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2023/nova.2023-02-21-16.00.html | |
| 16:40:23 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2023/nova.2023-02-21-16.00.txt | |
| 16:40:23 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2023/nova.2023-02-21-16.00.log.html | |
| 16:40:33 | elodilles | thanks o/ | |
| 16:40:57 | dansmith | bauzas: so, gmann and I were running that memory usage patch in periodic on tempest jobs for a few days to make sure it didn't substantially worsen things | |
| 16:41:07 | dansmith | and my survey at the moment indicates that it looks good | |
| 16:41:41 | dansmith | so I'll propose to make it enabled for ceph-multistore (which will also impact glance) and we'll see if gmann is cool with that when he's around | |
| 16:41:44 | bauzas | nice to hear | |
| 16:41:52 | bauzas | ack, do it and I'll vote | |
| 16:45:08 | opendevreview | Dan Smith proposed openstack/nova master: Use mysql memory reduction flags for ceph job https://review.opendev.org/c/openstack/nova/+/874664 | |
| 16:45:11 | dansmith | bauzas: ^ | |
| 17:43:02 | bauzas | dansmith: I doubt that cells_v2 map_instances could work with https://bugs.launchpad.net/nova/+bug/2007922 (even I asked for it) | |
| 17:43:19 | bauzas | dansmith: tl;dr: the instance mapping exists but the cell value is None | |
| 17:43:30 | bauzas | and we know the instance is in cell0 DB | |
| 17:43:53 | dansmith | yeah, as I said, I initially missed that the person said they had reference in the mappings table | |
| 17:43:56 | bauzas | dansmith: I guess the simpliest thing is to hack the DB to add the cell0 uuid in the instancemapping record, nope ? | |
| 17:44:08 | dansmith | probably | |
| 17:44:14 | bauzas | or do we have a better nova-manage command ? | |
| 17:44:18 | bauzas | looking at the docs, nope | |
| 17:44:29 | dansmith | not that I know of | |
| 17:44:33 | bauzas | this instance is somehow sit in the middle | |
| 17:44:46 | bauzas | not fully migrated but in between | |
| 17:44:58 | dansmith | not fully ... mapped? | |
| 17:45:06 | bauzas | sorry, yeah mapped | |
| 17:45:23 | bauzas | I'll propose the ALTER to the reporter | |
| 17:47:42 | dansmith | don't we have a mapped flag on the instance (or something else)? | |
| 17:48:36 | bauzas | in the instances table you mean ? | |
| 17:48:57 | dansmith | I thought it was.. that's how we survey instances that need to be mapped right? | |
| 17:49:13 | dansmith | just wondering if that flag matches or not | |
| 17:52:44 | bauzas | so | |
| 17:52:55 | bauzas | https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L874 | |
| 17:53:12 | bauzas | we just iterate over a limit and a marker on the instances table from a cell that's given | |
| 17:54:34 | dansmith | ah right | |
| 17:54:44 | bauzas | and I think I understand how the cell ID was set to None | |
| 17:54:51 | bauzas | https://github.com/openstack/nova/blob/439c67254859485011e7fd2859051464e570d78b/nova/objects/instance_mapping.py#L73 | |
| 17:55:31 | dansmith | it only does that if it's not none though | |
| 17:55:34 | bauzas | anyway, map_instances *could* work with cell0 | |
| 17:56:18 | bauzas | if the reporter runs map_instances with cell0 attribute, it will loop over the contents of cell0's instances table and will create an instancemapping object | |
| 17:56:42 | bauzas | oh wait, fuck no | |
| 17:56:42 | bauzas | https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L791-L792 | |
| 17:56:57 | bauzas | so, definitely the easier is to alter the db | |
| 18:00:12 | dansmith | again, I only thought it was useful to run map if the mapping didn't exist | |
| 18:03:45 | bauzas | yup | |
| 18:03:54 | bauzas | or the reporter could then delete the instance mapping | |
| 18:10:15 | sean-k-mooney | bauzas: elodilles can we prioritise review of this if possibel https://review.opendev.org/c/openstack/nova/+/874547 | |
| 18:10:30 | opendevreview | Takashi Natsume proposed openstack/placement master: Move implemented specs for Xena and Yoga release https://review.opendev.org/c/openstack/placement/+/853730 | |
| 18:11:18 | sean-k-mooney | this will help us fix our downstream ci | |
| 18:13:16 | bauzas | done but I leave you +W as I don't have a lot of context | |
| 18:15:39 | sean-k-mooney | tl;dr we use bindep in our downstream jobs to install deps before runing tox but rhel 8 nolonger has python-devel | |
| 18:15:47 | gmann | dansmith: +W on 'mysql memory reduction flags for ceph job' | |
| 18:15:55 | sean-k-mooney | i wanted to check with elodilles to make sure they were ok with the stable-only change | |
| 18:16:00 | dansmith | gmann: cool | |
| 18:16:04 | dansmith | thanks | |
| 18:24:10 | dansmith | gmann: oh jeez, I didn't realize the mysql periodic thing hadn't landed yet | |
| 18:24:32 | dansmith | gmann: do you think we should wait for that to soak for a bit? | |
| 18:24:47 | dansmith | I know the devstack one did, and I guess I misread that the tempest one hadn't yet | |
| 18:26:05 | gmann | dansmith: I also did not realize it when I checked that patch. but I think it is ok to enable it in ceph job and see. | |
| 18:26:25 | gmann | we can always revert it if it fail and make delay things during release time | |
| 18:26:34 | dansmith | okay, that's my preference too | |
| 18:44:46 | mnaser | i got a fun one. it looks like by default nova saves the az of the vm in the cell db, but it doesn't update the request_spec, but when we do migrations, we pass the request_spec to the scheduler (which contains az=null) which then moves you from one az to another in the migration | |
| 18:45:05 | mnaser | since .. https://github.com/openstack/nova/blob/90e2a5e50fbf08e62a1aedd5e176845ee22d96c9/nova/scheduler/request_filter.py#L138-L166 checks for request_spec az | |
| 18:45:40 | sean-k-mooney | mnaser: this was changed recently | |
| 18:45:56 | mnaser | this is in a scenario where an operator wants to make vms stick to their az if a user doesnt specify one | |
| 18:46:22 | sean-k-mooney | right so we spent a lot fo time trying to decide what the sematics shoudl be | |
| 18:46:56 | sean-k-mooney | im trying to find the spec | |
| 18:47:32 | sean-k-mooney | https://specs.openstack.org/openstack/nova-specs/specs/zed/implemented/unshelve-to-host.html | |
| 18:47:38 | sean-k-mooney | i guess this was for unshleve | |
| 18:48:05 | sean-k-mooney | mnaser: we epect tht the request spec would not have the az by the way if the user did not request one | |
| 18:48:44 | mnaser | makes sense cause that's their request | |
| 18:49:12 | mnaser | i understand it might not be everyone that wants this, but maybe for live migration use case it can cause issues if nova ends up doing cross-az migrations | |
| 18:49:54 | sean-k-mooney | for move operations where we supprot specifying a AZ it would be ok in some cases to set it in the request spec | |
| 18:50:13 | sean-k-mooney | mnaser: but we would want to have the same beahivor as in the unselve spec | |
| 18:50:29 | sean-k-mooney | i dont recal if we fixed the other move operatiosn to be consitent with that when we did this | |
| 18:50:35 | sean-k-mooney | Uggla: do you recall | |
| 18:51:36 | sean-k-mooney | mnaser: wiht unshelve to a specific az if you set it in the unshelve request and it was not set in the orgainl request spec it will be set after | |
| 18:52:52 | sean-k-mooney | mnaser: live migraiton does not currently supprot an az | |
| 18:52:55 | mnaser | sean-k-mooney: esentailly im thinking this is where this can be changed https://github.com/openstack/nova/blob/f01a90ccb85ab254236f84009cd432d03ce12ebb/nova/compute/api.py#L5499-L5500 | |
| 18:53:20 | mnaser | cause live migrating from one az to another could pretty much fail, and we can just have it as an option i guess if we dont want to change default behaviour | |
| 18:53:22 | sean-k-mooney | nor does migrate | |
| 18:53:35 | mnaser | in most worlds migrate or live migrate will fail across az's | |
| 18:53:40 | mnaser | esp if you're using different storage backends for example | |
| 18:53:43 | sean-k-mooney | mnaser: this would be an api change and need a spec | |
| 18:54:21 | sean-k-mooney | in general live migration betwen AZ will either work on not work depending on yoru deployment. in general i would expect it to work in most cases | |
| 18:55:17 | sean-k-mooney | it just comes down to if you have exchanged ssh keys such that the hyperviors can comunicate and if you are using az with cinder or not | |
| 18:55:25 | sean-k-mooney | and cross_az attach | |
| 18:56:00 | mnaser | Maybe we can make it so that if cross az attach = false then it would update the request spec to match? | |
| 18:56:07 | sean-k-mooney | no | |
| 18:56:15 | sean-k-mooney | no config drvent api behavior | |
| 18:56:20 | sean-k-mooney | this is not a bug | |
| 18:56:38 | sean-k-mooney | if we want to supprot move operation to target an AZ or change the request spec this is an api change | |
| 18:56:54 | mnaser | No it’s not to target an AZ | |
| 18:56:54 | mnaser | No it’s not to target an AZ | |