| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-01-18 | |||
| 14:46:55 | sean-k-mooney1 | which woudl be unsupported | |
| 14:47:03 | sean-k-mooney1 | since the contoler shoudl always be upgraded first | |
| 14:47:44 | sean-k-mooney1 | i mean if you clamp the rpc apis in the config you can praobly make it work but the assumtion we have is contoler are always newer or the same version as the computes | |
| 14:52:00 | sean-k-mooney | gibi: basicaly i think we shoudl either modify the check to only consider up compute servics or we shoudl move the check to the compute agent instead | |
| 14:52:41 | gibi | hehe, I had considered disabled old computes https://review.opendev.org/c/openstack/nova/+/738482/4..10/nova/service.py#430 | |
| 14:52:42 | sean-k-mooney | or add a config option to opt out of the check for ffu | |
| 14:53:21 | sean-k-mooney | for what its worth i did some testing with devstack not so long ago | |
| 14:53:49 | sean-k-mooney | if i disable the version check you can mix wallaby contoler with train computes | |
| 14:54:57 | sean-k-mooney | to enable better upgrades we are considering supproting that as an upgrade path in the future downstream. not for train to wallaby but from wallaby to whatever our next release would be based on | |
| 14:55:03 | gibi | hm, a disable but running old compute can cause trouble in the cluster, a disable + force down compute cannot as we define force_down as "admin-promised-it-is-not-running" | |
| 14:55:24 | gibi | also based on the discussion in the original patch | |
| 14:55:44 | sean-k-mooney | yes force down means this is down nova just has not noticed yet i promise | |
| 14:56:05 | gibi | so at leat keeping the force_down make sense | |
| 14:56:26 | gibi | I'm not sure about the disable but running compute | |
| 14:56:36 | sean-k-mooney | well i was not thinking about disable | |
| 14:56:42 | sean-k-mooney | i was thinking more up=false | |
| 14:56:45 | gibi | ahh | |
| 14:56:55 | sean-k-mooney | so heartbeat is missing and we mark it as down | |
| 14:56:59 | gibi | so the compute definition exists in the db but the compute is timed out | |
| 14:57:19 | sean-k-mooney | ya presujmably because you stop it on the compute hosts as part of the upgrade | |
| 14:57:28 | gibi | it could be a network partition but it also can be a manual stop yes | |
| 14:58:10 | sean-k-mooney | well if it a network partiion it cant talk to the conductor :) but honestly i think the check is better done on the comptue side then the conductor side | |
| 14:58:33 | gibi | yeah but the check on the compute side only work if the compute is restarted | |
| 14:58:37 | gibi | if the compute is running | |
| 14:58:40 | gibi | then no check happens | |
| 14:58:45 | sean-k-mooney | that is fair | |
| 14:58:47 | gibi | that prevent the old compute to send nonsense to new conductor | |
| 14:59:05 | gibi | probably there is no perfect solution | |
| 14:59:23 | gibi | as from the controller perspective a timed out compute can mean multiple things | |
| 14:59:29 | sean-k-mooney | i mean for now we have the workaround of before you FFU stop all compute agaents and force themn down in the api | |
| 15:00:00 | gibi | yeah, and while that is an extra step it does not create an extra inconsistency | |
| 15:00:12 | gibi | you just need to double confirm that those computes are not coming back | |
| 15:00:16 | gibi | without a restart | |
| 15:00:59 | sean-k-mooney | honestly unless we have done a major rpc bump the comptue wont impact the conductor if they start | |
| 15:01:30 | sean-k-mooney | but yes if you use force down you shoudl fulfil its requiremetns | |
| 15:02:38 | gibi | hm an old compute cannot decided if it is too old, as the old code does not know what is the service version of the N+2 release | |
| 15:03:01 | gibi | except if we backport fixes to stable branches after the N | |
| 15:03:04 | gibi | N+2 release | |
| 15:03:23 | sean-k-mooney | well i was more thingink it would try to connect to the conductor and it would say its too old | |
| 15:03:32 | sean-k-mooney | but ya they cant really | |
| 15:04:10 | gibi | yeah the conductor could tell the compute to go away you are too old | |
| 15:04:35 | sean-k-mooney | it will do that if a new compute tries to regeister | |
| 15:17:13 | bauzas | gibi: I could be late for the meeting, could you please start it if so ? | |
| 15:17:20 | bauzas | the agenda is updated | |
| 15:18:08 | bauzas | reminder : nova meeting starts in 42 mins here | |
| 15:22:20 | gibi | bauzas: sure | |
| 15:47:02 | gibi | just a quick reminder nova meeting starts in 13 minutes here in the channel | |
| 15:57:03 | bauzas | I'm back | |
| 15:57:09 | bauzas | right on time | |
| 16:00:00 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:00:00 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:00:00 | opendevmeet | Meeting started Tue Jan 18 16:00:00 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:00:00 | bauzas | #startmeeting nova | |
| 16:00:12 | gibi | o. | |
| 16:00:13 | gibi | o/ | |
| 16:00:55 | chateaulav | \o | |
| 16:00:56 | elodilles | o/ | |
| 16:01:19 | bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | |
| 16:01:53 | bauzas | ok, let's start | |
| 16:02:01 | bauzas | #topic Bugs (stuck/critical) | |
| 16:02:06 | bauzas | #info No Critical bug | |
| 16:02:10 | bauzas | #info No Critical bug | |
| 16:02:15 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 32 new untriaged bugs (-4 since the last meeting) | |
| 16:02:20 | bauzas | #help Nova bug triage help is appreciated https://wiki.openstack.org/wiki/Nova/BugTriage | |
| 16:02:24 | bauzas | #link https://storyboard.openstack.org/#!/project/openstack/placement 23 open stories (-2 since the last meeting) in Storyboard for Placement | |
| 16:02:34 | bauzas | thanks to those who helped to triage | |
| 16:02:43 | bauzas | (I dunno who about placement :p) | |
| 16:04:06 | gibi | bauzas: can be that the any-trait spec merges closed the related ticjket | |
| 16:04:09 | gibi | I will check | |
| 16:04:33 | bauzas | maybe | |
| 16:04:35 | bauzas | anyway | |
| 16:04:42 | bauzas | any bug to want to discuss ? | |
| 16:05:21 | bauzas | looks not | |
| 16:05:27 | bauzas | #topic Gate status | |
| 16:05:32 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | |
| 16:05:37 | bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status | |
| 16:05:43 | bauzas | #info Please look at the gate failures, file a bug, and add an elastic-recheck signature in the opendev/elastic-recheck repo (example: https://review.opendev.org/#/c/759967) | |
| 16:06:21 | gibi | (yepp the two missing placement ticket is the any-trait support, I will reopen them by creating an open task on them) | |
| 16:06:21 | bauzas | let's discuss about the centos8 job with elodilles in the stable branches topic | |
| 16:06:55 | bauzas | apart from this job, nothing to say | |
| 16:06:59 | bauzas | anyone ? | |
| 16:08:11 | bauzas | mmmm, ok, let's move | |
| 16:08:18 | bauzas | #topic Release Planning | |
| 16:08:26 | bauzas | #info FeatureApprovalFreeze was Jan 13st | |
| 16:08:32 | bauzas | #link https://blueprints.launchpad.net/nova/yoga 23 approved blueprints for the Yoga cycle with 16 of them having specs | |
| 16:08:36 | bauzas | #link https://etherpad.opendev.org/p/nova-yoga-blueprint-status Etherpad for blueprints tracking | |
| 16:08:48 | bauzas | so, as you see, I created an etherpad for yoga | |
| 16:08:55 | bauzas | I'll look at each of the blueprints | |
| 16:09:17 | bauzas | and I'll modify the etherpad to explain which ones are started, and which ones need reviews | |
| 16:09:40 | bauzas | but if folks want to modify the etherpad and explain the changes they have for their blueprint, fine by me :) | |
| 16:10:08 | bauzas | every meeting until FeatureFreeze, I'll provide this etherpad during the meeting | |
| 16:10:24 | gibi | cool | |
| 16:10:25 | bauzas | just remember that we now have 6 weeks before it | |
| 16:11:31 | bauzas | anyway, let's discuss about this etherpad by next week as I need to modify it for providing the changes | |
| 16:11:47 | bauzas | also, last thought | |
| 16:12:14 | bauzas | given it's difficult to merge API changes as they're conficting once we merge one | |
| 16:12:31 | bauzas | I'll provide all the changes wanting to add a new microversion | |
| 16:12:57 | sean-k-mooney | ya we can likely come up with a merge order when we get a little closer to FF | |
| 16:12:58 | bauzas | so during the reviews, people could organize themselves to see who and what could be reviewed first etc. | |
| 16:13:38 | bauzas | sean-k-mooney: yeah, here I just want to tell it by now so people see | |