Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-01-21
16:32:23 bauzas (sean-k-mooney) disable the heal instance info cache periodic
16:32:29 bauzas sean-k-mooney: around now ?
16:32:33 sean-k-mooney ya so context
16:33:26 sean-k-mooney about 2-3 years ago the neturon folks started seeing higher then expecte background load the neutorn server api in bug reports form upstream/downstream custoemrs
16:34:05 sean-k-mooney after some invstatiation thaty raised the topic of our periodic heal action in a neutron nova cross project session and asked is it really needed
16:34:12 bauzas indeed
16:34:25 sean-k-mooney at the time we determined that it should not be but it kind of fell off our radar
16:34:27 bauzas I do also remember some OpenInfra talks about that
16:34:55 sean-k-mooney recently this has come up again down stream and i wanted to bring th topic back upstrem
16:35:04 sean-k-mooney so there are two patches related to this in flight
16:35:18 sean-k-mooney the first just test disabling the perodic in nova next
16:35:32 sean-k-mooney i belive tha that has already merged
16:35:40 sean-k-mooney the second which is what i wanted to dicuss
16:35:53 sean-k-mooney is how do we feel about finally disbaling the perodic by default
16:36:19 bauzas given the operator feedback I heard, I'm pro-this
16:36:34 sean-k-mooney i would like to propose doign it this cycle but if we want to leave it back in the nova-next job to get more data that also a path forward
16:36:43 bauzas sean-k-mooney: here's my proposal,
16:37:07 bauzas send a thread to openstack-discuss by notifying the ops that we plan to do it this cycle
16:37:13 bauzas link to the proposed patch
16:37:29 bauzas and if nobody argues by the next weeks, then we can merge it
16:37:47 bauzas my bet is that operators will appreciate the default disabling
16:37:59 sean-k-mooney ok i can do that
16:38:15 sean-k-mooney shal we set a date to check back. say 2 weeks?
16:38:27 sean-k-mooney i would prefer not to wiat rith to FF to change the default
16:38:49 sean-k-mooney in case we decied we need to revert it
16:38:53 bauzas good point
16:39:07 bauzas yeah, 2 weeks seems a reasonable trade-off
16:39:07 masahito I agree the default change as a large scale OpenStack operator. it's nice update.
16:39:14 bauzas we have 5 weeks to FF
16:39:25 bauzas that would leave 3 weeks with the change running in CI
16:40:05 sean-k-mooney ack ill send a mail to the list later today
16:40:10 gibi I'm OK with the 2 weeks notice
16:40:45 sean-k-mooney the proceedign patch https://review.opendev.org/c/openstack/nova/+/939453/1
16:40:51 sean-k-mooney hasnet actully merge yet
16:41:05 sean-k-mooney that is just testing disabling it in nova-next only
16:41:18 sean-k-mooney if there is no objection ill recheck that
16:41:36 sean-k-mooney and that gives ues 2 addtional weeks fo feedback in the nova-next job while we wait
16:41:52 bauzas indeed
16:41:58 bauzas sounds a plan
16:42:42 bauzas #agreed sean-k-mooney will notify openstack-discuss about disabling by default the instance info cache and give a 2-week notice for disagreements
16:42:53 bauzas cool
16:42:56 bauzas moving on
16:43:55 bauzas (hemanth) Request for review - Compute memory monitor
16:43:59 bauzas is hemanth here?
16:44:48 bauzas looks not
16:44:51 sean-k-mooney so we have said no to extenind that inteface to coolect memory stats in the past
16:45:07 sean-k-mooney this would be a spec or specless blueprint too right
16:45:10 bauzas and this is a feature request, which would require an approval exception
16:45:27 bauzas given we're past the Feature Approval Freeze deadline
16:45:52 sean-k-mooney https://docs.openstack.org/nova/latest/contributor/policies.html#metrics-gathering
16:45:56 gibi yeah put it on the next PTG agenda
16:46:04 sean-k-mooney that speicficly stats that we wont implement new monditors
16:46:04 bauzas I'm particularly not against discussing that feature with the owner but not this cycle
16:46:16 sean-k-mooney that was added after intel propeosed extending this for numa/memory metrics
16:46:48 sean-k-mooney so we can revisit this but it would be a direct revsersal fo that proabition on new monitors
16:47:11 bauzas agreed, this is a broad discussion
16:47:54 bauzas and I'd want to have it done with a quorum
16:48:20 bauzas I agree with gibi, we could add it tothe PTG (provided I create the PTG agenda)
16:48:34 sean-k-mooney yep ptg disccsion woudl be good
16:49:17 gibi yeah so give the discussion a space and time but set expectation with hemanth that this is not an easy one due to the policy in place
16:49:24 bauzas tbc, one of the options being "continue to disregard any other monitors" or even "deprecate the CPU one"
16:50:03 sean-k-mooney so the exisitn inteface is currently frozen
16:50:10 bauzas gibi: 100% agreed, we would need to understand the usecase and why this is crucial to have Nova supporting such monitors
16:50:14 sean-k-mooney and we partly reverted the merged numa code
16:50:26 sean-k-mooney because of the perfoamce impact on the rpc bus
16:50:40 sean-k-mooney however i do know that peopel use the existing cpu monitor
16:50:49 sean-k-mooney namely our internal redhat cloud uses it
16:51:03 sean-k-mooney and it massivly helped them with there workload schdduleing
16:51:45 bauzas perhaps, but there are limitations that people need to understand
16:51:52 sean-k-mooney the specificly use the host cpu load metric via the metrics weigher to avoid hevlilly loaded hosts when placing new instnaces
16:52:13 bauzas I know, that's the most common usecase
16:52:54 bauzas as I said, there are usecases but also limitations
16:53:18 bauzas like, the timewindow is pretty large and if you rely your cloud capacity on such timewindow, you're a fool
16:53:35 sean-k-mooney yep we dont need to preempt the ptg topic
16:53:56 sean-k-mooney well i disagre with that statement but anyway there are lots of dragons here
16:54:15 sean-k-mooney the metrics are sent as aprt fo each schduelr report
16:55:04 sean-k-mooney anywya i think we can move on
16:55:46 bauzas #agreed as the current design policy explicitely avoiding new monitors, this feature requires a quorum and a long discussion to evaluate a policy revision, hence the Nova community proposing to revisit that topic at the PTG
16:56:00 bauzas sean-k-mooney: every 60 secs, right ? :)
16:56:17 bauzas (by default of course)
16:56:53 bauzas I'm just saying that planning your cloud capacity on a 60-sec time window is nice but not helpful
16:57:09 bauzas anyway, we're mostly at time
16:57:19 bauzas anything else ?
16:57:23 gibi -
16:57:42 bauzas thanks all
16:57:46 bauzas #endmeeting
16:57:46 opendevmeet Meeting ended Tue Jan 21 16:57:46 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
16:57:46 opendevmeet Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-01-21-16.00.html
16:57:46 opendevmeet Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-01-21-16.00.txt
16:57:46 opendevmeet Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-01-21-16.00.log.html
16:57:59 elodilles thanks o/
16:58:04 masahito thank you
16:58:33 gibi thanks
18:20:25 opendevreview Mark Goddard proposed openstack/nova master: ironic: Let Ironic handle deployment cleanup actions during destroy https://review.opendev.org/c/openstack/nova/+/883411
18:37:08 mikal sean-k-mooney: are you still around?
18:50:03 sean-k-mooney mikal: kind of
18:50:31 sean-k-mooney i will be for the next 20-30 mins or so
18:50:54 sean-k-mooney mikal: you updated your spcie patches in the last few days correct
18:52:06 sean-k-mooney i asume you woudl like to make progress on https://review.opendev.org/c/openstack/nova/+/927354/13 and the proceedign ones
20:06:05 mikal sean-k-mooney: yeah, the code didn't change, I just figured if I pushed again I'd get a newer CI run. I figure I'll do that weekly from now on.

Earlier   Later