| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-11 | |||
| 07:42:26 | gibi | and good Friday morning to you too | |
| 07:42:58 | bauzas | gibi: and I -w'd the prelude because of the xenapi change still in the gate + this one that I think which needs to be in the prelude | |
| 07:43:30 | gibi | yeah, I kikked the xenapi series this morning with a recheck | |
| 07:43:35 | gibi | *kicked | |
| 07:43:59 | bauzas | I saw | |
| 08:23:52 | sean-k-mooney | o/ | |
| 08:25:37 | sean-k-mooney | gibi: so i would still like to procee with the deprecation of the az filter and enableing the placment version by default in Victoria since its behavior is equivalent, but i think i am going to split out the Comptefilter changes and look to adress that in Wallaby when we have more time to think about it | |
| 08:27:08 | sean-k-mooney | i dont know if we want to reuse the COMPUTE_STATUS_DISABLED trait for force down or not and i dont really want ot rush figuring that out | |
| 08:28:15 | sean-k-mooney | i think having a sperate COMPUTE_STATUS_DOWN trait is proably more correct but that would need an os-traits release and i dont think this warrents a FFE for os-triats since its just tech debt removal | |
| 08:28:19 | gibi | sean-k-mooney: make sense, keep the ComputeFilter then for V and please open a bug to track the problem you detected with the pre-filter | |
| 08:29:17 | sean-k-mooney | yep i can do that that was actully going to be my question should i open a bug or blueprint. e.g. is this an oversight in the original filter that it ignore force down or a new feature | |
| 08:31:13 | gibi | i consider this as a bug | |
| 08:35:42 | sean-k-mooney | cool ill file one so then respin the patch to drop the computefitler changes | |
| 08:36:12 | gibi | thanks | |
| 08:38:46 | bauzas | sean-k-mooney: wait, what ? | |
| 08:38:56 | bauzas | sean-k-mooney: which filter are you going to deprecate ? | |
| 08:39:05 | bauzas | AZFilter or ComputeFilter ? | |
| 08:39:13 | sean-k-mooney | i was deprecating both | |
| 08:39:22 | sean-k-mooney | but the behavior of the ComputeFilter | |
| 08:39:31 | sean-k-mooney | is not identical to the placment prefilter for compute status | |
| 08:39:48 | sean-k-mooney | because the prefilter does not take accound of force_down | |
| 08:39:50 | bauzas | ok, now I better understand and yes | |
| 08:39:59 | bauzas | I was about to say this | |
| 08:40:32 | bauzas | but... are we considering the AZfilter deprecation for the Victoria cycle ? | |
| 08:40:35 | sean-k-mooney | so for now im just going to depercate teh az filter then file a bug for the gap in behavior and we can figure out the correct way to handel force_down in wallaby | |
| 08:40:38 | bauzas | I'd say it's a Wallaby thing | |
| 08:40:46 | sean-k-mooney | bauzas: yes i was | |
| 08:41:11 | sean-k-mooney | https://review.opendev.org/#/c/745605/ | |
| 08:41:34 | sean-k-mooney | i have had that open for a while but only dug into the fun test fialure properly yesterday | |
| 08:41:48 | bauzas | ok, I feel we need to talk then | |
| 08:42:04 | bauzas | because I agree with the fact that the AZfilter is on par with the pre-filter | |
| 08:42:49 | sean-k-mooney | i could delay this if you want by the way but if you dont have an object ot the AZfilter then i would prefer to proceed with just that | |
| 08:42:50 | bauzas | but I know ops use filters by ordering them like they want | |
| 08:43:01 | sean-k-mooney | yes | |
| 08:43:10 | sean-k-mooney | but the AZ filter always goes first right | |
| 08:43:29 | bauzas | no | |
| 08:43:39 | sean-k-mooney | no? | |
| 08:43:41 | bauzas | by default, ComputeFilter goes first | |
| 08:43:54 | bauzas | and then AZ | |
| 08:44:08 | bauzas | if we deprecate the AZ, this will change | |
| 08:44:39 | bauzas | oh shit | |
| 08:44:41 | bauzas | I'm wrong | |
| 08:44:51 | openstackgerrit | Dat Le proposed openstack/nova stable/rocky: Fix unplugging VIF when migrate/resize/rebuild VM between hosts have difference VIF type https://review.opendev.org/751186 | |
| 08:44:52 | sean-k-mooney | ya i was going to say https://review.opendev.org/#/c/745605/3/nova/conf/scheduler.py | |
| 08:44:57 | bauzas | that's... disturbing | |
| 08:45:13 | sean-k-mooney | it makes sense from an information gain point of view | |
| 08:45:14 | bauzas | I'm almost sure this wasn't the case years ago | |
| 08:45:27 | bauzas | but anyway | |
| 08:45:33 | bauzas | in this case, this sounds legit | |
| 08:45:48 | bauzas | sean-k-mooney: if so, I'll review your deprecation patch | |
| 08:46:06 | sean-k-mooney | the point is in anycase to move both operation eventually to the placment query | |
| 08:46:25 | sean-k-mooney | since in that case ordering does not matter | |
| 08:47:01 | sean-k-mooney | at least form teh point of view of the query construction | |
| 08:48:19 | sean-k-mooney | bauzas: ill ping you with the update patch with just the AZfilter deprecation in a few once its ready | |
| 08:48:34 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Spell out 3pp in vmware undeprecation reno https://review.opendev.org/751188 | |
| 08:48:53 | sean-k-mooney | whats 3pp | |
| 08:49:06 | sean-k-mooney | oh hehe | |
| 08:49:13 | sean-k-mooney | third party | |
| 08:49:28 | sean-k-mooney | i see that was bauzas nit | |
| 08:51:39 | bauzas | sean-k-mooney: also, I left a comment on your deprecation patch, having the sheduler being underoptimized is not a performance issue | |
| 08:55:34 | sean-k-mooney | bauzas: thanks that said the orginal doc state that enabling the prefilter improves performance, so when i flip the docs to state its enabled by default now does it not make sense to now also state that reverting to the az filter woudl reduce performance | |
| 08:56:05 | sean-k-mooney | bauzas: i can remove that i just basically inverted the previous statement but if you dont think it adds value ill just drop it | |
| 08:56:25 | sean-k-mooney | bauzas: you were refering to your comment here right https://review.opendev.org/#/c/745605/3/doc/source/admin/availability-zones.rsthttps://review.opendev.org/#/c/745605/3/doc/source/admin/availability-zones.rst | |
| 08:57:54 | sean-k-mooney | the funny thing is that change was orginally just ment to be the az filter | |
| 08:58:39 | sean-k-mooney | i only added the compute filter when i was doing the doc update and was like waith there is a prefilter that "does the same thing" that can go too. i was wrong :( | |
| 09:13:31 | openstackgerrit | Federico Ressi proposed openstack/nova master: Log stack trace when fails launching a defined domain https://review.opendev.org/750652 | |
| 09:13:47 | openstackgerrit | Federico Ressi proposed openstack/nova master: Log stack trace when fails launching a defined domain https://review.opendev.org/750652 | |
| 09:14:58 | bauzas | sean-k-mooney: I know, we have a lot of docs that are wrong by saying "heh, prefilters are better for performance" | |
| 09:15:09 | bauzas | tbc: NO | |
| 09:15:21 | bauzas | a prefilter just allows you to get more candidates | |
| 09:15:53 | sean-k-mooney | bauzas: that is not true | |
| 09:16:13 | bauzas | filters perfomance is not an issue | |
| 09:16:15 | sean-k-mooney | bauzas: if you enable a prefilter and disable a schduler filter the db outperform python | |
| 09:16:25 | bauzas | sean-k-mooney: again, it's wrong | |
| 09:16:26 | sean-k-mooney | bauzas: in general yes | |
| 09:16:41 | bauzas | lots of folks (eg. CERN) looked at the scheduler performance | |
| 09:16:46 | sean-k-mooney | bauzas: we have see this form operators reporting that its faster | |
| 09:16:53 | bauzas | and none of them said that the filters were outperfomed | |
| 09:17:10 | bauzas | the problem was with the DB | |
| 09:17:13 | sean-k-mooney | i acutlly tought cern said it did | |
| 09:17:13 | bauzas | not the filters | |
| 09:17:20 | sean-k-mooney | i guess it woudl only help | |
| 09:17:35 | sean-k-mooney | as you said if we dont return the 1000 limit on allocitons | |
| 09:17:47 | sean-k-mooney | e.g. if it reduces the set below that | |
| 09:17:56 | bauzas | no, the problem is that NOBODY thought about the performance for getting more than 1000 candidates | |
| 09:18:04 | bauzas | I mean the API performance | |
| 09:18:17 | bauzas | so we said, "ok let's limit to 1000 candidates" | |
| 09:18:39 | bauzas | and then we said "oh but then shit, we won't then verify all the candidates" | |
| 09:18:53 | bauzas | and then that's why we started using prefilters | |
| 09:19:02 | sean-k-mooney | ya i know that i was there | |
| 09:19:14 | sean-k-mooney | and NOBODY is a strech | |
| 09:19:39 | bauzas | if the Placement API was able to provide *all* candidates, the performance in between checking all of them with a filter, or instead querying the Placement *before* wouldn't be differnet | |
| 09:19:40 | sean-k-mooney | we did discuss the subseting problem before we started using placment | |
| 09:20:02 | sean-k-mooney | bauzas: the different is the number of candiates pass to the schduler | |
| 09:20:17 | bauzas | meh, the ship has sailed either way | |
| 09:20:29 | sean-k-mooney | if we have 1000 nodes in 2 az of 500 | |
| 09:20:36 | bauzas | again, not a problem | |
| 09:20:38 | sean-k-mooney | the az prefilter will limit the repsoce to at most 500 | |
| 09:21:05 | sean-k-mooney | without it all 1000 nodes asumming all other thing are equall would pass to the scudler | |
| 09:21:28 | sean-k-mooney | and the az filter would have to filter the 500 nodes then | |