| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-06-28 | |||
| 09:13:25 | gibi | yes, we not currently support it | |
| 09:13:40 | gibi | the placement way would be harder than a nova way as placmenet aggregates has no metadata | |
| 09:13:49 | sean-k-mooney | we have also rejected it in the past as we did not want to add more orchestration to nova | |
| 09:13:50 | gibi | while nova aggregates has metadata | |
| 09:14:18 | sean-k-mooney | well nova aggreate are mapped to palcemnt aggreates | |
| 09:14:27 | gibi | without the metadata piece | |
| 09:14:27 | sean-k-mooney | but sure for the rack affinty that is doable | |
| 09:14:46 | gibi | we need the metadata to mark an aggregate as target for affinity/anti-affinity | |
| 09:15:14 | sean-k-mooney | az anti affintiy which was the otehr request cant be done that way but we can do rack/row/room antiaffinity with aggreate metadta and a filter | |
| 09:15:29 | sean-k-mooney | gibi: ya so if wee were to do this i would make the filter generic | |
| 09:15:36 | gibi | sure | |
| 09:15:53 | gibi | that was an afterthought for me as well | |
| 09:15:54 | sean-k-mooney | so that you can define a set of lables and then express the anti affintiy requirement in the flavor | |
| 09:16:06 | sean-k-mooney | like you do with ceph | |
| 09:16:14 | sean-k-mooney | and the palcment maps | |
| 09:16:31 | gibi | yep generic label based affinity/anti-affinity either via the flavor or via the server group API | |
| 09:16:41 | sean-k-mooney | yep | |
| 09:17:08 | gibi | so we only need some devs to propose a spec and then the implementation :D | |
| 09:17:11 | gibi | easy peasy :D | |
| 09:17:12 | sean-k-mooney | that i would be ok with but you could map the info to placment too perhaps as a step 2 | |
| 09:17:32 | gibi | sean-k-mooney: you mean extend the aggregate concept in placement with metadta? | |
| 09:17:58 | sean-k-mooney | no we could likely modle this with aggreates and custom_traits | |
| 09:18:14 | sean-k-mooney | i have not fully tought that out | |
| 09:18:17 | gibi | I don't like it, as then the trait needs to be on the all the PRs in the aggregate | |
| 09:18:20 | sean-k-mooney | but it feels like we shoudl be able to do that | |
| 09:18:42 | sean-k-mooney | well i was thinking more like how misc_share_via_aggrate works | |
| 09:18:42 | gibi | technically doable but I would extend the aggregate concept instead in placement | |
| 09:19:13 | sean-k-mooney | i would be fine with extendign aggreates too if we had a clean way to extend it | |
| 09:19:36 | sean-k-mooney | perhaps traits on aggretes or some other metadta but doing it in nova first is alot simpler as you said | |
| 09:19:54 | sean-k-mooney | so get it working (nova) then make it fast (placment) | |
| 09:20:32 | gibi | we are in agreement :) | |
| 09:20:33 | sean-k-mooney | jkulik: interested in working on ^ | |
| 09:20:57 | sean-k-mooney | jkulik: its not an uncommen request so we can also try an pitch it to our pm dowstream | |
| 09:21:25 | sean-k-mooney | but no promices they will go for it and we wont look at it until at least the A cycle in any case | |
| 09:21:56 | sean-k-mooney | jkulik: but if you had time to work on it we have 2 week to spec freeze and we could proably reivew it if you had somethign ready before code freeze | |
| 09:22:10 | sean-k-mooney | jkulik:realisticly though its proably an A or lather cycle thing | |
| 09:22:20 | sean-k-mooney | but it woudl be doable in an out of tree schduler filter today | |
| 09:22:57 | sean-k-mooney | at least if you took the flavor approch initaly to expres the affinity requiremnt | |
| 09:23:32 | sean-k-mooney | extending teh server group api woudl require an api change which need a spec and is not backportable due to how api versioning works | |
| 09:24:14 | sean-k-mooney | gibi: what was the blazar usecase/imporance of this | |
| 09:24:45 | sean-k-mooney | i assume some sort of ha/fault tollernace usecase | |
| 09:25:12 | sean-k-mooney | oh no performance | |
| 09:25:13 | sean-k-mooney | i.e. reserve three GPU VMs in the same rack, where possible (hard/soft affinity). | |
| 09:25:15 | sean-k-mooney | Specifically this is to reduce network latency (MPI/horovod), and unnessacery TOR switch network contention. | |
| 09:25:21 | gibi | yepp | |
| 09:25:27 | sean-k-mooney | i mean i gues it could be used for either | |
| 09:25:38 | gibi | but there was limited time so we did not dig deep | |
| 09:25:44 | sean-k-mooney | ack | |
| 09:26:04 | sean-k-mooney | so there soft vs hard "requirement" would be filter vs weigher | |
| 09:26:20 | sean-k-mooney | but most of the logic would be the same so i woudl proably just do both | |
| 09:26:23 | jkulik | sean-k-mooney: I don't think I can make it in 2 weeks, but would be able to do it in the longer run | |
| 09:26:26 | sean-k-mooney | and try and shre the code | |
| 09:26:51 | sean-k-mooney | jkulik: ack | |
| 09:28:40 | jkulik | custom scheduler filter would be my idea, too. but we'd need it in the server-group API anyways as having one flavor per rack really doesn't scale | |
| 09:29:29 | sean-k-mooney | not per rack | |
| 09:29:36 | sean-k-mooney | so in the flavor you would have a policy | |
| 09:29:54 | sean-k-mooney | aggreate_anti_affinity:rack | |
| 09:30:12 | sean-k-mooney | aggreate_anti_affinity:room | |
| 09:30:26 | sean-k-mooney | aggreate_anti_affinity:row | |
| 09:30:30 | jkulik | ah, but still. customers are already overwhelmed by the number of flavors as is. I don't think we can add more for this :) | |
| 09:30:57 | sean-k-mooney | ya that is why the server-group api woudl be beeter | |
| 09:31:18 | sean-k-mooney | ther is a hack that you could use if you promise not to mention my name as the source :P | |
| 09:31:26 | jkulik | :D | |
| 09:31:37 | sean-k-mooney | so you could use server tags or instance metadta for now | |
| 09:31:42 | bauzas | I wonder how feasible a weigher could be | |
| 09:32:15 | bauzas | we could weigh per aggregatesd | |
| 09:32:17 | sean-k-mooney | so you could add aggreate_anti_affinity_* set of server tags | |
| 09:32:32 | sean-k-mooney | and then look at those in the filter/weigher | |
| 09:32:34 | bauzas | each host getting the same weigh | |
| 09:32:49 | jkulik | sean-k-mooney: that sounds wrong :D but thanks for the idea | |
| 09:32:51 | sean-k-mooney | bauzas: that is doable yes | |
| 09:33:32 | sean-k-mooney | jkulik: tags are somethimes used for this in other out of tree implemenation like tripleos instance ha feature | |
| 09:33:35 | jkulik | speaking of the server-group API: we've extended it to allow adding servers to server-groups after they're spawned. this can be helpful if you need to spawn a new instance anti-affin to a previously spawned one, where you didn't know that requirement, yet. | |
| 09:33:42 | bauzas | I'm still confused by the weights | |
| 09:33:48 | bauzas | but | |
| 09:33:48 | sean-k-mooney | its not how the api is inteded to be used but still | |
| 09:33:52 | jkulik | is that something that has a chance upstream, if we write a spec for it? | |
| 09:33:56 | bauzas | you could pass a hint | |
| 09:34:09 | bauzas | and then have a weigher looking up at the hint | |
| 09:34:14 | sean-k-mooney | schduler hint ya but that also versioned | |
| 09:34:26 | sean-k-mooney | bauzas: oh for the pack vs spread policy | |
| 09:34:33 | sean-k-mooney | for weighers | |
| 09:34:34 | bauzas | yup | |
| 09:35:06 | sean-k-mooney | ya so i was thining the weigher woudl jsut look at the metadta in the server-group policy | |
| 09:35:18 | sean-k-mooney | and ignore the multiplpere sign | |
| 09:35:31 | sean-k-mooney | or we woudl just set min =0 for the config option | |
| 09:35:47 | bauzas | I'm not in a favor of adding more in the existing server group API but... | |
| 09:35:51 | sean-k-mooney | so the multiplper for the weigher woudl afffect only magnitude | |
| 09:36:07 | bauzas | I guess we need to agree on the use case | |
| 09:36:08 | sean-k-mooney | well server group api is preferable to schduler hit | |
| 09:36:36 | bauzas | sean-k-mooney: from an UX, I think so | |
| 09:36:51 | bauzas | but, you know how much I like our server group implementations | |
| 09:37:54 | bauzas | like --min 2 --hint spread_my_stuff | |
| 09:45:31 | gibi | bauzas: you can look at the soft-anti-affinity weigher for reference how to do it | |
| 09:46:00 | gibi | ie how to do weigher for soft things | |
| 09:47:01 | gibi | jkulik: adding existing instance to server group needs a decision from a) allow adding an instance only if the policy is valid b)allow adding instance even if the policy is not valid yet but will be at the next move | |
| 09:47:28 | gibi | jkulik: I think in the past we was not able to agree on which way we should go | |
| 09:47:32 | gibi | but you can try again | |
| 09:53:40 | jkulik | we've opted for a) ... | |
| 09:53:51 | jkulik | where would be the right point to discuss this? | |