| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-06-28 | |||
| 06:04:57 | kashyap | frickler: Heh, I briefly wondered: if it's an upstream or a downstream-specific discussion | |
| 06:05:01 | sean-k-mooney | frickler: actully this was ment to be downstream but meh its a ooo bug | |
| 06:05:25 | sean-k-mooney | it was ment to be rhos-dev but i clicked the wrong tab | |
| 06:05:51 | sean-k-mooney | i have upstream on the top pane and downstream on the bottom | |
| 07:00:32 | gibi | o/ | |
| 07:17:52 | sean-k-mooney | gibi: o/ | |
| 07:21:38 | sean-k-mooney | frickler: so kashyap found https://review.opendev.org/c/openstack/puppet-tripleo/+/779313/ and i have filed the revert https://review.opendev.org/c/openstack/puppet-tripleo/+/847818 just to close the loop on the downstream topic i rasied upstream :) in case you were wondering | |
| 07:22:19 | sean-k-mooney | tl;dr is we forced netcat to workaround the lack of support in the rdo package for virt ssh-helper and since then netcat has been removed form teh ooo contianer | |
| 07:22:33 | sean-k-mooney | so we should just go back to auto now that we have support in the package | |
| 07:23:21 | whoami-rajat | hi #openstack-nova , would like to request reviews on my volume backend instance rebuild feature (3 patches in chain). the spec has merged and it was targeted for yoga cycle (but couldn't make it to the Feature freeze) so would like to get some early feedback | |
| 07:23:22 | whoami-rajat | https://review.opendev.org/c/openstack/nova/+/820368 | |
| 07:33:53 | sean-k-mooney | whoami-rajat: specs are approved per release so the imporant thing is that it was reappoved for zed | |
| 07:35:52 | whoami-rajat | sean-k-mooney, yep it was re-approved | |
| 07:36:22 | whoami-rajat | https://review.opendev.org/c/openstack/nova-specs/+/840155 | |
| 07:36:34 | sean-k-mooney | whoami-rajat: yep i remember | |
| 07:37:10 | sean-k-mooney | i have your review open. i need to do some downstream jira work quickly but ill review them when im done | |
| 07:37:24 | whoami-rajat | great, what i meant was it was approved last cycle as well but just wanted early reviews on it so i can make the deadline this time :) | |
| 07:37:29 | whoami-rajat | sean-k-mooney, thanks! | |
| 08:11:07 | frickler | sean-k-mooney: thx for the update. I really didn't want to drive you away, it just looked a bit out of context | |
| 09:03:38 | jkulik | Is there something in Nova that lets me express anti-affinity towards a group of hosts in the same rack? Use-case: we structure our HVs into racks and there might be customers not wanting their VM to run on the same HV (current anti-affinity I know about) and customers who don't want to run in the same rack as another VM, while staying in the same AZ. | |
| 09:06:09 | gibi | jkulik: I think there is no automatic way to express rack level (or host aggregate level) anti-affinity. What you can do is to create separate falvors for separate racks by referring the specific rack in the flavor | |
| 09:06:40 | gibi | but that does not scale well for many racks | |
| 09:06:44 | gibi | and many flavors | |
| 09:07:02 | gibi | I do remember discussing this on the last summit in berlin | |
| 09:07:24 | gibi | was it in the blazar session? | |
| 09:08:14 | gibi | https://etherpad.opendev.org/p/blazar-preemptible-and-gpus L64 | |
| 09:08:23 | gibi | or more like L57 | |
| 09:11:28 | sean-k-mooney | jkulik: we only supprot this at the az level | |
| 09:11:33 | sean-k-mooney | well no | |
| 09:11:36 | sean-k-mooney | at the host level | |
| 09:11:42 | sean-k-mooney | you can use AZs per rack | |
| 09:11:57 | sean-k-mooney | but we have no aggreate or az anti affintiy concept in nova | |
| 09:12:06 | sean-k-mooney | and its not simple to add | |
| 09:12:28 | gibi | sean-k-mooney: I had a suggestion in the above etherpad how to add it to nova | |
| 09:12:31 | sean-k-mooney | jkulik: if you want to isolate custoemr we have tenat isolation filters | |
| 09:13:00 | sean-k-mooney | gibi: we coudl do it via placment aggreate if we wanted too | |
| 09:13:13 | gibi | sean-k-mooney: nope | |
| 09:13:18 | sean-k-mooney | but i was just sayign we dont currently support it | |
| 09:13:25 | gibi | yes, we not currently support it | |
| 09:13:40 | gibi | the placement way would be harder than a nova way as placmenet aggregates has no metadata | |
| 09:13:49 | sean-k-mooney | we have also rejected it in the past as we did not want to add more orchestration to nova | |
| 09:13:50 | gibi | while nova aggregates has metadata | |
| 09:14:18 | sean-k-mooney | well nova aggreate are mapped to palcemnt aggreates | |
| 09:14:27 | gibi | without the metadata piece | |
| 09:14:27 | sean-k-mooney | but sure for the rack affinty that is doable | |
| 09:14:46 | gibi | we need the metadata to mark an aggregate as target for affinity/anti-affinity | |
| 09:15:14 | sean-k-mooney | az anti affintiy which was the otehr request cant be done that way but we can do rack/row/room antiaffinity with aggreate metadta and a filter | |
| 09:15:29 | sean-k-mooney | gibi: ya so if wee were to do this i would make the filter generic | |
| 09:15:36 | gibi | sure | |
| 09:15:53 | gibi | that was an afterthought for me as well | |
| 09:15:54 | sean-k-mooney | so that you can define a set of lables and then express the anti affintiy requirement in the flavor | |
| 09:16:06 | sean-k-mooney | like you do with ceph | |
| 09:16:14 | sean-k-mooney | and the palcment maps | |
| 09:16:31 | gibi | yep generic label based affinity/anti-affinity either via the flavor or via the server group API | |
| 09:16:41 | sean-k-mooney | yep | |
| 09:17:08 | gibi | so we only need some devs to propose a spec and then the implementation :D | |
| 09:17:11 | gibi | easy peasy :D | |
| 09:17:12 | sean-k-mooney | that i would be ok with but you could map the info to placment too perhaps as a step 2 | |
| 09:17:32 | gibi | sean-k-mooney: you mean extend the aggregate concept in placement with metadta? | |
| 09:17:58 | sean-k-mooney | no we could likely modle this with aggreates and custom_traits | |
| 09:18:14 | sean-k-mooney | i have not fully tought that out | |
| 09:18:17 | gibi | I don't like it, as then the trait needs to be on the all the PRs in the aggregate | |
| 09:18:20 | sean-k-mooney | but it feels like we shoudl be able to do that | |
| 09:18:42 | sean-k-mooney | well i was thinking more like how misc_share_via_aggrate works | |
| 09:18:42 | gibi | technically doable but I would extend the aggregate concept instead in placement | |
| 09:19:13 | sean-k-mooney | i would be fine with extendign aggreates too if we had a clean way to extend it | |
| 09:19:36 | sean-k-mooney | perhaps traits on aggretes or some other metadta but doing it in nova first is alot simpler as you said | |
| 09:19:54 | sean-k-mooney | so get it working (nova) then make it fast (placment) | |
| 09:20:32 | gibi | we are in agreement :) | |
| 09:20:33 | sean-k-mooney | jkulik: interested in working on ^ | |
| 09:20:57 | sean-k-mooney | jkulik: its not an uncommen request so we can also try an pitch it to our pm dowstream | |
| 09:21:25 | sean-k-mooney | but no promices they will go for it and we wont look at it until at least the A cycle in any case | |
| 09:21:56 | sean-k-mooney | jkulik: but if you had time to work on it we have 2 week to spec freeze and we could proably reivew it if you had somethign ready before code freeze | |
| 09:22:10 | sean-k-mooney | jkulik:realisticly though its proably an A or lather cycle thing | |
| 09:22:20 | sean-k-mooney | but it woudl be doable in an out of tree schduler filter today | |
| 09:22:57 | sean-k-mooney | at least if you took the flavor approch initaly to expres the affinity requiremnt | |
| 09:23:32 | sean-k-mooney | extending teh server group api woudl require an api change which need a spec and is not backportable due to how api versioning works | |
| 09:24:14 | sean-k-mooney | gibi: what was the blazar usecase/imporance of this | |
| 09:24:45 | sean-k-mooney | i assume some sort of ha/fault tollernace usecase | |
| 09:25:12 | sean-k-mooney | oh no performance | |
| 09:25:13 | sean-k-mooney | i.e. reserve three GPU VMs in the same rack, where possible (hard/soft affinity). | |
| 09:25:15 | sean-k-mooney | Specifically this is to reduce network latency (MPI/horovod), and unnessacery TOR switch network contention. | |
| 09:25:21 | gibi | yepp | |
| 09:25:27 | sean-k-mooney | i mean i gues it could be used for either | |
| 09:25:38 | gibi | but there was limited time so we did not dig deep | |
| 09:25:44 | sean-k-mooney | ack | |
| 09:26:04 | sean-k-mooney | so there soft vs hard "requirement" would be filter vs weigher | |
| 09:26:20 | sean-k-mooney | but most of the logic would be the same so i woudl proably just do both | |
| 09:26:23 | jkulik | sean-k-mooney: I don't think I can make it in 2 weeks, but would be able to do it in the longer run | |
| 09:26:26 | sean-k-mooney | and try and shre the code | |
| 09:26:51 | sean-k-mooney | jkulik: ack | |
| 09:28:40 | jkulik | custom scheduler filter would be my idea, too. but we'd need it in the server-group API anyways as having one flavor per rack really doesn't scale | |
| 09:29:29 | sean-k-mooney | not per rack | |
| 09:29:36 | sean-k-mooney | so in the flavor you would have a policy | |
| 09:29:54 | sean-k-mooney | aggreate_anti_affinity:rack | |
| 09:30:12 | sean-k-mooney | aggreate_anti_affinity:room | |
| 09:30:26 | sean-k-mooney | aggreate_anti_affinity:row | |
| 09:30:30 | jkulik | ah, but still. customers are already overwhelmed by the number of flavors as is. I don't think we can add more for this :) | |
| 09:30:57 | sean-k-mooney | ya that is why the server-group api woudl be beeter | |
| 09:31:18 | sean-k-mooney | ther is a hack that you could use if you promise not to mention my name as the source :P | |