| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-26 | |||
| 10:29:09 | sean-k-mooney | ok | |
| 10:29:44 | songwenping | libvirt allocate the same vgpu for these two vms | |
| 10:29:47 | sean-k-mooney | we only support doing that via 2 differnt api request today | |
| 10:30:12 | sean-k-mooney | we do not supprot creating 2 vms that use vgpus in the same api request | |
| 10:30:37 | songwenping | does the api check the request? | |
| 10:30:44 | sean-k-mooney | no | |
| 10:30:54 | sean-k-mooney | we will not block it at the api | |
| 10:31:20 | sean-k-mooney | we docuement it in the api as part of the previous bug | |
| 10:31:57 | songwenping | why donot we block it at api? | |
| 10:31:59 | sean-k-mooney | gibi: speaking of which if you use pci device in placment for now at least multi create will not work for the same reason | |
| 10:32:12 | sean-k-mooney | songwenping: becuase at the api we dont know fi it uses nested resouce provder or not | |
| 10:32:27 | sean-k-mooney | so we documented the limistaion | |
| 10:32:57 | songwenping | could you please find the doc? | |
| 10:35:18 | sean-k-mooney | songwenping: https://docs.openstack.org/nova/yoga/admin/virtual-gpu.html#caveats | |
| 10:36:59 | sean-k-mooney | we added this section https://review.opendev.org/c/openstack/nova/+/723884/6/doc/source/admin/virtual-gpu.rst | |
| 10:37:19 | sean-k-mooney | we coud make that more clear that we do not support multi creat a all but that is our current stnace | |
| 10:37:35 | sean-k-mooney | if a server uses neste reseouce proverder we do not support multi create | |
| 10:38:13 | sean-k-mooney | this applys to vgpu, cyborg, neuton bandwith or packet per second qos or any other usage of resouce form nested resouce providers | |
| 10:38:25 | sean-k-mooney | like pmem | |
| 10:50:54 | songwenping | sean-k-mooney: i see, thanks. | |
| 10:50:54 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix typos https://review.opendev.org/c/openstack/nova/+/843127 | |
| 10:59:38 | gibi | sean-k-mooney: I don't have the context why the multi create fail on nested allocations but I can take a look later | |
| 11:00:19 | sean-k-mooney | gibi: we just dont have any code for it to handel mappign the allcoation to the differnt instance properly | |
| 11:00:55 | sean-k-mooney | we woudl need to do multipel placment queries and orchstrate that in the schduler | |
| 11:00:56 | gibi | hm, I thought that normal multicreate uses the alternatives from the scheduler | |
| 11:01:47 | sean-k-mooney | it does i think but apprently that is not enough | |
| 11:02:21 | sean-k-mooney | this has basicaly never worked if you have nested rps | |
| 11:02:25 | sean-k-mooney | it work fine if you dont | |
| 11:04:54 | kevko | Hi nova team ! | |
| 11:05:15 | kevko | I would like to ask if there is any chance to do something as i will describe below : | |
| 11:06:01 | kevko | I really would like to schedule 2 instances in server group with anti-affinity policy to be scheduled to different host but also in different datacentre (or az) | |
| 11:06:35 | kevko | something similar as it is here | |
| 11:06:45 | kevko | https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/complex-anti-affinity-policies.html | |
| 11:07:06 | kevko | instead of max_server_per_host key should be something as max_server_per_az | |
| 11:07:20 | sean-k-mooney | kevko: that is not currently possibel or trivial to do | |
| 11:07:39 | kevko | yeah, i was checking code :/ | |
| 11:07:44 | sean-k-mooney | the request spec currently modles an az requst as a string | |
| 11:07:56 | kevko | it's mainly because of octavia and two datacentres .. | |
| 11:08:02 | sean-k-mooney | it cant accpet a list of stings so you cant do that in a singel request | |
| 11:08:22 | sean-k-mooney | kevko: in the past we have declared this out of scope of nova | |
| 11:08:36 | sean-k-mooney | kevko: as somethign a higher level orchestrator shoudl mange | |
| 11:08:48 | kevko | hmm | |
| 11:09:14 | kevko | it's mainly because of octavia loadbalancers .. | |
| 11:09:30 | sean-k-mooney | yep so octavia coudl orchestrate this | |
| 11:09:54 | sean-k-mooney | they already mange the vm creation | |
| 11:09:59 | sean-k-mooney | via calling nova | |
| 11:10:09 | sean-k-mooney | so they could make requsts with differnt AZs | |
| 11:11:04 | sean-k-mooney | i woudl have ot refresh my memory of server goups but i cant recall if you can add more server to the group in later request or not | |
| 11:11:21 | sean-k-mooney | if you can you could just group the loadbalnce request per AZ | |
| 11:12:01 | kevko | i am not sure if I understand | |
| 11:13:08 | sean-k-mooney | i was trying to recall if you needed to populate the server group in one go or coudl you make multiple server create request and resuse the same server group | |
| 11:13:12 | sean-k-mooney | i think its the latter | |
| 11:13:24 | sean-k-mooney | we have 2 or 3 ways to do anti afinity in nova | |
| 11:13:25 | kevko | you can set az in octavia ...an if you have az created from two datacentres ... server group and anti affinity will not work ..there is big chance that nova just schedule to two hosts ..but in same datacentre | |
| 11:13:35 | sean-k-mooney | and some requrie it all in one request and other allow it to be exteded. | |
| 11:14:11 | sean-k-mooney | kevko: right so ocativa could have an az anti affintiy policy in its api | |
| 11:14:18 | kevko | currently I think there is only one way to do it ... have 1 az from two hosts from two different datacentres ... but this is no way ..because we have hundreds of amphoraes :( | |
| 11:15:51 | sean-k-mooney | so what i would expect is you would creat at least 1 AZ per datacenter right | |
| 11:16:00 | sean-k-mooney | and then create a singel server group | |
| 11:16:05 | sean-k-mooney | with anti affintiy | |
| 11:16:32 | sean-k-mooney | and then in 2 differnet calls to nova create the vms for datacenter A and B | |
| 11:17:41 | kevko | it's not true i think | |
| 11:18:06 | sean-k-mooney | im describing the change you need to make to ocativa not how it works today | |
| 11:18:07 | kevko | anti-affinity works per host ...so there could be situation that in one datacentre instance will be scheduled on two hosts | |
| 11:18:21 | sean-k-mooney | kevko: correct | |
| 11:18:24 | kevko | yeah, | |
| 11:18:35 | sean-k-mooney | that is why you need ot use differnt AZ in the two diffenrt calls to nova | |
| 11:18:48 | kevko | octavia has availability_zone as one parameter in configuration | |
| 11:19:15 | sean-k-mooney | right if you want to supprot this you need to supprot this at the octavia api | |
| 11:19:15 | kevko | if this can be list and setup list of AZs where AZ is 2 hosts from two datacentres ... it will be work i think | |
| 11:19:18 | sean-k-mooney | not config driven | |
| 11:19:32 | sean-k-mooney | kevko: az in nova are nto the same as AZs in aws | |
| 11:19:41 | sean-k-mooney | the are jsut metadtaa on a a host aggreate. | |
| 11:20:01 | sean-k-mooney | there is not gurantee or expectiaon of a sperate falut domain | |
| 11:20:18 | sean-k-mooney | and conversily there is not guranteeor or expecation fo network connectivy between them | |
| 11:20:28 | sean-k-mooney | that is all determin by how you deploy your cloud | |
| 11:21:28 | kevko | yeah i know that it is only metadata defined for host (or how to say :P ...i really don't know how it looks like in deep) | |
| 11:21:48 | sean-k-mooney | its a strign defiend on a host aggareate | |
| 11:21:55 | sean-k-mooney | and then host are mapped to aggreates | |
| 11:23:33 | kevko | what about regions ? | |
| 11:23:38 | kevko | can be helpfull ? | |
| 11:23:42 | sean-k-mooney | the do not existin in nova | |
| 11:23:51 | sean-k-mooney | they are a keystone only construct | |
| 11:24:00 | kevko | yeah ... correct | |
| 11:24:03 | kevko | cells ? | |
| 11:24:24 | sean-k-mooney | cells are internal to nova | |
| 11:24:36 | sean-k-mooney | and not expsoed at the api to end users | |
| 11:24:47 | sean-k-mooney | they are a way to shared the db and message bus | |
| 11:24:52 | kevko | yeah i know .. | |
| 11:24:58 | sean-k-mooney | and are explictly not for fault tollerance | |
| 11:25:08 | sean-k-mooney | but they can kind fo help in a way | |
| 11:25:15 | kevko | so currently i don't have way how to deal with it | |
| 11:25:45 | sean-k-mooney | we coudl support az anti affintiy in nova | |
| 11:25:56 | sean-k-mooney | but we have rejected it in the past the last 2 times it came up | |
| 11:27:24 | sean-k-mooney | kevko: filters and weigher can be added out fo tree by the way | |
| 11:28:18 | sean-k-mooney | but all instnace will be mapped ot an az even if oenis not requested | |
| 11:28:49 | sean-k-mooney | so really if you want to use AZs to map to datacenters | |
| 11:29:19 | sean-k-mooney | then octavia need to accpet the AZ as an api parameter when creating loadblancers | |
| 11:29:48 | sean-k-mooney | and if you want ti to supprot spreaing loadbalncer between azs then it allso need to supprot doing that iself really | |
| 11:30:56 | sean-k-mooney | since AZ are not really visable to neutron by the way there is no guarentee at teh api level that a newton network can span azs | |