| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-05-26 | |||
| 09:41:00 | sean-k-mooney | songwenping: we do not suppport multi create with nested resouce providres | |
| 09:42:01 | sean-k-mooney | songwenping: https://bugs.launchpad.net/nova/+bug/1874664 | |
| 09:42:49 | sean-k-mooney | songwenping: actully it looks like the patches are merged | |
| 09:43:01 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/723884/ and https://review.opendev.org/c/openstack/nova/+/723858/ | |
| 09:45:20 | sean-k-mooney | songwenping: i dont think this fully resolves the issue bauzas ^ | |
| 09:46:28 | sean-k-mooney | songwenping: ya reading them we only added documentation and test to show that it does not work in some cases | |
| 09:46:44 | sean-k-mooney | so form the comit | |
| 09:46:47 | sean-k-mooney | - you can ask for a flavor with 2 VGPU with --max 2 | |
| 09:46:49 | sean-k-mooney | - but you can't ask for a flavor with 4 VGPU and --max 2 | |
| 09:47:23 | kashyap | gibi: Hey, thanks a ton for trying! Also, what do you mean by "it should now do it as venv is separate"? I can't quite parse it :) | |
| 09:47:31 | kashyap | Pls rephrase it | |
| 09:47:55 | gibi | tox should build a new virtual env that is sealed off from python packages on the host | |
| 09:49:02 | kashyap | Yeap; thanks! | |
| 09:49:18 | gibi | interestingly the dockerfile in the bugreport shows the issue to me but my dockerfile doesn't | |
| 09:49:24 | gibi | craxy | |
| 09:53:03 | opendevreview | Rajat Dhasmana proposed openstack/nova-specs master: Repropose volume backed server rebuild spec https://review.opendev.org/c/openstack/nova-specs/+/840155 | |
| 09:54:09 | whoami-rajat | sean-k-mooney, ^^ updated the spec | |
| 09:57:34 | sean-k-mooney | whoami-rajat: thanks ill review it shortly | |
| 09:57:52 | whoami-rajat | thanks! | |
| 10:28:07 | songwenping | sean-k-mooney: no, not this bug. | |
| 10:28:54 | sean-k-mooney | songwenping: are you refering to creating a vm with multipel vgpus? | |
| 10:28:57 | songwenping | i create two vms with flavor that has 1 vgpu metadata | |
| 10:29:09 | sean-k-mooney | ok | |
| 10:29:44 | songwenping | libvirt allocate the same vgpu for these two vms | |
| 10:29:47 | sean-k-mooney | we only support doing that via 2 differnt api request today | |
| 10:30:12 | sean-k-mooney | we do not supprot creating 2 vms that use vgpus in the same api request | |
| 10:30:37 | songwenping | does the api check the request? | |
| 10:30:44 | sean-k-mooney | no | |
| 10:30:54 | sean-k-mooney | we will not block it at the api | |
| 10:31:20 | sean-k-mooney | we docuement it in the api as part of the previous bug | |
| 10:31:57 | songwenping | why donot we block it at api? | |
| 10:31:59 | sean-k-mooney | gibi: speaking of which if you use pci device in placment for now at least multi create will not work for the same reason | |
| 10:32:12 | sean-k-mooney | songwenping: becuase at the api we dont know fi it uses nested resouce provder or not | |
| 10:32:27 | sean-k-mooney | so we documented the limistaion | |
| 10:32:57 | songwenping | could you please find the doc? | |
| 10:35:18 | sean-k-mooney | songwenping: https://docs.openstack.org/nova/yoga/admin/virtual-gpu.html#caveats | |
| 10:36:59 | sean-k-mooney | we added this section https://review.opendev.org/c/openstack/nova/+/723884/6/doc/source/admin/virtual-gpu.rst | |
| 10:37:19 | sean-k-mooney | we coud make that more clear that we do not support multi creat a all but that is our current stnace | |
| 10:37:35 | sean-k-mooney | if a server uses neste reseouce proverder we do not support multi create | |
| 10:38:13 | sean-k-mooney | this applys to vgpu, cyborg, neuton bandwith or packet per second qos or any other usage of resouce form nested resouce providers | |
| 10:38:25 | sean-k-mooney | like pmem | |
| 10:50:54 | songwenping | sean-k-mooney: i see, thanks. | |
| 10:50:54 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix typos https://review.opendev.org/c/openstack/nova/+/843127 | |
| 10:59:38 | gibi | sean-k-mooney: I don't have the context why the multi create fail on nested allocations but I can take a look later | |
| 11:00:19 | sean-k-mooney | gibi: we just dont have any code for it to handel mappign the allcoation to the differnt instance properly | |
| 11:00:55 | sean-k-mooney | we woudl need to do multipel placment queries and orchstrate that in the schduler | |
| 11:00:56 | gibi | hm, I thought that normal multicreate uses the alternatives from the scheduler | |
| 11:01:47 | sean-k-mooney | it does i think but apprently that is not enough | |
| 11:02:21 | sean-k-mooney | this has basicaly never worked if you have nested rps | |
| 11:02:25 | sean-k-mooney | it work fine if you dont | |
| 11:04:54 | kevko | Hi nova team ! | |
| 11:05:15 | kevko | I would like to ask if there is any chance to do something as i will describe below : | |
| 11:06:01 | kevko | I really would like to schedule 2 instances in server group with anti-affinity policy to be scheduled to different host but also in different datacentre (or az) | |
| 11:06:35 | kevko | something similar as it is here | |
| 11:06:45 | kevko | https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/complex-anti-affinity-policies.html | |
| 11:07:06 | kevko | instead of max_server_per_host key should be something as max_server_per_az | |
| 11:07:20 | sean-k-mooney | kevko: that is not currently possibel or trivial to do | |
| 11:07:39 | kevko | yeah, i was checking code :/ | |
| 11:07:44 | sean-k-mooney | the request spec currently modles an az requst as a string | |
| 11:07:56 | kevko | it's mainly because of octavia and two datacentres .. | |
| 11:08:02 | sean-k-mooney | it cant accpet a list of stings so you cant do that in a singel request | |
| 11:08:22 | sean-k-mooney | kevko: in the past we have declared this out of scope of nova | |
| 11:08:36 | sean-k-mooney | kevko: as somethign a higher level orchestrator shoudl mange | |
| 11:08:48 | kevko | hmm | |
| 11:09:14 | kevko | it's mainly because of octavia loadbalancers .. | |
| 11:09:30 | sean-k-mooney | yep so octavia coudl orchestrate this | |
| 11:09:54 | sean-k-mooney | they already mange the vm creation | |
| 11:09:59 | sean-k-mooney | via calling nova | |
| 11:10:09 | sean-k-mooney | so they could make requsts with differnt AZs | |
| 11:11:04 | sean-k-mooney | i woudl have ot refresh my memory of server goups but i cant recall if you can add more server to the group in later request or not | |
| 11:11:21 | sean-k-mooney | if you can you could just group the loadbalnce request per AZ | |
| 11:12:01 | kevko | i am not sure if I understand | |
| 11:13:08 | sean-k-mooney | i was trying to recall if you needed to populate the server group in one go or coudl you make multiple server create request and resuse the same server group | |
| 11:13:12 | sean-k-mooney | i think its the latter | |
| 11:13:24 | sean-k-mooney | we have 2 or 3 ways to do anti afinity in nova | |
| 11:13:25 | kevko | you can set az in octavia ...an if you have az created from two datacentres ... server group and anti affinity will not work ..there is big chance that nova just schedule to two hosts ..but in same datacentre | |
| 11:13:35 | sean-k-mooney | and some requrie it all in one request and other allow it to be exteded. | |
| 11:14:11 | sean-k-mooney | kevko: right so ocativa could have an az anti affintiy policy in its api | |
| 11:14:18 | kevko | currently I think there is only one way to do it ... have 1 az from two hosts from two different datacentres ... but this is no way ..because we have hundreds of amphoraes :( | |
| 11:15:51 | sean-k-mooney | so what i would expect is you would creat at least 1 AZ per datacenter right | |
| 11:16:00 | sean-k-mooney | and then create a singel server group | |
| 11:16:05 | sean-k-mooney | with anti affintiy | |
| 11:16:32 | sean-k-mooney | and then in 2 differnet calls to nova create the vms for datacenter A and B | |
| 11:17:41 | kevko | it's not true i think | |
| 11:18:06 | sean-k-mooney | im describing the change you need to make to ocativa not how it works today | |
| 11:18:07 | kevko | anti-affinity works per host ...so there could be situation that in one datacentre instance will be scheduled on two hosts | |
| 11:18:21 | sean-k-mooney | kevko: correct | |
| 11:18:24 | kevko | yeah, | |
| 11:18:35 | sean-k-mooney | that is why you need ot use differnt AZ in the two diffenrt calls to nova | |
| 11:18:48 | kevko | octavia has availability_zone as one parameter in configuration | |
| 11:19:15 | sean-k-mooney | right if you want to supprot this you need to supprot this at the octavia api | |
| 11:19:15 | kevko | if this can be list and setup list of AZs where AZ is 2 hosts from two datacentres ... it will be work i think | |
| 11:19:18 | sean-k-mooney | not config driven | |
| 11:19:32 | sean-k-mooney | kevko: az in nova are nto the same as AZs in aws | |
| 11:19:41 | sean-k-mooney | the are jsut metadtaa on a a host aggreate. | |
| 11:20:01 | sean-k-mooney | there is not gurantee or expectiaon of a sperate falut domain | |
| 11:20:18 | sean-k-mooney | and conversily there is not guranteeor or expecation fo network connectivy between them | |
| 11:20:28 | sean-k-mooney | that is all determin by how you deploy your cloud | |
| 11:21:28 | kevko | yeah i know that it is only metadata defined for host (or how to say :P ...i really don't know how it looks like in deep) | |
| 11:21:48 | sean-k-mooney | its a strign defiend on a host aggareate | |