| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-06-22 | |||
| 17:03:12 | bauzas | and defer to cyborg-agent binding if so | |
| 17:03:42 | bauzas | or just call libvirt if the allocated RP was owned by nova | |
| 17:04:12 | opendevreview | Stephen Finucane proposed openstack/nova master: scheduler: Remove 'USES_ALLOCATION_CANDIDATES' https://review.opendev.org/c/openstack/nova/+/773640 | |
| 17:04:12 | opendevreview | Stephen Finucane proposed openstack/nova master: scheduler: Remove 'hosts_up' https://review.opendev.org/c/openstack/nova/+/773641 | |
| 17:04:13 | opendevreview | Stephen Finucane proposed openstack/nova master: trivial: Remove FakeScheduler (for realz) https://review.opendev.org/c/openstack/nova/+/773642 | |
| 17:04:13 | opendevreview | Stephen Finucane proposed openstack/nova master: scheduler: Merge 'FilterScheduler' into base class https://review.opendev.org/c/openstack/nova/+/773643 | |
| 17:04:14 | opendevreview | Stephen Finucane proposed openstack/nova master: docs: Drop references to non-filter scheduler drivers https://review.opendev.org/c/openstack/nova/+/773645 | |
| 17:04:14 | opendevreview | Stephen Finucane proposed openstack/nova master: scheduler: Merge driver into manager https://review.opendev.org/c/openstack/nova/+/773644 | |
| 17:04:15 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Merge 'test_utils', 'test_scheduler_utils' https://review.opendev.org/c/openstack/nova/+/773646 | |
| 17:04:16 | opendevreview | Stephen Finucane proposed openstack/nova master: conf: Remove deprecated aliases https://review.opendev.org/c/openstack/nova/+/773647 | |
| 17:04:16 | opendevreview | Stephen Finucane proposed openstack/nova master: scheduler: 'USES_ALLOCATION_CANDIDATES' removal cleanup https://review.opendev.org/c/openstack/nova/+/797513 | |
| 17:04:20 | bauzas | sean-k-mooney: this isn't a scheduling decision until you wanna explicitely shard your resources between nova and cyborg (the non-mixed case) | |
| 17:04:40 | bauzas | and for the non-mixed case, we can have different RCs | |
| 17:04:54 | bauzas | instead of a owner trait | |
| 17:05:30 | sean-k-mooney | bauzas: if the request is in the flavor and ti does not come form a device profile then nova shoudl ensure it does not come form an nova created RP | |
| 17:06:18 | sean-k-mooney | the instresting part is if you have resouces:vgpu=1 | |
| 17:06:26 | bauzas | the mixed case | |
| 17:06:27 | sean-k-mooney | which today can only come form nova | |
| 17:06:38 | bauzas | the non-mixed case can be solved with custom RCs | |
| 17:06:57 | sean-k-mooney | and the question then is what hsoudl the bhavior be in the absense of a trait | |
| 17:07:09 | sean-k-mooney | anyway let talk tommorow | |
| 17:07:13 | bauzas | for the mixed case, you can admit that it's OK for a spawn to go to the nova-compute | |
| 17:07:27 | bauzas | and then the nova-compute can either call cyborg-agent to bind | |
| 17:07:33 | bauzas | or, call libvirt to just spawn | |
| 17:07:34 | sean-k-mooney | no that is not ok | |
| 17:07:48 | sean-k-mooney | well maybe | |
| 17:07:55 | sean-k-mooney | but it would be rather complicated | |
| 17:08:06 | sean-k-mooney | ill think about it tonight | |
| 17:08:23 | bauzas | we already get allocations before we call libvirt | |
| 17:08:30 | sean-k-mooney | but it would require change not currently in the spec | |
| 17:08:40 | bauzas | so we already know the allocated RPs | |
| 17:09:01 | sean-k-mooney | bauzas: correct but we will need the condurtor to determin if the rp in the allocation is owned by nova or cyborg | |
| 17:09:07 | bauzas | it's just the nova-compute service which doesn't care and directly passes the allocations as arguments | |
| 17:09:14 | sean-k-mooney | and then have it start the bidning before we call the compute node | |
| 17:09:14 | opendevreview | Merged openstack/nova master: Fix max cpu topologies with numa affinity https://review.opendev.org/c/openstack/nova/+/769614 | |
| 17:09:31 | sean-k-mooney | sicne we do the arq binding in the conductor | |
| 17:09:34 | bauzas | sean-k-mooney: that's not the workflow I see in https://review.opendev.org/c/openstack/nova-specs/+/780452/8/specs/xena/approved/support-vGPU-nova-cyborg-interaction.rst#132 | |
| 17:09:42 | sean-k-mooney | and without a device procfile we will not know how to create teh arq | |
| 17:09:44 | bauzas | holy shit | |
| 17:09:57 | bauzas | 'nova's is both the conductor and the compute service | |
| 17:09:59 | sean-k-mooney | so we cant make that work without changing the cybrog api | |
| 17:10:03 | bauzas | in the workflow diagram | |
| 17:10:23 | bauzas | well, we get allocations in the conductor, right? | |
| 17:10:39 | bauzas | in order to know which compute services to call, right? | |
| 17:10:41 | sean-k-mooney | yes and we create and bind arqs in the conductor too | |
| 17:10:57 | bauzas | ok, so it's just an extra bit | |
| 17:10:59 | sean-k-mooney | but to create an arq we need a device profile | |
| 17:11:34 | sean-k-mooney | so if we have resouce:vgpu in the flavor it cant be used to create a cybrog ARQ today as we dont have a device profile | |
| 17:11:38 | bauzas | sure, I see it in the diagram | |
| 17:12:05 | bauzas | we create a device profile before we call scheduling, right? | |
| 17:12:12 | sean-k-mooney | so right now a refilter can eailar determin if the vgpu request is from nvoa or cyborg | |
| 17:12:29 | sean-k-mooney | bauzas: yes its a resouce in the cyborg api | |
| 17:12:32 | bauzas | because we never mix | |
| 17:12:49 | sean-k-mooney | so the admihn create a device profile and then creates a flavor that refrence s it | |
| 17:12:59 | sean-k-mooney | bauzas: a device profile is like a pci aliasa | |
| 17:13:03 | bauzas | yup, I got it | |
| 17:13:19 | sean-k-mooney | your are 13 mins passed your hard stop :) | |
| 17:13:41 | bauzas | but it's premature I think to get a device profile based on an assumption that a flavor asked for a vgpu | |
| 17:14:04 | sean-k-mooney | bauzas: that not how that works | |
| 17:14:09 | bauzas | you should get the device profile only if the allocated resource is owned by cyborg | |
| 17:14:15 | bauzas | sean-k-mooney: I know | |
| 17:14:28 | sean-k-mooney | right now the only way to request cybrog resoruce is via a device profile | |
| 17:14:44 | sean-k-mooney | supprot cybrog via the resouce: sysntax is totallyout of scope | |
| 17:14:47 | bauzas | sean-k-mooney: I understand and I don't wanna change this | |
| 17:15:01 | bauzas | ok, ok, | |
| 17:15:27 | bauzas | so a flavor is set with a device profile that's turned into a resources query by a prefilter, right? | |
| 17:15:49 | sean-k-mooney | yes | |
| 17:15:52 | bauzas | in this case, there is ZERO possibility to mix resources | |
| 17:16:01 | bauzas | whatever the resource is | |
| 17:16:01 | sean-k-mooney | well not refilter i thik we have explict logic for cyborg | |
| 17:16:15 | sean-k-mooney | today correct | |
| 17:16:26 | bauzas | so the mixed case is impossible today | |
| 17:16:30 | sean-k-mooney | yes | |
| 17:16:42 | bauzas | so, we're discussing about non-mixed resources | |
| 17:16:48 | bauzas | which can be solved with custom RCs | |
| 17:16:55 | sean-k-mooney | unless tehy break one fo the rule we have about palcment useage | |
| 17:17:19 | sean-k-mooney | bauzas: it can but it does not have to be | |
| 17:17:40 | bauzas | sean-k-mooney: the other possibility implies a huge change for existing deployments, right? | |
| 17:17:47 | sean-k-mooney | no | |
| 17:17:50 | sean-k-mooney | not really | |
| 17:18:08 | sean-k-mooney | just that we would add a trait to all nova rps | |
| 17:18:17 | sean-k-mooney | and we would automatically add it with a prefilter | |
| 17:18:28 | bauzas | yup, I got this, this is a technical answer | |
| 17:18:53 | bauzas | which prevents us to discuss about alternatives | |
| 17:19:06 | sean-k-mooney | we can discuss alternitves | |
| 17:19:18 | bauzas | and the one I'm proposing (again, for the non-mixed case) doesn't imply upgrade impacts | |
| 17:19:19 | sean-k-mooney | but we have discsuted this for 2+ cycle now | |
| 17:19:43 | sean-k-mooney | correct but it means we dotn need os-resouce classes | |
| 17:19:54 | sean-k-mooney | since resouce classes will have to be different per service | |
| 17:20:06 | sean-k-mooney | so its not free | |
| 17:20:51 | sean-k-mooney | bauzas: if we ant differnt service to use diffeernt resouces classes for the same resouce class we can do that | |
| 17:20:54 | bauzas | the whole situation is that we need some scheduling query for telling "yo' I really want cyborg" | |
| 17:20:59 | sean-k-mooney | but its restricting the usfulness of placment | |
| 17:21:18 | sean-k-mooney | or i really want nova but yes | |
| 17:21:59 | sean-k-mooney | which is why the ownwer trait was proposed so we could make a qualitave statmetn about the resouce provier | |
| 17:22:03 | bauzas | nope, in general, you don't need to tell placement "give me nova stuff" | |
| 17:22:28 | bauzas | because if you ask for nova resouces, you get nova stuff | |
| 17:22:36 | sean-k-mooney | you may not | |
| 17:22:50 | bauzas | agreed | |
| 17:22:52 | sean-k-mooney | zun could create RPs with cpu ram and disk | |