| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-12-05 | |||
| 21:15:56 | efried | Thanks :) | |
| 21:38:38 | efried | mriedem: reserved=total for which resource? And how do you stop the virt driver from overwriting that? (Re ML post about CERN workaround for low alloc candidates limit) | |
| 21:40:24 | mriedem | VCPU? all of them? | |
| 21:40:27 | dansmith | for the compute node | |
| 21:40:35 | dansmith | for all of them yeah | |
| 21:40:57 | mriedem | the compute would probably need to know if it's service is disabled and if so, not ovewrite it | |
| 21:41:04 | dansmith | mriedem: I think we discussed just having an rpc cast to compute to have it do it, vs. a periodic | |
| 21:41:04 | mriedem | *its | |
| 21:41:22 | mriedem | dansmith: sure, but the next update_available_resource periodic would overwrite it | |
| 21:41:30 | mriedem | when reporting inventory | |
| 21:41:34 | dansmith | the compute can stop calling the virt driver's update method if it's disabled I would think | |
| 21:41:36 | sean-k-mooney | efried: the virt dirver really should not be touching the reserved value excpet when its first creating the RP | |
| 21:41:46 | mriedem | sean-k-mooney: the ironic driver does all the time | |
| 21:41:49 | dansmith | sean-k-mooney: uh, why? | |
| 21:41:49 | mriedem | when the node is being cleaned | |
| 21:41:57 | dansmith | the reserved amount is owned by the virt driver, IMHO | |
| 21:42:02 | mriedem | that's exactly why the placement api change was made so that reserved can equal total | |
| 21:42:04 | dansmith | only the virt driver knows what it should be | |
| 21:42:25 | sean-k-mooney | dansmith: well for the same reason as teh have the cpu allocation ratios vs inital cpu allocation ratios spec | |
| 21:42:28 | efried | agreed, all the inventory values ought to be owned by the virt driver, period. We start making exceptions, we end up with messes like allocation ratio... and reserved. | |
| 21:42:36 | sean-k-mooney | controling via api or config | |
| 21:42:57 | efried | but making that contract stick for this kind of workaround is going to be tricksy. | |
| 21:42:58 | dansmith | sean-k-mooney: that's not an example that helps your case I think :) | |
| 21:43:24 | mriedem | the alternative was a trait i think | |
| 21:43:32 | mriedem | and a pre-request placement filter in nova-scheduler | |
| 21:43:32 | efried | unless we do it with a handy-dandy provider config yaml file https://review.openstack.org/#/c/612497/ | |
| 21:43:35 | mriedem | or something like that | |
| 21:43:51 | mriedem | this isn't blues clues | |
| 21:43:55 | dansmith | mriedem: yeah, that's also an option | |
| 21:44:28 | mriedem | so api sets a trait (or removes it), virt doesn't overwrite it, and scheduler filters on it (essentially it becomes the ComputeFilter) | |
| 21:44:31 | efried | omg, now you totally remind me of Steve from blues clues. | |
| 21:44:35 | sean-k-mooney | efried: provider config yaml for what? | |
| 21:44:43 | mriedem | efried: there were a couple of steves | |
| 21:44:48 | mriedem | or was it one and then another guy? | |
| 21:44:50 | dansmith | mriedem: not scheduler filters, request filter | |
| 21:44:51 | efried | no, there was only one steve. | |
| 21:44:55 | sean-k-mooney | how does that allw you to set it via the api and not have the compute overriede | |
| 21:44:56 | efried | the other guy was... | |
| 21:44:59 | efried | joe? | |
| 21:45:07 | mriedem | yes joe | |
| 21:45:17 | mriedem | http://www.gstatic.com/tv/thumb/persons/250022/250022_v9_ba.jpg | |
| 21:45:48 | mriedem | sean-k-mooney: b/c the compute/virt driver isn't supposed to overwrite externally set traits | |
| 21:46:01 | mriedem | and the api can set traits just like it mirrors aggregates | |
| 21:46:06 | sean-k-mooney | anyway if we say only the entity that created the RP may set the reserved value on the invetories im fine with that if we document it | |
| 21:46:58 | dansmith | I thought there was some other reason for doing the rpc cast on disable, but I can't remember what it was | |
| 21:47:46 | dansmith | maybe so ironic driver can do something? | |
| 21:47:52 | sean-k-mooney | on disableing a compute node. i think some dirver can take actions on disable and ironic might be one of them but i dont remember eiter | |
| 21:48:14 | efried | <do a right thing to disable the inventory> | |
| 21:48:14 | efried | if self.disabled: | |
| 21:48:14 | efried | we can ask each driver's upt to do | |
| 21:48:39 | dansmith | efried: we still have to do it as soon as the api is called though, | |
| 21:48:44 | dansmith | so waiting for the next periodic is not good enough | |
| 21:48:51 | efried | mm | |
| 21:49:08 | dansmith | oh yeah, so.. | |
| 21:49:11 | mriedem | right i could disable an entire cell, | |
| 21:49:11 | efried | do we have a ComputeDriver hook that we call on disable? | |
| 21:49:15 | dansmith | disable is on the service not the node, right? | |
| 21:49:23 | efried | or could/should we implement one? | |
| 21:49:25 | mriedem | er half a cell, and then live migrate stuff within the cell | |
| 21:49:27 | dansmith | so if you disable the thing running on ironic, the nodes will still be schedule-able | |
| 21:49:30 | mriedem | dansmith: yes on the service | |
| 21:49:53 | mriedem | efried: we don't have one for os-services disable no | |
| 21:49:55 | dansmith | so you have to do the service->nodes translation, and set the trait on all the nodes | |
| 21:50:04 | mriedem | we had one for os-hosts update status, but that was something else that only xen implemented | |
| 21:50:09 | dansmith | but if you rpc to the compute, it gets to do that itself | |
| 21:50:10 | mriedem | dansmith: yup | |
| 21:50:11 | efried | um, or we could make upt that thing - i.e. call it immediately on disable | |
| 21:50:40 | dansmith | mriedem: that also gets wonky if you disable a service and then they rebalance, but you have set the trait on some nodes that rebalanced out | |
| 21:50:42 | sean-k-mooney | mriedem: so the trait way woudl be add status_down trait and change placment query to include status_down forbien trait? | |
| 21:50:46 | dansmith | I guess they'll eventually un-disable, but.. ick | |
| 21:50:52 | dansmith | well, not actualy | |
| 21:51:00 | dansmith | because we just said virt has to leave those alone if the api set them | |
| 21:51:18 | mriedem | sean-k-mooney: essentially yes | |
| 21:51:28 | mriedem | if ComputeFilter is enabled or something | |
| 21:51:30 | dansmith | efried: right point being we don't have an rpc call for disable right now, but I think we probably should | |
| 21:51:35 | mriedem | or some new boolean that replaces ComputeFilter | |
| 21:51:57 | efried | oh, because the disabling is being prompted not by the compute itself. | |
| 21:52:03 | mriedem | correct | |
| 21:52:04 | dansmith | it's an api call | |
| 21:52:09 | dansmith | in most cases | |
| 21:52:12 | mriedem | https://developer.openstack.org/api-ref/compute/#compute-services-os-services | |
| 21:52:14 | dansmith | however, | |
| 21:52:16 | sean-k-mooney | well the trait could elminate the host before the filster so we dont need the computeFilter anymore | |
| 21:52:24 | mriedem | if the hypervisor dies the libvirt driver auto-disables it's own compute service | |
| 21:52:28 | dansmith | mriedem: computes can self-disable, so you'd also have to have them be able to set/clear the trait | |
| 21:52:38 | efried | this would be a great use case for sean-k-mooney's reverse-required traits :P | |
| 21:52:39 | mriedem | i smell spec | |
| 21:52:52 | dansmith | if you just let the compute always manage this in response to an rpc call, the ownership of it is a lot more clear I think | |
| 21:52:59 | efried | ++ | |
| 21:53:03 | mriedem | sure i'm fine with rpc call to compute to do the thing | |
| 21:53:18 | sean-k-mooney | efried: you mean the " you cant be schduled to this RP without asking for trait X" | |
| 21:53:23 | mriedem | btw this was the old thing from the os-hosts API https://github.com/openstack/nova/blob/master/nova/compute/api.py#L5026 | |
| 21:53:25 | efried | sean-k-mooney: yeah, that one. | |
| 21:53:57 | mriedem | we could maybe re-use that | |
| 21:54:38 | mriedem | that was only used by this before https://developer.openstack.org/api-ref/compute/#update-host-status | |
| 21:54:41 | mriedem | which is deprecated | |
| 21:55:14 | dansmith | I wonder if jaypipes is okay with a status_enabled or status_disabled trait on compute nodes | |
| 21:55:35 | sean-k-mooney | mriedem was that repalce by the mark host down thing that we added for opnfv | |
| 21:55:46 | mriedem | sean-k-mooney: on | |
| 21:55:47 | mriedem | *no | |