| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-22 | |||
| 19:41:04 | jaypipes | cfriesen: omg, I'm gonna slap you. | |
| 19:41:09 | jaypipes | :) | |
| 19:41:12 | cfriesen | jaypipes: what about a whole new allocation request | |
| 19:41:22 | jaypipes | cfriesen: it's non-deterministic from the viewpoint of the scheduler | |
| 19:41:33 | sean-k-mooney | jaypipes: ok so we just define it a retry and maybe you could use a weigher to minimies the change it would happen | |
| 19:42:17 | sean-k-mooney | e.g. you land on a host. figure out you need more resouce to run on that host then you asked for and retry on next host | |
| 19:42:24 | jaypipes | cfriesen: for a whole new allocation request, we'd need to re-submit to GET /allocation_candidates with a new requested resource amount, which would give us back a different set of compute hosts, which we would send to the NUMATopologyFilter, which would re-work the allocation request again, and we'd end up in an infinite loop of sadnsees. | |
| 19:42:24 | cfriesen | sean-k-mooney: so now we're saying that ISOLATE can't possible run on a host with HT enabled. how is this different from an aggregate? | |
| 19:42:59 | cfriesen | jaypipes: can we specify a particular compute node when doing the allocation request? | |
| 19:43:04 | sean-k-mooney | cfriesen: actully you can make it work but it doubles the amount of flavors | |
| 19:43:14 | jaypipes | cfriesen: no. | |
| 19:43:31 | cfriesen | jaypipes: how do we handle specifying the compute node when doing a migration? | |
| 19:43:33 | sean-k-mooney | cfriesen: you set flavor.vcpu=4 and resouce[vcpu]=8 and it will work only on ht systems | |
| 19:43:56 | cfriesen | sean-k-mooney: that'd work on non-ht as well, technically | |
| 19:43:56 | jaypipes | cfriesen: you can ask for an aggregate, an amount of resources, required traits, but not a specific provider (since that would defeat the entire purpose of the GET /allocation_candidates endpoint. | |
| 19:44:30 | jaypipes | cfriesen: we don't call GET /allocation_candidates when specifying a compute node (force_host). | |
| 19:44:37 | sean-k-mooney | cfriesen: yes but it really expecive maybe add trait:HT_COUNT_2=required to avoid that | |
| 19:45:03 | rybridges | mlavalle: This is how I am creating the network / segment / subnet: http://paste.openstack.org/show/708997/ As you can see the aggregate host list is empty. I was under the impression that this process creates a routed network. Is there something I am missing? | |
| 19:45:05 | jaypipes | sean-k-mooney: that's a trait that looks suspiciously like a quantity of resources. | |
| 19:45:27 | sean-k-mooney | jaypipes: the trait is based on how we said we would do cpu frequency | |
| 19:45:58 | jaypipes | sean-k-mooney: you mean vGPU display heads? | |
| 19:46:05 | sean-k-mooney | e.g. tag it with multipel tratis so 4GHZ cpu would have 1GHZ,2GHZ,3GHZ and 4GHZ traits applied | |
| 19:46:36 | mriedem | melwitt: that's a known issue http://status.openstack.org/elastic-recheck/#1758054 | |
| 19:46:40 | mriedem | everything is blocked until that's merged | |
| 19:46:48 | sean-k-mooney | jaypipes: ya i think that does the same thing but did not look at that that closely | |
| 19:46:59 | jaypipes | sean-k-mooney: but in this case, the amount of resources being *requested* changes depending on which host a workload ends up on :( that's the whole problem with this... | |
| 19:47:06 | melwitt | mriedem: ah, thanks | |
| 19:47:09 | dansmith | is gerrit sucking hard for everyone else? | |
| 19:47:42 | sean-k-mooney | jaypipes: yes so if you wanted an isoleated vm on a host with HT on you would have a flavor like this | |
| 19:48:17 | cfriesen | jaypipes: what do we call when specifying a compute node on a migration? | |
| 19:48:41 | sean-k-mooney | flavor.vcpu=4,resouce[vcpu]=8:traits:HT_count_2=forbid thread_policy=isolate | |
| 19:48:41 | jaypipes | dansmith: I'm too busy wanting to shoot myself in the head with a bazooka to feel any pain from gerrit. | |
| 19:48:52 | dansmith | jaypipes: roger that | |
| 19:49:10 | cfriesen | jaypipes: and why couldn't we do that in the scheduler if we realize after selecting a host that we need to account for some extra PCPU resources? | |
| 19:49:30 | cfriesen | sean-k-mooney: at that point you may as well use a host aggregate | |
| 19:49:45 | jaypipes | cfriesen: we call PUT /allocations/{migration_uuid} to reserve resources on the source host for the migration and PUT /allocations/{instance_uuid} to consume the instance resources on the destination host. | |
| 19:49:57 | jaypipes | cfriesen: we don't go through the shceduler at all when force_host. | |
| 19:50:24 | cfriesen | jaypipes: we do for migrations (when it's really a "suggested host" rather than force) | |
| 19:51:03 | sean-k-mooney | cfriesen: well you can have lavor.vcpu=4,resouce[vcpu]=8:traits:ht_count=require thread_policy=isolate and flavor.vcpu=4,resouce[vcpu]=4:traits:HT_count_2=forbid thread_policy=isolate | |
| 19:51:47 | sean-k-mooney | cfriesen: it should have been HT_count_2=require when resouce[vcpu]=8 not forbid originally | |
| 19:52:08 | jaypipes | sean-k-mooney: that just doesn't seem right to me. | |
| 19:52:10 | cfriesen | I really don't want to have multiple extra-spec keys that depend on the value of other extra-spec keys | |
| 19:52:22 | jaypipes | and certainly isn't very understandable to me. | |
| 19:52:28 | cfriesen | agreed, that's a mess. :) | |
| 19:53:13 | mriedem | we don't go through the scheduler when a host is forced, but conductor does the resource allocation 'claim' | |
| 19:53:22 | mriedem | for live migrate and evacuate | |
| 19:53:41 | sean-k-mooney | so we keep coming back to we deprecate and remove tread_policy=isolate or extend the allocation on the compute host which breaks the workflow | |
| 19:54:29 | sean-k-mooney | or we change isolate to mean "host with HT off" | |
| 19:55:35 | cfriesen | mriedem: for "cold migrate to this compute node" we need to be going through the scheduler to do cpu pinning, pci, etc. | |
| 19:57:29 | mriedem | cfriesen: we do go through the scheduler for cold migrate | |
| 19:57:34 | mriedem | there is no force for cold migrate | |
| 19:57:35 | mriedem | remember | |
| 19:57:37 | mriedem | -5 | |
| 19:57:41 | cfriesen | jaypipes: is there an actual API spec somewhere for placement? https://docs.openstack.org/nova/latest/user/placement.html doesn't seem to document the HTTP calls. | |
| 19:57:56 | mriedem | cfriesen: https://developer.openstack.org/api-ref/placement/ | |
| 19:58:17 | mriedem | https://docs.openstack.org/nova/latest/user/placement.html#rest-api goes to ^ | |
| 19:58:19 | cfriesen | mriedem: we allow an optional host, which is not a "force" but is a suggested destination | |
| 19:58:28 | mriedem | cfriesen: yeah, and | |
| 19:58:29 | mriedem | ? | |
| 19:58:32 | mriedem | it goes through the scheduler filters | |
| 19:58:33 | mriedem | and placement | |
| 19:58:58 | cfriesen | mriedem: right, but for that placement call do we ask for all the possible allocation candidates or do we specify a particular host? | |
| 19:59:15 | jaypipes | sean-k-mooney: at this point, I'd much prefer a single trait called HW_CPU_HYPERTHREADING whose absence indicates that hyperthreads are not enabled on the host. | |
| 19:59:30 | mriedem | cfriesen: placement doesn't know about 'hosts' | |
| 19:59:41 | jaypipes | sean-k-mooney: and using the forbidden traits stuff to find compute hosts that don't have hyperthreading enabled. | |
| 19:59:42 | mriedem | cfriesen: we say, 'hey placement, here is my request, give me your shit' | |
| 19:59:51 | mriedem | and then we restrict to just that requested host for the filtering | |
| 20:00:00 | jaypipes | mriedem: and placement goes 'sorry, I don't speak jive'. | |
| 20:00:14 | mriedem | is that a 418? | |
| 20:00:19 | jaypipes | correct. | |
| 20:00:19 | cfriesen | mriedem: okay, so the call to placement still filters all the RPs, then the scheduler filters narrow it down to the requested host? | |
| 20:00:29 | mriedem | cfriesen: yes | |
| 20:00:31 | jaypipes | cfriesen: yes | |
| 20:00:41 | sean-k-mooney | jaypipes: ya im leaning that way too. just get rid of hw:cpu_thread_policy as technical debt but i know that will piss of several people | |
| 20:01:18 | jaypipes | sean-k-mooney: if by several people you do accurately mean 2-3 people in the world, I'm willing to live with that. | |
| 20:01:35 | mriedem | doesn't stephenfin have a tattoo for that extra spec? | |
| 20:01:47 | sean-k-mooney | mriedem: i dont think so | |
| 20:01:52 | cfriesen | jaypipes: can we update the allocation? | |
| 20:02:03 | jaypipes | mriedem: no, but dansmith told me he just got an NFV tattoo. | |
| 20:02:04 | sean-k-mooney | mriedem: if he does that a life lesson in its self | |
| 20:02:07 | jaypipes | on his butt cheek. | |
| 20:02:28 | mriedem | oh i was thinking of https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/share-pci-between-numa-nodes.html | |
| 20:02:29 | dansmith | do not do no | |
| 20:02:46 | sean-k-mooney | cfriesen: we could but this would be the only place we update teh allocation from the compute node in the future | |
| 20:02:54 | sean-k-mooney | cfriesen: hence why we should not | |
| 20:03:06 | cfriesen | sean-k-mooney: I was thinking update it from the scheduler after picking the destination | |
| 20:03:48 | sean-k-mooney | cfriesen: but then again we need to down call to the compute node to know if we need to update it and then have to go to placement and make another allocation_candiates request | |
| 20:04:10 | cfriesen | sean-k-mooney: no, I'm pretty sure that the scheduler has information on number of ht siblings | |
| 20:04:29 | sean-k-mooney | cfriesen: in the host numa topology blob yes | |
| 20:05:08 | cfriesen | sean-k-mooney: I'm looking at the PUT /allocations/{consumer_uuid} operation, which seems to allow updating an existing allocation | |
| 20:05:14 | sean-k-mooney | but im not sure we still have the host_state object at the point we are selecting the node | |
| 20:05:23 | mlavalle | rybridges: what microversion of Placement you are using? | |
| 20:05:53 | cfriesen | sean-k-mooney: that's easy to solve | |
| 20:06:10 | jaypipes | cfriesen: no, we really can't update the allocation... | |
| 20:06:10 | sean-k-mooney | jaypipes: oh an when i say several people i mainly mean my management chain so meh | |
| 20:06:20 | cfriesen | jaypipes: what is that PUT operation then? | |
| 20:06:48 | jaypipes | cfriesen: are you referring to within the scheduler or somewhere else? | |
| 20:06:59 | cfriesen | https://developer.openstack.org/api-ref/placement/#update-allocations | |
| 20:07:49 | sean-k-mooney | jaypipes: yes he means update the claim form the schduler by checking the numa toplogy blob from the host state object for the selected host at the very end before we down call to the compute node | |
| 20:08:10 | jaypipes | cfriesen: that call overwrites the allocations that were made for an instance against one or more providers. | |