| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-18 | |||
| 17:04:10 | dansmith | I'm massively happier about all this if it's restricted to CUSTOM_ | |
| 17:04:17 | efried | so yeah, CUSTOM only for both traits and resource classes is okay for this rev | |
| 17:04:31 | cdent | if the point of the yaml file is to allow people to experiment on thing that they then expect to some day become "normal" | |
| 17:04:51 | efried | cdent: yeah, that's where I was leaning | |
| 17:04:51 | dansmith | cdent: that may be one use of it, but not the primary use I expect | |
| 17:05:30 | cdent | I was going on what the spec says, because I _still_ have trouble comprehending who the user is here | |
| 17:05:49 | sean-k-mooney | there may be some things we never want nova to own on the comptue node like the cyborg aclleartos so something may move out of the provdier.yaml to other service or jsut stay there | |
| 17:05:52 | dansmith | cdent: the user is intel :/ | |
| 17:05:54 | cdent | it reads as "some hardware vendors want to get some stuff in faster than nova can do it,but still want nova to do it" | |
| 17:06:09 | dansmith | cdent: pretty much :) | |
| 17:06:17 | cdent | hawt | |
| 17:06:32 | sean-k-mooney | well RMD is one usecase | |
| 17:06:34 | sean-k-mooney | there are others | |
| 17:06:53 | sean-k-mooney | probably | |
| 17:07:00 | cdent | whereas when it was coming in from the jay angle, I understood more as "sometimes we want to shape the hardware that is present to make it look different from what it says" | |
| 17:07:21 | dansmith | I don't think that's jay's angle but I could be wrong | |
| 17:07:41 | sean-k-mooney | well jay would like to abstract where it makes sense too | |
| 17:07:52 | sean-k-mooney | to not expose every faset of the hardware topology | |
| 17:07:57 | dansmith | and in his absence, I would claim that his angle is my angle, which is that this is for custom stuff, like accounting for fan capacity in aisles, etc | |
| 17:08:01 | sean-k-mooney | just the bits that we need | |
| 17:08:05 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: [DNM] testing bug/1813789 revert resize events https://review.opendev.org/671303 | |
| 17:08:29 | sean-k-mooney | dansmith: ya one of the usecase i was thinking of was power and termal cpasity | |
| 17:08:30 | cdent | fans in aisles are so cool | |
| 17:08:39 | dansmith | sean-k-mooney: yeah | |
| 17:08:47 | efried | So then this suggests a flow like: | |
| 17:08:47 | efried | 0) parse and schema-validate the file. (This happens just once, on startup, and fails the compute service if something is wrong) | |
| 17:08:47 | efried | Then in the RT _update flow: | |
| 17:08:47 | efried | 1) update_provider_tree (as today). (If we ever decide we want to give the driver direct pre/post processing control over the providers.yaml content, we could pass the json blob as a kwarg here. But not now.) | |
| 17:08:47 | efried | 2) automatic traits (as today) | |
| 17:08:48 | efried | 3) merge in providers.yaml stuff. (I think a second level of validation happens here, and fails the compute service if something is wrong -- that should only be possible the first time through, so we're not worried about the service dying after having been running for a while) | |
| 17:08:48 | efried | 4) update_from_provider_tree (flush to placement) (as today) | |
| 17:10:37 | efried | I'm not sure how to detect a conflict between a custom RC from upt and one from providers.yaml. | |
| 17:11:04 | sean-k-mooney | the validation of 3 might be as simple as make sure the intersection between the inventoires form each part of the merger is 0 | |
| 17:11:19 | efried | yeah, that's what I'm noodling | |
| 17:11:27 | sean-k-mooney | e.g. an invenotry is not defiend both the provider and virt driver trees | |
| 17:11:35 | efried | that puts a bit of extra onus on upt to make sure it doesn't "ignore resource classes it doesn't know about" | |
| 17:12:05 | efried | cause today we're careful to tell upt it must "ignore child providers it doesn't know about" | |
| 17:12:09 | sean-k-mooney | ya that is related to the general problem of shareign RP between service. | |
| 17:12:13 | efried | but I think that doesn't conflict. | |
| 17:12:15 | sean-k-mooney | which for now we have punted | |
| 17:12:20 | efried | yes | |
| 17:12:23 | cdent | we keep having discussions about things knowing about things, and that makes me anxious. how does anything know? | |
| 17:13:02 | efried | it's the virt driver's job to know about the hardware resources it has the ability to assign to VMs. | |
| 17:13:35 | sean-k-mooney | right ad the provider.yamls would know about resoce that are not assinged to the vm but are indrecly consumed by it | |
| 17:13:43 | sean-k-mooney | e.g power | |
| 17:14:05 | sean-k-mooney | or cache if the cache is affiend outside the virt dirver | |
| 17:14:21 | dansmith | efried: the validation becomes "is CUSTOM_ and not already in the stuff I got from the virt driver" right? | |
| 17:14:21 | sean-k-mooney | which you can do with rmd or sysfs and a crons script | |
| 17:14:56 | sean-k-mooney | dansmith: i would hope so. or at least not signifcatly more complex | |
| 17:15:10 | sean-k-mooney | the majority fo the validate would be done when parsing the file | |
| 17:15:13 | dansmith | yeah | |
| 17:15:17 | dansmith | yeah | |
| 17:15:29 | efried | dansmith: yes, the concern I had was with idempotency. The second time through, the provider_tree we hand to upt will already contain the CUSTOM_ RCs from the last iteration when we processed providers.yaml. So upt has to *remove* those, or we'll blow up when we process the providers.yaml blob the second time. | |
| 17:16:00 | dansmith | ah I see | |
| 17:16:06 | efried | if we think of upt's responsibility as simply "completely overwite the inventories for the providers I own" then it works. | |
| 17:16:07 | dansmith | maybe just not fail if they're identical? | |
| 17:16:17 | efried | yeah, or that, that could work. | |
| 17:16:25 | efried | um | |
| 17:16:28 | dansmith | just "isn't already defined or is exactly the same" | |
| 17:16:42 | efried | except "identical" needs to be forgiving of defaults | |
| 17:16:54 | efried | because my providers.yaml is allowed to give only "total" and let the other fields default. | |
| 17:17:04 | dansmith | it's already limited to CUSTOM_ so there wouldn't be any of those right? | |
| 17:17:08 | sean-k-mooney | is identical would require the virt driver to have the infor from the yaml to compare agaisnt | |
| 17:17:19 | efried | default inventory values, like min_unit etc. | |
| 17:17:19 | cdent | good luck, gotta make dinner | |
| 17:17:21 | dansmith | efried: oh I see, | |
| 17:17:23 | efried | o/ cdent | |
| 17:17:29 | sean-k-mooney | unless the virt driver kept a list of the invetores it managed and only looked at those | |
| 17:17:30 | dansmith | well, I'm sure you can work that out | |
| 17:17:35 | sean-k-mooney | cdent: o/ | |
| 17:17:40 | efried | sean-k-mooney: the virt driver does that | |
| 17:17:51 | dansmith | sean-k-mooney: I think what you mean is just mark in the provider tree where each thing came from | |
| 17:17:54 | sean-k-mooney | right so if it igores inventories not in that list | |
| 17:17:56 | dansmith | so we can check the origin | |
| 17:18:05 | efried | except "only looked at those" is what I'm saying *doesn't* work. | |
| 17:18:29 | sean-k-mooney | ya but we dont have an owner on the inventory to do that currently | |
| 17:18:32 | sean-k-mooney | or ever | |
| 17:18:33 | efried | because we're enforcing that providers.yaml is not allowed to muck with resources upt controls. | |
| 17:18:41 | sean-k-mooney | so we would have to determin that in the RT | |
| 17:18:51 | efried | yes | |
| 17:18:57 | sean-k-mooney | whci we could do if we updated form teh provder.yaml first | |
| 17:19:01 | sean-k-mooney | then called the virt dirver | |
| 17:19:07 | efried | not the second time around. | |
| 17:19:08 | sean-k-mooney | then added the auto tratis | |
| 17:19:27 | sean-k-mooney | we should be able to mark them second time around | |
| 17:19:33 | efried | "mark" how? | |
| 17:19:49 | efried | You're talking about adding a metadata field to the ProviderTree object or something? | |
| 17:20:10 | dansmith | efried: that's what I meant, but I'll leave it to you about what approach is best for that conflict resolution | |
| 17:20:12 | sean-k-mooney | we woudl still get the set form teh file and then the tree we pass into upt we embed a flag in each inventory | |
| 17:20:33 | efried | ugh, that seems overly complicated to me. | |
| 17:20:34 | sean-k-mooney | efried: yes but not one that need to ever leave nova | |
| 17:20:40 | efried | yeah, I get that | |
| 17:20:56 | efried | I would just hate to explode the compute service because of a cache bug. | |
| 17:21:17 | efried | maybe we take advantage of the init kwarg | |
| 17:21:21 | efried | and only do that check the first time through. | |
| 17:21:35 | efried | because that's the only time we're worried about it anyway. | |
| 17:21:38 | dansmith | as long as we don't re-load the yaml, that would be reasonable It hink | |
| 17:21:57 | sean-k-mooney | well the work flwo would be. get prov_tree from placement. generate tree from file. merge them and mark any that are common as from the file then pass it to the driver upt meethod | |
| 17:21:59 | efried | yeah, we're definitely only loading the yaml once. | |
| 17:22:10 | efried | otherwise the admin can blow up the compute service with a typo. | |