| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-18 | |||
| 16:57:03 | sean-k-mooney | im not sure if those are actully good examples | |
| 16:57:06 | efried | that is, we've identified the problem before, but punted on trying to solve it because it's too hard. | |
| 16:57:20 | dansmith | efried: CUSTOM_ would make it pretty easy | |
| 16:57:25 | sean-k-mooney | dansmith: well the hypertreading tread is explcitly not owned by the virt driver | |
| 16:57:42 | efried | hum | |
| 16:57:43 | dansmith | sean-k-mooney: why? | |
| 16:57:52 | sean-k-mooney | its stated that way to allow the cpu tread polices to work | |
| 16:58:12 | dansmith | sean-k-mooney: give me a reason that makes sense, not one that is tied to how things work today :) | |
| 16:58:44 | sean-k-mooney | this is why i said that might not be a good example but its in che CPU standaisation in placement spec | |
| 16:59:14 | dansmith | ack | |
| 16:59:29 | sean-k-mooney | last paragrh in https://github.com/openstack/nova-specs/blob/master/specs/train/approved/cpu-resources.rst#add-hw_cpu_hyperthreading-trait | |
| 16:59:42 | sean-k-mooney | well that section in general. | |
| 16:59:55 | sean-k-mooney | im not sure if there are standard traits you would want to manage this way | |
| 17:00:04 | sean-k-mooney | i just didnt want to assuem there wasnt | |
| 17:00:40 | dansmith | I think saying that these things are all restricted to the CUSTOM_ realm is a good line, IMHO | |
| 17:00:43 | efried | I think I can buy CUSTOM_ only. | |
| 17:00:51 | sean-k-mooney | again for v1 we could say just CUSTOM_ untill we have a usecase that requires standard traits | |
| 17:01:00 | dansmith | efried: you willing to say that for traits and inventory or just the former? | |
| 17:01:33 | sean-k-mooney | e.g. you can only have CUSTOM_ resouce classes too? | |
| 17:01:41 | dansmith | I'm less sure about it for inventory, but it does eliminate a lot of my concern over overriding virt inventory | |
| 17:01:47 | efried | well, in both cases it would be nice to be able to propose standard so that, if/when those features become native, you don't have to reshape. | |
| 17:01:54 | sean-k-mooney | i think invntories are less of an issue | |
| 17:01:56 | efried | but | |
| 17:02:09 | efried | I can see where it simplifies things to say CUSTOM_ only for now. | |
| 17:02:12 | dansmith | efried: seems like a clearer line though | |
| 17:02:15 | efried | yes | |
| 17:02:39 | dansmith | efried: do you actually expect people to define their own non-custom classes in placement? if they do they run up against id conflict right? | |
| 17:02:48 | sean-k-mooney | we have previosly agreed that that virt divers should not use CUSTOM_ stuff in general right | |
| 17:02:57 | efried | dansmith: no, they would have to propose them in the actual repos | |
| 17:03:01 | dansmith | sean-k-mooney: except for ironic which *relies* on it | |
| 17:03:08 | dansmith | efried: right, so then they have to reshape anyway | |
| 17:03:14 | sean-k-mooney | and i gues the vPMEM stuff | |
| 17:03:28 | sean-k-mooney | ok ignore the virt driver wont use it thing | |
| 17:03:32 | efried | I'm saying if they propose the standard ones before they start customizing, they don't need to reshape. | |
| 17:03:46 | dansmith | efried: that's a big if, but okay :) | |
| 17:03:51 | efried | anyway, I think trying to anticipate how reshapes play in here is a fool's errand | |
| 17:04:07 | cdent | It's an important question though | |
| 17:04:10 | dansmith | I'm massively happier about all this if it's restricted to CUSTOM_ | |
| 17:04:17 | efried | so yeah, CUSTOM only for both traits and resource classes is okay for this rev | |
| 17:04:31 | cdent | if the point of the yaml file is to allow people to experiment on thing that they then expect to some day become "normal" | |
| 17:04:51 | efried | cdent: yeah, that's where I was leaning | |
| 17:04:51 | dansmith | cdent: that may be one use of it, but not the primary use I expect | |
| 17:05:30 | cdent | I was going on what the spec says, because I _still_ have trouble comprehending who the user is here | |
| 17:05:49 | sean-k-mooney | there may be some things we never want nova to own on the comptue node like the cyborg aclleartos so something may move out of the provdier.yaml to other service or jsut stay there | |
| 17:05:52 | dansmith | cdent: the user is intel :/ | |
| 17:05:54 | cdent | it reads as "some hardware vendors want to get some stuff in faster than nova can do it,but still want nova to do it" | |
| 17:06:09 | dansmith | cdent: pretty much :) | |
| 17:06:17 | cdent | hawt | |
| 17:06:32 | sean-k-mooney | well RMD is one usecase | |
| 17:06:34 | sean-k-mooney | there are others | |
| 17:06:53 | sean-k-mooney | probably | |
| 17:07:00 | cdent | whereas when it was coming in from the jay angle, I understood more as "sometimes we want to shape the hardware that is present to make it look different from what it says" | |
| 17:07:21 | dansmith | I don't think that's jay's angle but I could be wrong | |
| 17:07:41 | sean-k-mooney | well jay would like to abstract where it makes sense too | |
| 17:07:52 | sean-k-mooney | to not expose every faset of the hardware topology | |
| 17:07:57 | dansmith | and in his absence, I would claim that his angle is my angle, which is that this is for custom stuff, like accounting for fan capacity in aisles, etc | |
| 17:08:01 | sean-k-mooney | just the bits that we need | |
| 17:08:05 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: [DNM] testing bug/1813789 revert resize events https://review.opendev.org/671303 | |
| 17:08:29 | sean-k-mooney | dansmith: ya one of the usecase i was thinking of was power and termal cpasity | |
| 17:08:30 | cdent | fans in aisles are so cool | |
| 17:08:39 | dansmith | sean-k-mooney: yeah | |
| 17:08:47 | efried | So then this suggests a flow like: | |
| 17:08:47 | efried | 0) parse and schema-validate the file. (This happens just once, on startup, and fails the compute service if something is wrong) | |
| 17:08:47 | efried | Then in the RT _update flow: | |
| 17:08:47 | efried | 1) update_provider_tree (as today). (If we ever decide we want to give the driver direct pre/post processing control over the providers.yaml content, we could pass the json blob as a kwarg here. But not now.) | |
| 17:08:47 | efried | 2) automatic traits (as today) | |
| 17:08:48 | efried | 3) merge in providers.yaml stuff. (I think a second level of validation happens here, and fails the compute service if something is wrong -- that should only be possible the first time through, so we're not worried about the service dying after having been running for a while) | |
| 17:08:48 | efried | 4) update_from_provider_tree (flush to placement) (as today) | |
| 17:10:37 | efried | I'm not sure how to detect a conflict between a custom RC from upt and one from providers.yaml. | |
| 17:11:04 | sean-k-mooney | the validation of 3 might be as simple as make sure the intersection between the inventoires form each part of the merger is 0 | |
| 17:11:19 | efried | yeah, that's what I'm noodling | |
| 17:11:27 | sean-k-mooney | e.g. an invenotry is not defiend both the provider and virt driver trees | |
| 17:11:35 | efried | that puts a bit of extra onus on upt to make sure it doesn't "ignore resource classes it doesn't know about" | |
| 17:12:05 | efried | cause today we're careful to tell upt it must "ignore child providers it doesn't know about" | |
| 17:12:09 | sean-k-mooney | ya that is related to the general problem of shareign RP between service. | |
| 17:12:13 | efried | but I think that doesn't conflict. | |
| 17:12:15 | sean-k-mooney | which for now we have punted | |
| 17:12:20 | efried | yes | |
| 17:12:23 | cdent | we keep having discussions about things knowing about things, and that makes me anxious. how does anything know? | |
| 17:13:02 | efried | it's the virt driver's job to know about the hardware resources it has the ability to assign to VMs. | |
| 17:13:35 | sean-k-mooney | right ad the provider.yamls would know about resoce that are not assinged to the vm but are indrecly consumed by it | |
| 17:13:43 | sean-k-mooney | e.g power | |
| 17:14:05 | sean-k-mooney | or cache if the cache is affiend outside the virt dirver | |
| 17:14:21 | dansmith | efried: the validation becomes "is CUSTOM_ and not already in the stuff I got from the virt driver" right? | |
| 17:14:21 | sean-k-mooney | which you can do with rmd or sysfs and a crons script | |
| 17:14:56 | sean-k-mooney | dansmith: i would hope so. or at least not signifcatly more complex | |
| 17:15:10 | sean-k-mooney | the majority fo the validate would be done when parsing the file | |
| 17:15:13 | dansmith | yeah | |
| 17:15:17 | dansmith | yeah | |
| 17:15:29 | efried | dansmith: yes, the concern I had was with idempotency. The second time through, the provider_tree we hand to upt will already contain the CUSTOM_ RCs from the last iteration when we processed providers.yaml. So upt has to *remove* those, or we'll blow up when we process the providers.yaml blob the second time. | |
| 17:16:00 | dansmith | ah I see | |
| 17:16:06 | efried | if we think of upt's responsibility as simply "completely overwite the inventories for the providers I own" then it works. | |
| 17:16:07 | dansmith | maybe just not fail if they're identical? | |
| 17:16:17 | efried | yeah, or that, that could work. | |
| 17:16:25 | efried | um | |
| 17:16:28 | dansmith | just "isn't already defined or is exactly the same" | |
| 17:16:42 | efried | except "identical" needs to be forgiving of defaults | |
| 17:16:54 | efried | because my providers.yaml is allowed to give only "total" and let the other fields default. | |
| 17:17:04 | dansmith | it's already limited to CUSTOM_ so there wouldn't be any of those right? | |
| 17:17:08 | sean-k-mooney | is identical would require the virt driver to have the infor from the yaml to compare agaisnt | |