Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-07-18
16:54:48 efried okay.
16:54:49 efried so
16:54:50 sean-k-mooney if we allw modifcation of inventores/traits creted by the virt driver i also conceed we might want to delgate processign to the virt driver at that point
16:55:05 dansmith efried: don't we have a declaration of which traits are owned by the compute and virt, so that we can wholly box off those anyway?
16:55:17 efried tomorrow: update_provider_tree => process_providers_yaml => update_auto_traits
16:55:17 efried today: update_provider_tree => update_auto_traits
16:55:24 efried dansmith: no, unfortunately not
16:55:38 sean-k-mooney dansmith: not really. unless we say provdier.yaml can only use CUSTOM_
16:55:38 efried and in fact, the above mixes poorly
16:55:40 dansmith efried: well, maybe we should do that as part of this.. I know we discussed it before
16:55:44 sean-k-mooney which i think is too restrictive
16:55:51 dansmith sean-k-mooney: well, that would probably not be too bad, IMHO
16:55:54 efried because update_provider_tree may have some traits it enforces
16:56:13 dansmith it would clearly draw a box around the things you're doing as being purely localized customization and not mucking with internal features
16:56:20 efried but then update_auto_traits (which is not under control of the virt driver -- except as declared by the compute capabilities dict) has others
16:56:21 sean-k-mooney it would be nice to be able to use a standard hyper treading treat or secure boot trait
16:56:33 sean-k-mooney and use the file to add them if its supported
16:56:42 efried dansmith: unfortunately it's not easy to do. See venn diagram here https://docs.openstack.org/nova/latest/reference/update-provider-tree.html#taxonomy-of-traits-and-capabilities
16:56:57 dansmith sean-k-mooney: but why would that not be exposed by the driver if it's available and enabled? making operators track that is dumb
16:57:03 sean-k-mooney im not sure if those are actully good examples
16:57:06 efried that is, we've identified the problem before, but punted on trying to solve it because it's too hard.
16:57:20 dansmith efried: CUSTOM_ would make it pretty easy
16:57:25 sean-k-mooney dansmith: well the hypertreading tread is explcitly not owned by the virt driver
16:57:42 efried hum
16:57:43 dansmith sean-k-mooney: why?
16:57:52 sean-k-mooney its stated that way to allow the cpu tread polices to work
16:58:12 dansmith sean-k-mooney: give me a reason that makes sense, not one that is tied to how things work today :)
16:58:44 sean-k-mooney this is why i said that might not be a good example but its in che CPU standaisation in placement spec
16:59:14 dansmith ack
16:59:29 sean-k-mooney last paragrh in https://github.com/openstack/nova-specs/blob/master/specs/train/approved/cpu-resources.rst#add-hw_cpu_hyperthreading-trait
16:59:42 sean-k-mooney well that section in general.
16:59:55 sean-k-mooney im not sure if there are standard traits you would want to manage this way
17:00:04 sean-k-mooney i just didnt want to assuem there wasnt
17:00:40 dansmith I think saying that these things are all restricted to the CUSTOM_ realm is a good line, IMHO
17:00:43 efried I think I can buy CUSTOM_ only.
17:00:51 sean-k-mooney again for v1 we could say just CUSTOM_ untill we have a usecase that requires standard traits
17:01:00 dansmith efried: you willing to say that for traits and inventory or just the former?
17:01:33 sean-k-mooney e.g. you can only have CUSTOM_ resouce classes too?
17:01:41 dansmith I'm less sure about it for inventory, but it does eliminate a lot of my concern over overriding virt inventory
17:01:47 efried well, in both cases it would be nice to be able to propose standard so that, if/when those features become native, you don't have to reshape.
17:01:54 sean-k-mooney i think invntories are less of an issue
17:01:56 efried but
17:02:09 efried I can see where it simplifies things to say CUSTOM_ only for now.
17:02:12 dansmith efried: seems like a clearer line though
17:02:15 efried yes
17:02:39 dansmith efried: do you actually expect people to define their own non-custom classes in placement? if they do they run up against id conflict right?
17:02:48 sean-k-mooney we have previosly agreed that that virt divers should not use CUSTOM_ stuff in general right
17:02:57 efried dansmith: no, they would have to propose them in the actual repos
17:03:01 dansmith sean-k-mooney: except for ironic which *relies* on it
17:03:08 dansmith efried: right, so then they have to reshape anyway
17:03:14 sean-k-mooney and i gues the vPMEM stuff
17:03:28 sean-k-mooney ok ignore the virt driver wont use it thing
17:03:32 efried I'm saying if they propose the standard ones before they start customizing, they don't need to reshape.
17:03:46 dansmith efried: that's a big if, but okay :)
17:03:51 efried anyway, I think trying to anticipate how reshapes play in here is a fool's errand
17:04:07 cdent It's an important question though
17:04:10 dansmith I'm massively happier about all this if it's restricted to CUSTOM_
17:04:17 efried so yeah, CUSTOM only for both traits and resource classes is okay for this rev
17:04:31 cdent if the point of the yaml file is to allow people to experiment on thing that they then expect to some day become "normal"
17:04:51 dansmith cdent: that may be one use of it, but not the primary use I expect
17:04:51 efried cdent: yeah, that's where I was leaning
17:05:30 cdent I was going on what the spec says, because I _still_ have trouble comprehending who the user is here
17:05:49 sean-k-mooney there may be some things we never want nova to own on the comptue node like the cyborg aclleartos so something may move out of the provdier.yaml to other service or jsut stay there
17:05:52 dansmith cdent: the user is intel :/
17:05:54 cdent it reads as "some hardware vendors want to get some stuff in faster than nova can do it,but still want nova to do it"
17:06:09 dansmith cdent: pretty much :)
17:06:17 cdent hawt
17:06:32 sean-k-mooney well RMD is one usecase
17:06:34 sean-k-mooney there are others
17:06:53 sean-k-mooney probably
17:07:00 cdent whereas when it was coming in from the jay angle, I understood more as "sometimes we want to shape the hardware that is present to make it look different from what it says"
17:07:21 dansmith I don't think that's jay's angle but I could be wrong
17:07:41 sean-k-mooney well jay would like to abstract where it makes sense too
17:07:52 sean-k-mooney to not expose every faset of the hardware topology
17:07:57 dansmith and in his absence, I would claim that his angle is my angle, which is that this is for custom stuff, like accounting for fan capacity in aisles, etc
17:08:01 sean-k-mooney just the bits that we need
17:08:05 openstackgerrit Artom Lifshitz proposed openstack/nova stable/rocky: [DNM] testing bug/1813789 revert resize events https://review.opendev.org/671303
17:08:29 sean-k-mooney dansmith: ya one of the usecase i was thinking of was power and termal cpasity
17:08:30 cdent fans in aisles are so cool
17:08:39 dansmith sean-k-mooney: yeah
17:08:47 efried 2) automatic traits (as today)
17:08:47 efried 1) update_provider_tree (as today). (If we ever decide we want to give the driver direct pre/post processing control over the providers.yaml content, we could pass the json blob as a kwarg here. But not now.)
17:08:47 efried Then in the RT _update flow:
17:08:47 efried 0) parse and schema-validate the file. (This happens just once, on startup, and fails the compute service if something is wrong)
17:08:47 efried So then this suggests a flow like:
17:08:48 efried 4) update_from_provider_tree (flush to placement) (as today)
17:08:48 efried 3) merge in providers.yaml stuff. (I think a second level of validation happens here, and fails the compute service if something is wrong -- that should only be possible the first time through, so we're not worried about the service dying after having been running for a while)
17:10:37 efried I'm not sure how to detect a conflict between a custom RC from upt and one from providers.yaml.
17:11:04 sean-k-mooney the validation of 3 might be as simple as make sure the intersection between the inventoires form each part of the merger is 0
17:11:19 efried yeah, that's what I'm noodling
17:11:27 sean-k-mooney e.g. an invenotry is not defiend both the provider and virt driver trees
17:11:35 efried that puts a bit of extra onus on upt to make sure it doesn't "ignore resource classes it doesn't know about"
17:12:05 efried cause today we're careful to tell upt it must "ignore child providers it doesn't know about"
17:12:09 sean-k-mooney ya that is related to the general problem of shareign RP between service.
17:12:13 efried but I think that doesn't conflict.
17:12:15 sean-k-mooney which for now we have punted
17:12:20 efried yes
17:12:23 cdent we keep having discussions about things knowing about things, and that makes me anxious. how does anything know?
17:13:02 efried it's the virt driver's job to know about the hardware resources it has the ability to assign to VMs.
17:13:35 sean-k-mooney right ad the provider.yamls would know about resoce that are not assinged to the vm but are indrecly consumed by it

Earlier   Later