| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-12 | |||
| 16:32:47 | efried | Otherwise why would we be bothering to do this in a periodic? | |
| 16:32:52 | efried | cdent: Yes, or *removed* | |
| 16:32:58 | cdent | yes | |
| 16:33:35 | efried | So I go back to asserting this doesn't work for the general case. | |
| 16:34:02 | jaypipes | efried: "the general case" <-- what is that? | |
| 16:34:14 | efried | jaypipes: I just mean "for all cases". | |
| 16:34:20 | edleafe | If an admin wants to "remove" a trait FOO set by the virt driver, she could add a CUSTOM_FOO_BAD trait, and then make that a forbidden trait | |
| 16:35:07 | efried | edleafe: I think we're past that one. It doesn't have a realistic use case, most likely. | |
| 16:35:40 | edleafe | efried: good | |
| 16:37:28 | efried | dansmith, jaypipes: Specifically the case I'm talking about is, for example, hot-pluggable storage. Let's say you start with some disk. Virt driver presents inventory of DISK_GB and trait STORAGE_DISK_SSD. Then you pull out the disk. The next u_p_t needs to be able to remove the DISK_GB inventory *and* remove the STORAGE_DISK_SSD trait. With this design, it can't do the latter. The admin would have to do it manually. | |
| 16:37:31 | jaypipes | efried: it's not that I don't see your use case (for example, dynamically marking some child providers representing a PF with a CUSTOM_TRUSTED trait or something). But I also don't necessarily think that what dansmith is proposing would *preclude* us from moving in a more complicated direction if and when such a direction was determined to be required. | |
| 16:39:55 | dansmith | this is what I think we need: https://pastebin.com/mM6NMzQ4 | |
| 16:40:18 | dansmith | note the operator's traits are untouched, yet the virt and compute-owned traits are asserted or de-asserted as things change | |
| 16:41:28 | cdent | oh that's interesting | |
| 16:41:58 | efried | dansmith: What I'm asserting is that the virt driver has no way of knowing it wants to -SOMETHING | |
| 16:42:02 | cdent | but can/does that interop with how u_p_t is planned to behave | |
| 16:42:27 | ygl | hi all | |
| 16:42:35 | dansmith | efried: it totally does | |
| 16:42:43 | efried | cdent: What we decided on DublinFriday is that we're going to merge u_p_t as currently written and then bolt this on after. | |
| 16:42:55 | jaypipes | dansmith: that's essentially what my traits:always and traits:ignore in the provider config file format were doing. :) | |
| 16:43:20 | dansmith | efried: I assert we don't have to worry about cases where the operator adds a trait to an x86 compute node of TRAIT_CPU_POWER7_THINGY | |
| 16:43:21 | ygl | i want to launch a vm with a dummy nic interface without an IP assigned to it . can someone help me how to do it please | |
| 16:43:45 | jaypipes | dansmith: but clearly I don't segregate between what the compute and the virt think of as asserted/removed | |
| 16:43:46 | efried | ygl: Try #openstack | |
| 16:43:46 | dansmith | jaypipes: this is not always and ignore, this is always and remove, or something | |
| 16:43:54 | jaypipes | dansmith: understood. | |
| 16:44:11 | dansmith | jaypipes: and this is exposed from the virt driver, not encoded in a separate editable file | |
| 16:44:44 | jaypipes | dansmith: would this be exposed as a config option (or 2 options)? | |
| 16:44:50 | dansmith | jaypipes: no | |
| 16:45:02 | efried | dansmith: I'm not talking about the admin adding traits - we already decided she gets to do that. I'm talking about a trait the virt driver needs to remove because e.g. someone hot-unplugged a disk. | |
| 16:45:16 | dansmith | jaypipes: the virt driver would return things like this, which may vary based on virt driver config and talking to the hypevisor | |
| 16:45:24 | efried | The virt driver operates by discovering what it has and asserting inventory/traits accordingly. | |
| 16:45:24 | jaypipes | k | |
| 16:45:51 | efried | If it discovers at time T and has a disk, it will assert some DISK_GB inventory and the STORAGE_DISK_SSD trait. | |
| 16:46:18 | jaypipes | dansmith: a perfectly satisfactory solution. | |
| 16:46:37 | dansmith | jaypipes: this is what I was describing from my corner on friday while we were discussing this | |
| 16:46:43 | efried | Then it discovers at time T+1 and it doesn't have a disk; it will not assert DISK_GB inventory (which works, because it's the only source of inventory info) and it will not assert the STORAGE_DISK_SSD trait. | |
| 16:46:56 | efried | But it *doesn't* know that it needs to *de*assert the STORAGE_DISK_SSD trait. | |
| 16:47:11 | efried | Because it has no memory that it asserted it at time T. It just knows it isn't there now. | |
| 16:47:13 | dansmith | efried: I completely don't understand that | |
| 16:47:13 | jaypipes | dansmith: yes, I recognize the +/- syntax now. just didn't have in my mind how the virt driver would expose this (automatically vs. config-based) | |
| 16:47:22 | dansmith | jaypipes: ack | |
| 16:47:52 | dansmith | efried: if you as a virt driver ever expose trait FOO, then you need to de-assert FOO if you determine there is no reason to assert it, right? | |
| 16:49:04 | efried | That goes exactly back to my earlier statement about how the virt driver would have to know the full set of traits it *might* be responsible for, and then what subset of those it thinks should be set, so that it can explicitly deassert the remainder. | |
| 16:49:19 | dansmith | why is that unreasonable? | |
| 16:49:32 | dansmith | it doesn't have to be the full set of traits that any virt driver might expose, | |
| 16:49:39 | dansmith | just the ones _it_ might expose | |
| 16:50:19 | dansmith | we could make a turbo simple data structure that they all use, which requires them to declare any traits they may want to use, then assert the ones it finds, and the rest will be de-asserted | |
| 16:50:34 | dansmith | to avoid leaking some asserted ones that are never de-asserted | |
| 16:50:42 | dansmith | I must be missing why this is a hard problem | |
| 16:50:44 | bauzas | dansmith: so the virt driver would provide all the possible CPU traits, either with a "+" or a "-" prefix, depending on what it knows to support ? | |
| 16:50:58 | dansmith | bauzas: all the cpu traits it knows about yeah | |
| 16:51:10 | bauzas | that would work then | |
| 16:51:18 | bauzas | okay, I see your idea | |
| 16:51:21 | dansmith | bauzas: it doesn't need to worry about traits for PPC processors, for example | |
| 16:51:32 | dansmith | if the operator added one of those then, sucks for that operator | |
| 16:51:45 | bauzas | yeah, but it would still report "negative" traits | |
| 16:51:51 | dansmith | yeah | |
| 16:51:55 | bauzas | cool with me then | |
| 16:52:02 | dansmith | so if you changed the cpu_model later, it would sync up with all the flags that are now legit | |
| 16:52:15 | bauzas | based on the specific compute version it runs | |
| 16:52:29 | ygl | bauzas: can you help me with my issue please if you can | |
| 16:52:31 | dansmith | sure, but as time goes forward, the cpu flag set only grows | |
| 16:52:41 | bauzas | that's a reasonable assumption | |
| 16:52:53 | dansmith | and for non-cpu flag things, | |
| 16:53:00 | ygl | bauzas: i want to launch a vm with a dummy nic interface without an IP assigned to it | |
| 16:53:01 | dansmith | it becomes a compatibility thing kinda like our db schema, | |
| 16:53:04 | bauzas | and honestly, who cares if the operator sets a trait that the virt driver doesn't know ? | |
| 16:53:19 | dansmith | where you need to not just add/remove traits willy-nilly over releases without doing some cleanup | |
| 16:53:26 | dansmith | ygl: this channel is for development, see topic please | |
| 16:53:31 | bauzas | the operator would suck less if they would use a custom trait for that | |
| 16:53:41 | bauzas | yeah, I like that idea actually | |
| 16:53:48 | ygl | dansmith: i tried other channels but they r not responding | |
| 16:53:53 | bauzas | because we explicitly then provide what the virt driver knows | |
| 16:53:58 | dansmith | bauzas: yup, we could have an audit mode where the compute node logs any traits it didn't calculate, which should ideally be just the ones the operator set | |
| 16:54:28 | dansmith | so any that got leaked over time would be easy to identify | |
| 16:56:50 | ygl | bauzas: sorry to disturb you. can you help me with my issue, if you can | |
| 16:57:13 | efried | Note that the ironic virt driver is gleaning traits from inspector. So that interface will need to change: either to support +/-; or to add a separate "get all the traits I might care about" call. | |
| 16:57:49 | dansmith | efried: the interface from the ironic driver to ironic? | |
| 16:57:54 | efried | yeah | |
| 16:57:59 | sean-k-mooney | dansmith: the cpu traits would be reported on the CPU resouce providers not the compute node correct nulless the cpu inventory is under the compute node. just thinking of the numa case where the inventory would be per numa node | |
| 16:58:24 | dansmith | efried: depends on the traits I guess, and what is going on right now.. if the ironic driver is exposing traits that don't need to be deleted later, then it won't really matter | |
| 16:58:46 | dansmith | sean-k-mooney: yeah, that doesn't change this though | |
| 16:59:39 | sean-k-mooney | dansmith: yep just checking. i would expect most operator traits to be applied to the compute node RP but they may want to tag sub resouce providres also. | |
| 17:00:01 | dansmith | sean-k-mooney: yeah, network-related traits will need to be below compute node | |
| 17:00:11 | dansmith | efried: ah, looks like we're just proxying any trait they want? | |
| 17:00:17 | efried | yup. | |
| 17:00:49 | dansmith | efried: yeah, well, that kinda sucks, but it's resolvable I think | |
| 17:00:54 | efried | Assuming we go forward with this, all the merge logic needs to be handled by the individual virt drivers. Unless we're going to invent some interface outside of u_p_t. | |
| 17:00:55 | efried | except for get_traits, which I freakin knew was going to bite us in the ass. | |
| 17:01:20 | dansmith | I guess we could also provide the current set of traits to the virt driver so it gets to decide if it thinks any of the ones set are important to ack or nak | |
| 17:01:29 | efried | dansmith: That's what we do. | |
| 17:01:34 | efried | in u_p_t | |
| 17:02:11 | dansmith | efried: I don't think the virt drivers should be doing merging in the general case | |
| 17:02:27 | dansmith | efried: just being write-only for the majority of things, this ironic thing notwithstanding | |
| 17:02:28 | efried | dansmith: Then we need nontrivial redesign of u_p_t. | |
| 17:02:49 | dansmith | well I dunno what to say dude.. this is clearly a thing we missed in the design :) | |
| 17:03:20 | efried | The way it's currently designed: The resource tracker builds the ProviderTree, which includes all the providers' traits as known by placement, and passes that to u_p_t. The virt driver mungs the ProviderTree as it sees fit and then returns. Then resource tracker flushes those changes back to placement. | |
| 17:03:54 | dansmith | yeah, well, we can provide merging routines to the virt drivers to use on the tree, | |
| 17:04:00 | dansmith | and expect that most just use those | |