| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-16 | |||
| 14:31:49 | kosamara | efried: That's what I was converging to, I think that's best | |
| 14:32:05 | efried | this would give the admin the right clue as to why their device doesn't show up, if they were in fact expecting that field to be processed. | |
| 14:32:20 | kosamara | which satisfies my scenario | |
| 14:32:36 | sean-k-mooney | dansmith: that is true. | |
| 14:32:52 | efried | "WARNING: Device identification field XXX in inventory.yaml not recognized by the YYY driver; ignoring" | |
| 14:33:48 | sean-k-mooney | efried: do you need to raise an exception? | |
| 14:34:03 | efried | no, that's what kosamara and I are talking about. | |
| 14:34:11 | sean-k-mooney | sorry you said not raise an exception | |
| 14:34:41 | efried | If it's a common thing for configs to be copied around a cloud, then we should log and ignore. | |
| 14:35:36 | sean-k-mooney | efried: ya if we raise an exception from config parsing we normall hard crash the agent. but that seam wrong in this case | |
| 14:37:05 | efried | sean-k-mooney: This processing is happening in the compute service, in the virt driver's update_provider_tree method, on startup and/or SIGHUP, so I don't think it crashes the compute service; but it would make lots of things not work, depending where the exception was raised in update_provider_tree. | |
| 14:40:29 | sean-k-mooney | efried: sorry i was not being clear. it not where the parsing is done. we have two classes of config options. 1 where if its invalid we stop the agent form running and a second where we log a warning and continue on | |
| 14:40:46 | efried | sean-k-mooney: Note that this isn't oslo.config | |
| 14:40:49 | sean-k-mooney | this feels more like a warning to me then stop everything | |
| 14:41:31 | sean-k-mooney | efried: yes. the impentation is not what im concerned about its the semantics im thinking about | |
| 14:43:36 | efried | But, to avoid baffling the operator, we should log a warning that she'll see when she goes to investigate why her device isn't showing up. | |
| 14:43:36 | efried | For this reason, as well as in anticipation of future support for a global inventory.yaml on the conductor, it makes most sense *not* to treat an unrecognized field as an error. | |
| 14:43:36 | efried | Right. And semantically we converged on the fact that it's (apparently) fairly common for operators to write a config file once and then copy it around the cloud. | |
| 14:43:59 | efried | kosamara: Fair summary? ^ | |
| 14:44:40 | sean-k-mooney | efried: its not that they write them once it more most deployment tools you a base config template with minimal per host overrides so the confige are largly the same. | |
| 14:45:06 | efried | cool. | |
| 14:45:54 | sean-k-mooney | the downside to that is you typo somthing once and you break everything. | |
| 14:46:30 | sean-k-mooney | i may have hard locked 15 servers due to a config typo in the past.. | |
| 15:07:32 | melwitt | . | |
| 15:21:34 | jaypipes | melwitt: , | |
| 15:21:45 | melwitt | ! | |
| 15:21:48 | jaypipes | ? | |
| 15:21:58 | jaypipes | :P | |
| 15:22:08 | melwitt | we're at an impasse... | |
| 15:22:17 | jaypipes | melwitt: was just a friendly comma saying hi to your full stop. | |
| 15:22:32 | melwitt | heh | |
| 15:36:51 | kosamara | efried: yes | |
| 15:48:01 | openstackgerrit | Claudiu Belu proposed openstack/nova master: tests: autospecs all the mock.patch usages https://review.openstack.org/470775 | |
| 16:05:25 | sean-k-mooney | melwitt: dansmith. i have found an issue in my livemigration. specifcally there is a case where livemigration fails, the instance is still running on the soruce but the vif_type is none beacuse the host_id is not set. | |
| 16:05:52 | sean-k-mooney | i need to do more digging to localise the actull issue | |
| 16:06:06 | dansmith | because we update the instance info in the db from the failure yeah? | |
| 16:06:20 | dansmith | I think mriedem had a patch for that recently, IIRC | |
| 16:07:19 | dansmith | sean-k-mooney: does the healer periodic fix it after it runs? | |
| 16:07:39 | sean-k-mooney | im not sure. i think we may have activated the port binding for the dest and not reactivated for source. | |
| 16:08:28 | sean-k-mooney | dansmith: ill get the logs and file a propper bug | |
| 16:08:42 | dansmith | okay there was a situation where we saved the failed binding info cache recently that matt was working on, so might be related | |
| 16:08:46 | dansmith | but yeah | |
| 16:09:37 | sean-k-mooney | dansmith: i dont think its related as teh vif_type is none in neutron not binding_failed. | |
| 16:09:46 | dansmith | ah okay | |
| 16:23:35 | kosamara | efried: On the namespacing of driver-generated traits: if we demand that all driver-generated traits exist within a namespace, we don't allow the drivers to generate standard traits. | |
| 16:28:38 | kosamara | I propose to allow the driver to generate either namespaced or standard traits. | |
| 17:34:27 | openstackgerrit | melanie witt proposed openstack/nova-specs master: Move rocky implemented specs https://review.openstack.org/592622 | |
| 17:35:48 | openstackgerrit | Chris Dent proposed openstack/nova master: Add explanatory prefix to post_test_perf output https://review.openstack.org/591850 | |
| 17:35:49 | openstackgerrit | Chris Dent proposed openstack/nova master: Add trait query to placement perf check https://review.openstack.org/592624 | |
| 17:42:10 | openstackgerrit | melanie witt proposed openstack/nova-specs master: Dynamically find releases for move-implemented-specs https://review.openstack.org/592628 | |
| 17:54:38 | Sundar | Hi all, could somebody tell me the status of nested resource providers, including virt driver changes (to avoid stomping on custom resources)? If there is a bp to track it, please point me to that. Thanks! | |
| 17:55:57 | melwitt | I think the placement side is considered done, but none of the nova integration side has been started yet. I'm not aware of a bp tracking it yet | |
| 17:58:55 | Sundar | melwitt: Thanks. So, in particular, the work to prevent virt drivers from overwriting custom resources has not started yet, right? If so, what is the blocker -- is it still upgrade considerations? | |
| 17:59:54 | dansmith | Sundar: resources or traits? | |
| 18:00:38 | Sundar | dansmith: I think both. When last checked, the virt drivers just overwrite the inventory of the compute node. | |
| 18:00:54 | dansmith | well, nova-compute does that with the help of the virt driver, | |
| 18:01:00 | openstackgerrit | melanie witt proposed openstack/nova-specs master: Add a script for counting blueprints https://review.openstack.org/581914 | |
| 18:01:09 | dansmith | but inventory of a provider should only be owned by a single entity, | |
| 18:01:27 | sean-k-mooney | Sundar: in dublin we said external service were not allowed to create inventoei in resouce providers they did not create | |
| 18:01:30 | dansmith | and since there is only one provider for a compute node right now, overwriting inventory entirely is kinda expected | |
| 18:01:38 | dansmith | right, what sean-k-mooney said | |
| 18:01:50 | dansmith | so until the providers start reporting things as nested, | |
| 18:02:07 | dansmith | there's no real opportunity for the coexistence of other nested providers with their own inventory they manage | |
| 18:02:16 | efried | kosamara: Yes, tricky question as to who "owns" a standard trait. | |
| 18:02:19 | dansmith | and that is dependent on the upgrade bit getting worked out | |
| 18:02:37 | Sundar | Yes. Even with nested RPs, we had the issue that the libvirt driver would overwrite the nested RP config. This was about a month or so ago. | |
| 18:03:31 | dansmith | Sundar: because it's not nested-aware, AFAIK | |
| 18:03:42 | efried | true ^ | |
| 18:04:02 | efried | dansmith: The upgrade (reshape) series is ready for review btw. | |
| 18:04:37 | dansmith | yeah I know | |
| 18:04:46 | efried | :P | |
| 18:05:24 | sean-k-mooney | Sundar: cyborg will have to create a seperate resouce provider tree under the compute node. it should be possible to do that today but im not sure if the virt driver will remove them or not | |
| 18:05:29 | efried | kosamara: Do you understand the concern about trait "ownership" in general? | |
| 18:05:47 | efried | sean-k-mooney: The virt driver won't remove them; but we also won't schedule to them either. | |
| 18:06:05 | sean-k-mooney | efried: can you allocate from them | |
| 18:06:32 | sean-k-mooney | the intent would to still schduler to the compute node but consume the cyborg resouce from the cyborg subtrees | |
| 18:06:42 | efried | sean-k-mooney: basically, that whole code path has some bits that will work, some bits that definitely won't, and some that are still a mystery. This is work that needs to be done in Stein. | |
| 18:07:08 | efried | sean-k-mooney: I think the scheduler *should* consume the cyborg resources. | |
| 18:07:20 | efried | It's just not allowed to muck with the cyborg providers' inventories or traits. | |
| 18:07:23 | Sundar | sean-k-mooney: Yes, cyborg is trying to do nRPs. We hit the virt driver overwrite issue and developers decided to wait for that to be resolved. | |
| 18:07:54 | sean-k-mooney | efried: right. we may also have to revisit the idea of have a seperate root node just for the phyical server and then have per projec subtrees | |
| 18:08:31 | sean-k-mooney | Sundar: again if the overriete issue is you tried to create invtories on the comptue node resouce provider that will never be supported | |
| 18:08:33 | efried | revisit? I don't remember visiting that in the first place, ever. Per-project subtrees? | |
| 18:09:14 | sean-k-mooney | efried: it came up in dublin on the friday when we were all tired | |
| 18:09:38 | Sundar | sean-k-mooney: no, we created nested RPs within the compute node. | |
| 18:09:42 | efried | I think Sundar is talking about creating child providers of the compute node provider but having them ignored by the scheduler and/or deleted by the resource tracker. The latter may have happened in libvirt when it was still using get_inventory. | |
| 18:09:56 | Sundar | Yes ^ | |
| 18:09:58 | efried | I'm guessing you haven't tried it since we implemented libvirt's update_provider_tree method. | |
| 18:10:14 | Sundar | efried: when was that done? | |
| 18:10:25 | efried | Sundar: So if you have the resources and inclination, you could try that again. I guarantee it will still break, but it may get slightly further than it did before. | |
| 18:10:29 | efried | Sundar: Let me find the patch... | |
| 18:11:01 | sean-k-mooney | efried: the idea being that nova would nolonger own the root node of the tree. the root would have no invetores and just represent teh server then each service on the serve would create there own nested resouce provider beneate that root node | |
| 18:11:18 | Sundar | efried: "I guarantee it will still break" -- that won't help me persuade other Cyborg developers to start moving fwd :) | |
| 18:11:26 | efried | Sundar: https://review.openstack.org/#/c/560444/ merged 5/25 | |
| 18:11:53 | sean-k-mooney | efried: it was a fallback incase we did not figure out how to have multple service own different nodes under the current tree create by nova | |
| 18:11:54 | efried | sean-k-mooney: Nono, nova still owns the root provider. But nova does *not* own the device providers - cyborg owns those. | |
| 18:12:04 | efried | oh, sorry, you're talking about the project-based thing. | |
| 18:12:27 | sean-k-mooney | efried: ya sorry i was | |
| 18:12:58 | efried | sean-k-mooney: Without further placement API work, we would probably have to do that by lassoing the project subtree in an aggregate corresponding to the project ID. | |
| 18:13:26 | sean-k-mooney | efried: by project id you mean service id? | |