| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-20 | |||
| 14:15:23 | jroll | blah | |
| 14:15:28 | dansmith | right | |
| 14:15:46 | mriedem | Hypervisor: VCPU information unavailable _report_hypervisor_resource_view /usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py:676 | |
| 14:15:53 | mriedem | indicates the node is not available, right ^ ? | |
| 14:16:40 | jroll | mriedem: yeah, but later it figures things out: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/12d034e/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-03-19_07_14_43_029 | |
| 14:16:51 | EmilienM | (in the meantime, we're testing the revert here: https://review.openstack.org/#/c/447514) | |
| 14:17:03 | jroll | log level filters here would be super useful, btw :P | |
| 14:17:34 | dansmith | jroll: yeah :( | |
| 14:17:41 | EmilienM | jroll: it's in our (long) todo | |
| 14:17:48 | dansmith | jaypipes: a scheduling decision looks like /resource_providers?stuff=stuff right? | |
| 14:17:49 | EmilienM | but yeah, I agree | |
| 14:17:51 | dansmith | jaypipes: in placement | |
| 14:17:58 | jaypipes | dansmith: yep. | |
| 14:18:05 | dansmith | jaypipes: I don't see a single one of those in placement's log | |
| 14:18:15 | jroll | I do | |
| 14:18:22 | jroll | GET /placement/resource_providers?resources=DISK_GB%3A40%2CMEMORY_MB%3A4096%2CVCPU%3A1 | |
| 14:18:23 | dansmith | I think we should be asking placement, then using that list of uuids to filter the compute_nodes list right? | |
| 14:18:37 | dansmith | jroll: ah, yeah, okay | |
| 14:18:47 | jaypipes | EmilienM: jroll is referring to better logging in the scheduler/placement service. | |
| 14:18:48 | dansmith | must have fat-fingered my search | |
| 14:19:09 | jroll | jaypipes: no, I'm referring to being able to filter compute logs by log level in tripleo logs :) | |
| 14:19:10 | mriedem | jaypipes: i think it's per-job | |
| 14:19:13 | mriedem | right | |
| 14:19:16 | mriedem | we have filtering on the dsvm jbos | |
| 14:19:17 | EmilienM | jaypipes: ah ok. I was talking about the capacity of filtering logs like devstack does | |
| 14:19:18 | mriedem | *jobs | |
| 14:19:21 | jaypipes | dansmith: right, but placement is returning 0 providers. | |
| 14:19:30 | dansmith | jaypipes: is it? | |
| 14:19:49 | jroll | looks like it to me | |
| 14:19:56 | jroll | "Starting with 0 host(s) get_filtered_objects" | |
| 14:19:59 | dansmith | okay that's different than no compute node records | |
| 14:20:02 | dansmith | jroll: no, that's too late to know | |
| 14:20:08 | jroll | is it? | |
| 14:20:10 | jroll | lol | |
| 14:20:11 | jaypipes | dansmith: "Starting with 0 hosts" | |
| 14:20:23 | dansmith | jaypipes: that's after we've correlated the compute nodes and the placement result | |
| 14:21:03 | jroll | ah, yeah | |
| 14:21:34 | dansmith | I wanna know if placement is returning zero, or we have no CNs in the DB | |
| 14:21:38 | dansmith | I have been assuming the latter | |
| 14:21:46 | dansmith | since we see us reporting from the compute node to placemetn | |
| 14:21:48 | jroll | ya, needs more logging | |
| 14:22:59 | EmilienM | is the log level of tripleo jobs is enough? let me know if I can help | |
| 14:23:23 | jroll | EmilienM: it's a failure to log things in nova code, not the log level | |
| 14:23:25 | jaypipes | EmilienM: yeah, it is I think. it's just we need more debug logging in nova-scheduler. | |
| 14:23:30 | openstackgerrit | Dan Smith proposed openstack/nova master: Log placement result in scheduler https://review.openstack.org/447520 | |
| 14:23:32 | dansmith | this ^ | |
| 14:23:44 | jaypipes | thx dansmith | |
| 14:23:51 | jroll | ++ | |
| 14:24:13 | dansmith | EmilienM: can you make something run on top of that log patch? | |
| 14:24:26 | mriedem | that's going to blow up | |
| 14:24:53 | mriedem | that's just a debug change right? are we planning on merging that? | |
| 14:24:59 | dansmith | mriedem: no, just for debug | |
| 14:25:17 | dansmith | mriedem: the real one need not log rps in the warning case I think | |
| 14:25:18 | dansmith | I just wanted to know if it's empty or none | |
| 14:25:19 | mriedem | please DNM prefix the commit message or something, or -2 | |
| 14:25:21 | jroll | can we merge it just so mriedem gets slammed with logs? | |
| 14:25:38 | mriedem | dansmith: agree, but i could see someone mistakenly approving this | |
| 14:25:56 | EmilienM | dansmith: yes | |
| 14:26:00 | EmilienM | my revert | |
| 14:26:04 | EmilienM | oh wait | |
| 14:26:06 | EmilienM | no | |
| 14:26:11 | openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Log placement result in scheduler https://review.openstack.org/447520 | |
| 14:26:12 | EmilienM | gimme a sec | |
| 14:28:02 | EmilienM | dansmith: here: https://review.openstack.org/#/c/447522/ | |
| 14:31:27 | dansmith | jroll: yeah, | |
| 14:31:29 | openstackgerrit | Lee Yarwood proposed openstack/nova master: encryptors: Switch to os-brick encryptor classes https://review.openstack.org/391597 | |
| 14:31:40 | efried | So it turns out that assert_called_once and assert_not_called aren't actually methods on a mock. Without speccing, they'll succeed happily just like any nonexistent method would. There's 80ish instances of these in the nova tree. May I open a bug and fix this, or is there something I don't know? | |
| 14:31:41 | dansmith | I don't even see some debug logs that we should be seeing in scheduler | |
| 14:31:59 | jroll | the only thing I've noticed so far is that the flavor isn't an exact match to the node | |
| 14:32:01 | jroll | oh, hm | |
| 14:32:03 | dansmith | like we should see this: https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L590-L591 | |
| 14:32:12 | dansmith | but unless I just suck at searching I can't find it | |
| 14:32:34 | jroll | agree | |
| 14:33:11 | mriedem | efried: are you looking at std lib or the mock library? | |
| 14:33:36 | efried | mriedem How do I know which is which? | |
| 14:33:40 | mriedem | efried: b/c they are in the lib https://github.com/testing-cabal/mock/blob/2.0.0/mock/mock.py#L890 | |
| 14:33:41 | mriedem | and we use the lib | |
| 14:33:44 | mriedem | in the py27 job | |
| 14:34:00 | mriedem | https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L149 | |
| 14:34:19 | efried | What I do know is that the tests I found this out on are using 'import mock'. | |
| 14:34:23 | jroll | dansmith: I wonder how relevant this is compute_manager=ironic.nova.compute.manager.ClusteredComputeManager | |
| 14:34:27 | jroll | that should not be there | |
| 14:34:33 | dansmith | jroll: w..t..f | |
| 14:34:35 | efried | So I guess that behaves differently depending on whether the mock library is installed or not? | |
| 14:34:43 | jroll | dansmith: wait, I think that's not even in ironic tree, wat | |
| 14:34:47 | efried | ...the import, that is? | |
| 14:35:03 | dansmith | jroll: that would be majorly problematic for a lot of things we've just done | |
| 14:35:11 | jroll | dansmith: indeed | |
| 14:35:26 | jroll | dansmith: confirmed, that module doesn't exist in ironic master | |
| 14:35:37 | dansmith | ummm.... | |
| 14:35:41 | mriedem | you can't load up out of tree compute managers anymore | |
| 14:35:50 | mriedem | we dropped the ability to do that awhile ago, like newton i think | |
| 14:35:51 | jroll | mriedem: so that config is a noop yeah? | |
| 14:35:57 | mriedem | most likely | |
| 14:36:00 | jroll | ok | |
| 14:36:03 | dansmith | oh, compute manager, I was thinking host manager | |
| 14:36:17 | jroll | efried: we always use the third party lib, which is api-compatible with the 3.6 version of stdlib | |
| 14:36:39 | mriedem | http://codesearch.openstack.org/?q=ironic.nova.compute.manager.ClusteredComputeManager&i=nope&files=&repos= | |
| 14:36:41 | efried | jroll Okay, must be something in my local env that's using the stdlib instead, then. | |
| 14:36:44 | openstackgerrit | Sean Dague proposed openstack/nova master: update i18n guide for nova https://review.openstack.org/447528 | |