| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-20 | |||
| 14:42:26 | mriedem | *see | |
| 14:42:44 | dansmith | I don't think so, but let me look | |
| 14:42:50 | jroll | shouldn't be | |
| 14:42:59 | dansmith | yeah, no | |
| 14:43:58 | dansmith | I think the nova hash is 47d90a3 | |
| 14:44:15 | dansmith | in this run http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/12d034e/logs/rpm-qa.txt.gz | |
| 14:44:34 | EmilienM | dansmith: yes that | |
| 14:45:18 | dansmith | so actually, if we get nothing back from placement, we'd not hit the messages I'm looking for, I think | |
| 14:45:19 | dansmith | so it must be that | |
| 14:45:20 | dansmith | but I have no idea why | |
| 14:45:27 | EmilienM | ok so I need to remove ironic.nova.compute.manager.ClusteredComputeManager | |
| 14:45:39 | dansmith | EmilienM: yeah, that's just a dead config opt | |
| 14:45:48 | EmilienM | replaced by what? | |
| 14:45:56 | dansmith | not replaced by anything | |
| 14:45:58 | mriedem | EmilienM: https://github.com/openstack/nova/commit/d9a7c16c305cd7016195cada883867411d3a5eb9 | |
| 14:46:28 | EmilienM | ok | |
| 14:46:35 | jroll | replaced by AWESOME | |
| 14:47:06 | EmilienM | do you think removing the option would help tripleo issue? | |
| 14:47:07 | jroll | EmilienM: the need for that was replaced by https://github.com/openstack/nova/commit/6047d790a32ef5a65d4d6b029f673ce53c3d4141 | |
| 14:47:10 | jroll | no | |
| 14:47:11 | EmilienM | or just cleanup | |
| 14:47:17 | jroll | just cleanup | |
| 14:47:18 | EmilienM | k | |
| 14:47:56 | EmilienM | "There's no configuration to enable this mode; it's always running." | |
| 14:48:11 | EmilienM | jroll: ❤ | |
| 14:48:33 | jroll | :) | |
| 14:48:57 | mriedem | seems wasteful for FilterScheduler._schedule to even call HostManager.get_filtered_hosts if we already know we don't have any hosts (resource providers from placement) | |
| 14:49:32 | openstackgerrit | Gábor Antal proposed openstack/nova master: Add json style checking for sample notifications https://review.openstack.org/443677 | |
| 14:49:35 | mriedem | but maybe the spaghetti makes sense someohw | |
| 14:49:37 | dansmith | mriedem: right and we don't | |
| 14:50:02 | dansmith | mriedem: https://github.com/openstack/nova/blob/master/nova/scheduler/filter_scheduler.py#L175-L176 | |
| 14:50:03 | openstack | Launchpad bug 1674375 in puppet-nova "compute_manager is deprecated, we should stop using it" [High,Triaged] - Assigned to Emilien Macchi (emilienm) | |
| 14:50:03 | EmilienM | jroll: /me taking care of this cleanup https://bugs.launchpad.net/puppet-nova/+bug/1674375 | |
| 14:50:18 | mriedem | dansmith: i'm talking about this https://github.com/openstack/nova/blob/master/nova/scheduler/filter_scheduler.py#L108 | |
| 14:50:21 | jroll | EmilienM: nice, thanks! s/deprecated/removed/ btw :) | |
| 14:50:23 | mriedem | hosts is [] as you just poined out | |
| 14:50:25 | mriedem | *pointed | |
| 14:50:38 | mriedem | but we still go through self.host_manager.get_filtered_hosts and all of it's checks on an empty list | |
| 14:50:43 | mriedem | which seems like a waste of cycles | |
| 14:50:53 | openstackgerrit | Silvan Kaiser proposed openstack/nova master: Removes getfattr from Quobyte Nova driver https://review.openstack.org/428646 | |
| 14:50:54 | dansmith | mriedem: ah, well, it's empty, but yeah I see what you mean | |
| 14:50:56 | mriedem | unrelated to this problem | |
| 14:50:57 | dansmith | not very impactful | |
| 14:50:59 | dansmith | yeah | |
| 14:51:10 | mriedem | but i thought we were always paranoid about scheduler perf | |
| 14:51:18 | mriedem | like, no logging | |
| 14:51:40 | dansmith | that will fail after the first filter, though, | |
| 14:51:43 | dansmith | and the first one is retryfilter, | |
| 14:51:46 | dansmith | which does basically nothing | |
| 14:51:50 | dansmith | so it's really small I think | |
| 14:51:58 | dansmith | still wasteful, but.. minor I beleive | |
| 14:51:59 | mriedem | first default filter | |
| 14:52:01 | mriedem | sure | |
| 14:52:08 | dansmith | sure, first default | |
| 14:52:21 | EmilienM | jroll: done, thx | |
| 14:52:37 | mriedem | the first filter in my custom deployment actually calls out to my backend inventory datastore and gathers all of the latest goodies, just to filter them on an empty list :) | |
| 14:52:49 | bauzas | dansmith: EmilienM: just seeing your discussions, any questions I could help you ? | |
| 14:53:18 | dansmith | bauzas: we don't need any more questions.. we need ANSWERS! | |
| 14:53:27 | dansmith | mriedem: that's quite unfortunate :) | |
| 14:53:40 | bauzas | so, I could help for answers :p | |
| 14:53:41 | dansmith | mriedem: especially for that to be before retryfilter :D | |
| 14:53:58 | mriedem | it also ssh's into all of my computes and gathers data | |
| 14:54:00 | mriedem | it's not very smart | |
| 14:54:04 | mriedem | but it's enterprise ready | |
| 14:54:07 | dansmith | lol | |
| 15:05:09 | mriedem | how do we know which flavor was actually used in the server create request? | |
| 15:05:22 | mriedem | EmilienM: you pointed at http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/12d034e/logs/undercloud/var/log/nova/nova-api.txt.gz#_2017-03-19_06_38_20_764 but there are more than that flavor | |
| 15:05:23 | dansmith | mriedem: you can look at the placement request | |
| 15:05:29 | mriedem | ok | |
| 15:05:46 | mriedem | we don't correlate the request ids between the nova and placement services yet do we? | |
| 15:05:46 | dansmith | mriedem: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/12d034e/logs/undercloud/var/log/nova/nova-placement-api.txt.gz#_2017-03-19_07_19_34_113 | |
| 15:05:58 | dansmith | mriedem: bauzas was just telling me that we do | |
| 15:06:10 | mriedem | cdent: you were the last to look at that i thought | |
| 15:06:32 | dansmith | mriedem: looks like the request I linked matches the 40,4g,1 flavor | |
| 15:06:42 | EmilienM | jroll: https://review.openstack.org/#/q/topic:bug/1674375 | |
| 15:06:43 | bauzas | dansmith: mriedem: enf of Ocata-3, we were not yet passing req-id | |
| 15:07:02 | bauzas | dansmith: mriedem: but I thought cdent threw a patch for that | |
| 15:07:03 | mriedem | i'm looking for req-702e96af-318d-4611-af19-d6a1f3321c6a | |
| 15:07:09 | mriedem | http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/12d034e/logs/undercloud/var/log/nova/nova-api.txt.gz#_2017-03-19_07_19_33_201 | |
| 15:07:13 | mriedem | that's the ceph storage instance | |
| 15:07:41 | mriedem | i know we log the placement request id from the response in the scheduler report client | |
| 15:07:54 | mriedem | but we don't pass and use a request id from n-sch to placement and log with that req id in the placement logs | |
| 15:07:57 | mriedem | which is what we need | |
| 15:08:16 | jroll | we don't do that between n-cpu and ironic either :( | |
| 15:08:17 | dansmith | mriedem: they all look the same to me | |
| 15:08:26 | openstackgerrit | Sean Dague proposed openstack/nova master: update i18n guide for nova https://review.openstack.org/447528 | |
| 15:08:39 | mriedem | but yeah otherwise GET /placement/resource_providers?resources=DISK_GB%3A40%2CMEMORY_MB%3A4096%2CVCPU%3A1 | |
| 15:09:41 | bauzas | mriedem: last time I debugged that, I was just looking at the time instead :( | |
| 15:10:04 | mriedem | in the n-cpu logs, i see the 3 nodes reporting capacity just before that | |
| 15:10:07 | mriedem | so those are all good | |
| 15:10:34 | mriedem | at least i n_report_hypervisor_resource_view | |
| 15:10:42 | cdent | bauzas, mriedem: not fully contextualized, but I improved some of the logging, but there was no super handy way to make the request id used on the nova side also because used on the placement id, it would require an additional header, or changing the middleware | |
| 15:10:53 | mriedem | i figured we could pass a header | |
| 15:11:05 | mriedem | and if the header is there, use it, else generate a new request id | |
| 15:11:31 | cdent | yeah, that would require changing the oslo middleware, so I stalled out | |
| 15:11:46 | dansmith | don't we already have a way to do that between services? | |
| 15:11:52 | dansmith | I thought we did between nova and glance, for example | |
| 15:12:01 | cdent | I think I left that as a comment on the bug, but it may have been lost to the sands of time | |
| 15:12:14 | mriedem | i think it's something that keeps coming up with operators | |
| 15:12:43 | mriedem | sdague might know | |