| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-13 | |||
| 12:46:04 | artom | sfinucan, pretty sure | |
| 12:46:51 | artom | sfinucan, the little ones are waking up, let me drive them to daycare and I'll continue after | |
| 12:47:02 | sfinucan | artom: (y) | |
| 12:56:11 | erol | hi everyone, I wonder Is there any way to flush Nova Schedular RetryFilter count? I hit max attempt. What can i do now? | |
| 12:56:16 | sean-k-mooney | sc68cal: if you get a chance can you review https://review.openstack.org/#/c/444788 it will fix our third party ci after https://review.openstack.org/#/c/435575/ added a test requirement that break ovs compilation | |
| 12:57:01 | sean-k-mooney | sc68cal: it should also give a speed up in the ci jobs as less stuff needs to be downloaded which is a nice side effect | |
| 12:58:19 | sean-k-mooney | erol: usually you delete the instance and try again but if you hit max attempts its often because there was a problem with the request | |
| 12:58:48 | sean-k-mooney | erol: either no host satifies the request or you have an infra issue that is stoping the cloud from working correctly | |
| 13:03:37 | erol | sean-k-mooney: Thanks. I try to run Ironic and have an issue on provisioning. After I attempted several times. I came a cross with these output in schedular log http://paste.openstack.org/show/602488/ | |
| 13:12:39 | openstackgerrit | Rafael Folco proposed openstack/nova master: DELETE all inventory for a resource provider https://review.openstack.org/416669 | |
| 13:12:40 | openstackgerrit | Rafael Folco proposed openstack/nova master: Raising http codes on old microversion https://review.openstack.org/444497 | |
| 13:41:58 | openstackgerrit | Sergey Nikitin proposed openstack/nova master: [WIP] PIC NUMA policy PoC https://review.openstack.org/390520 | |
| 13:48:55 | mriedem | o/ | |
| 13:51:55 | dansmith | alex_xu: you still around? | |
| 13:55:15 | bauzas | oh, you US folks already had DST shift ? | |
| 13:55:20 | edleafe | Scheduler subteam meeting in 5 minutes in #openstack-meeting-alt | |
| 13:55:35 | leakypipes | rfolco: were you working on adding error codes (in addition to error messages) to the placement API? can't remember if that was you or diga... | |
| 13:55:35 | mriedem | bauzas: that was this weekend | |
| 13:55:36 | edleafe | bauzas: yep - yeaterday | |
| 13:56:10 | mriedem | leakypipes: neither https://review.openstack.org/#/c/418393/ | |
| 13:56:22 | leakypipes | mriedem: ah! | |
| 13:56:30 | leakypipes | it was _gryf | |
| 13:57:17 | bauzas | mriedem: edleafe: cool then, France is now closer to the US :) | |
| 13:58:01 | rfolco | leakypipes, morning. Not me. | |
| 13:59:45 | leakypipes | rfolco: np :) thanks! | |
| 14:00:40 | edleafe | leakypipes: time to plug the leak | |
| 14:00:49 | jaypipes | :) | |
| 14:04:43 | _gryf | yup, I did that bp | |
| 14:05:35 | markus_z | mriedem: JFYI, the VNC issue we talked about 1w ago will be fixed with: https://review.openstack.org/#/c/444164/ | |
| 14:07:34 | markus_z | mriedem: I also have to pester you about: https://review.openstack.org/#/c/406898/ All info you asked for is added to that review. | |
| 14:08:06 | mriedem | ok | |
| 14:09:10 | openstackgerrit | dane-fichter proposed openstack/nova-specs master: Trusted flavors https://review.openstack.org/312225 | |
| 14:10:01 | sfinucan | jaypipes: Might you be so kind as to stick this on your review backlog for this week? https://review.openstack.org/#/c/361140/ | |
| 14:10:31 | sfinucan | johnthetubaguy too ^, if you could | |
| 14:12:30 | dane-fichter | mriedem , sdague : Could you guys take a look at my change adding the security tempest tests as an experimental check in Nova? https://review.openstack.org/#/c/443167/ | |
| 14:13:59 | mriedem | dane-fichter: sure | |
| 14:14:11 | dane-fichter | mriedem: thanks! | |
| 14:18:57 | jaypipes | sfinucan: ya | |
| 14:19:19 | sfinucan | jaypipes: Thank you, sir | |
| 14:19:35 | jaypipes | sfinucan: np. you know I love my numa. :P | |
| 14:20:11 | sfinucan | jaypipes: I saw numa'y stuff and decided to r̶u̶i̶n̶ make your day | |
| 14:22:43 | mriedem | markus_z: the multinode vnc fix looks good to me, that's even called out in the devstack multinode docs | |
| 14:24:04 | jaypipes | sfinucan: :) | |
| 14:25:33 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Add a note about 'cells_api' https://review.openstack.org/444926 | |
| 14:26:18 | dansmith | johnthetubaguy: alex_xu: I'm totally lost.. where does cachingscheduler override things in HostManager? | |
| 14:26:52 | johnthetubaguy | dansmith: it uses get_all_hosts() basically | |
| 14:27:02 | johnthetubaguy | rather than get_by_uuid | |
| 14:27:07 | johnthetubaguy | s | |
| 14:27:42 | johnthetubaguy | dansmith: https://github.com/openstack/nova/blob/master/nova/scheduler/caching_scheduler.py#L75 | |
| 14:28:09 | johnthetubaguy | dansmith: might be time to deprecate that thing soon, although I was hoping to wait until we do allocations in the conductor really | |
| 14:28:17 | dansmith | johnthetubaguy: okay the main scheduler uses that purely to prime the data structures, but not for an actual schedule.. wouldn't the caching scheduler only cache hosts that are valid for a particular spec_obj if we made that change? | |
| 14:28:35 | johnthetubaguy | dansmith: no, it caches the full list of hosts | |
| 14:28:48 | johnthetubaguy | and bypasses the placement PAI | |
| 14:28:49 | johnthetubaguy | API | |
| 14:29:41 | johnthetubaguy | dansmith: did we have moving allocations into the conductor pencilled in for this cycle yet? | |
| 14:29:52 | dansmith | johnthetubaguy: right, but.. I don't understand what you think needs to change.. the _all_cells method takes a spec_obj only if you want to cause it to limit the host list based on a requested cell, | |
| 14:29:56 | dansmith | which you'd never want to do with that right? | |
| 14:30:30 | johnthetubaguy | dansmith: oh, sorry, I see your point now... it just doesn't work for the caching scheduler | |
| 14:30:43 | dansmith | right | |
| 14:31:31 | dansmith | johnthetubaguy: either we'd have to honor the cell request somewhere else, or just not have this happen if you're using cachingscheduler | |
| 14:31:32 | dansmith | I dunno who all uses it, tbh, so I guess I'm not sure the implications of that | |
| 14:32:29 | johnthetubaguy | dansmith: large deployments currently suck without using that, like >100 nodes or something like that, so probably only the large ones | |
| 14:32:42 | dansmith | johnthetubaguy: what about large deployments now that we have placement? | |
| 14:32:54 | dansmith | johnthetubaguy: we should be getting the host list waaayyy faster now as of this cycle | |
| 14:32:57 | johnthetubaguy | dansmith: they are much worse, if I understand it correctly | |
| 14:33:06 | dansmith | WAT | |
| 14:33:10 | johnthetubaguy | yep | |
| 14:33:13 | dansmith | why? | |
| 14:33:13 | vsaienk0 | mriedem: could you please approve https://blueprints.launchpad.net/nova/+spec/ironic-hotplug-interfaces for Pike, the code is ready on both nova and ironic side, and it is very small change which already covered by CI tests. It would be great to have this implemented in Pike | |
| 14:33:24 | johnthetubaguy | dansmith: we don't do the allocations soon enough | |
| 14:33:40 | dansmith | jaypipes: ^ | |
| 14:33:42 | johnthetubaguy | dansmith: so you quickly hit the three retries, because everyone picks the same host | |
| 14:33:53 | dansmith | johnthetubaguy: placement makes that _worse_ ? | |
| 14:34:08 | dansmith | I thought cachingscheduler was for get_all_hosts() performance... | |
| 14:34:13 | johnthetubaguy | dansmith: worse than using caching scheduler, I mean | |
| 14:34:25 | johnthetubaguy | dansmith: its performance, and caching of results between API calls | |
| 14:34:54 | dansmith | johnthetubaguy: does cachingscheduler affect allocation timing though? I thought it was just about finding the list of hosts quickly, which placement *should*be improving | |
| 14:35:00 | johnthetubaguy | jaypipes: in my head I had us killing the caching scheduler once allocations are written in the conductor | |
| 14:35:40 | johnthetubaguy | dansmith: no, its also about that cache being updated with the previous results, thats why it only works if you have one nova-scheduler, and fails badly if you have two caching schedulers | |
| 14:35:57 | dansmith | johnthetubaguy: ugh, okay | |
| 14:36:13 | johnthetubaguy | dansmith: yeah, its really non-obvious from the code thats how it works | |
| 14:37:24 | johnthetubaguy | dansmith: that logic that makes when we scheduler a request of 5 builds at once not all pick the same host activates for every request using the same cached list | |
| 14:37:26 | mriedem | vsaienk0: done | |
| 14:37:57 | vsaienk0 | mriedem: thanks! | |
| 14:38:07 | jaypipes | johnthetubaguy: I'd appreciate a hangout discussion with you about the caching scheduler and the placement service. | |
| 14:38:15 | jaypipes | johnthetubaguy: do you have some time to discuss it now? | |
| 14:38:19 | dansmith | johnthetubaguy: I guess I'm surprised that's a real problem with randomizing the result list and sufficient capacity | |
| 14:38:22 | dansmith | jaypipes: I want in on that | |
| 14:38:28 | dansmith | because cells. | |
| 14:38:34 | mriedem | can't you configure the scheduler to spread rather than pack? or does just no one do that in production? | |
| 14:38:37 | jaypipes | dansmith: I figured as much :) | |
| 14:38:41 | johnthetubaguy | jaypipes: dansmith: I have 20 mins now if you are both free? | |
| 14:38:47 | dansmith | I am | |
| 14:38:51 | jaypipes | me too. | |
| 14:38:52 | mriedem | i'm jumping in too | |
| 14:38:54 | jaypipes | sec, getting link. | |
| 14:38:58 | johnthetubaguy | mriedem: right, no one wants spread, because you run our of large instances | |
| 14:39:23 | jaypipes | https://hangouts.google.com/call/s6p6dqm3dfbyxjwu5ppyhj4y2ae | |
| 14:39:54 | jaypipes | mriedem: ^ | |
| 14:40:21 | mriedem | yeah, damn firefox doesn't support hangouts anymore on the latest version | |