| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-26 | |||
| 19:15:04 | mriedem | jaypipes: comments / questions inline | |
| 19:15:33 | erhudy | eandersson: if every other request is failing it sounds like it's trying to round-robin between the memcached instances and can hit one but not the other | |
| 19:15:38 | jaypipes | edleafe: because it's not related... it's a different branch. | |
| 19:15:59 | jaypipes | edleafe: if the auto-import merges first, I'll quickly rebase. no biggie. | |
| 19:16:01 | erhudy | so, iptables or memcached configuration seem like possible culprits | |
| 19:16:17 | mriedem | melwitt: https://review.openstack.org/#/c/460284/ | |
| 19:16:43 | eandersson | erhudy, every other nova service works fine though | |
| 19:16:56 | eandersson | it's just nova-novnc that is having issues with this configuration | |
| 19:17:17 | edleafe | jaypipes: just thought I'd save ya some of that ugly typing... :-P | |
| 19:17:40 | eandersson | but yes it looks like it is round-robin | |
| 19:18:16 | erhudy | maybe strace nova-novncproxy and see what it's sad about? strace is usually what i turn to in Mysterious Situations | |
| 19:19:05 | erhudy | or strace memcached, even | |
| 19:19:25 | erhudy | once upon a time i had strange issues after going kilo->liberty and strace revealed it was because liberty was using memcached more aggressively and memcached had silently run out of connections | |
| 19:19:54 | erhudy | we can take this over to #openstack though instead of filling up dev chat with ops stuff | |
| 19:21:45 | melwitt | mriedem: +W | |
| 19:27:13 | melwitt | jaypipes: thanks for braving the quota patches. working on replies | |
| 19:32:01 | jaypipes | mriedem: answerd. | |
| 19:32:16 | jaypipes | melwitt: sure, no prob. it's a mess in there :) | |
| 19:34:47 | melwitt | jaypipes: I tried to make it make sense but I failed, so sorry about that :( | |
| 19:35:00 | jaypipes | melwitt: and you're not the cause of the indecipherability :) | |
| 19:35:33 | melwitt | my goal was to code comment it enough to where laypersons could understand it | |
| 19:36:47 | melwitt | because learning it myself was extremely painful and wanted to make it easier for others | |
| 19:37:41 | melwitt | jaypipes: the dicts returned from count() are evil and that was me, I will try to think of another way | |
| 19:38:06 | jaypipes | melwitt: yeah, I hear ya. definitely not bemoaning your code. it's just a pit of vipers in general. | |
| 19:38:35 | melwitt | I know. not taken that way :) | |
| 19:38:44 | jaypipes | k, cool :) | |
| 19:43:00 | mriedem | jaypipes: ok, replied in https://review.openstack.org/#/c/460281/ as well, | |
| 19:43:05 | melwitt | mriedem: been meaning to tell you about this, I found a bug in the BackOffLoopingCall that was causing that ironic test to timeout intermittently (160s test timeout whereas the backoff timeout should have expired well before that) https://review.openstack.org/#/c/459790/ | |
| 19:43:22 | mriedem | jaypipes: also mentioned at a higher level that i don't know that the plan as you describe it in there is actually written down anywhere | |
| 19:44:25 | mriedem | melwitt: heh, nice, i guess that really bugged you | |
| 19:45:31 | jaypipes | mriedem: I got dinged for being too verbose in specs, then dinged for not having everything written down ahead of time... | |
| 19:45:38 | jaypipes | can't win for losin. | |
| 19:46:02 | melwitt | mriedem: yup, I became obsessed with it | |
| 19:46:17 | mriedem | jaypipes: i know | |
| 19:46:32 | jaypipes | mriedem: I'll try to write something up tomorrow. | |
| 19:46:46 | mriedem | jaypipes: i'm +2 on the change after you explained the plan, i just wouldn't have known otherwise | |
| 19:46:54 | jaypipes | understood. | |
| 19:52:37 | mordred | jaypipes: write down more things, but then delete hide them | |
| 19:52:47 | jaypipes | heh | |
| 19:59:54 | mriedem | jaypipes: dansmith: anything off the top of your head that we already know is going to be a focus for queens? besides just anything that doesn't make pike? things that come to mind are volume multiattach, cinder ephemeral / no local disk | |
| 19:59:58 | mriedem | removing caching scheduler isn't really a feature | |
| 20:00:08 | mriedem | but claims in scheduler would be required in queens | |
| 20:00:22 | jaypipes | mriedem: VGPU. | |
| 20:00:34 | dansmith | mriedem: full shared storage, shared network reporting and stuff | |
| 20:00:41 | dansmith | mriedem: hopefully some locality/affinity in placement | |
| 20:00:41 | jaypipes | mriedem: port pair allocations with SR-IOV. | |
| 20:00:50 | openstackgerrit | Abhishek Sharma M proposed openstack/nova master: Catching OverQuota Exception https://review.openstack.org/460293 | |
| 20:00:50 | openstackgerrit | Abhishek Sharma M proposed openstack/nova master: Catching OverQuota Exception https://review.openstack.org/455118 | |
| 20:00:54 | dansmith | yeah, GPU and SRIOV love in placement | |
| 20:00:57 | jaypipes | dansmith: yeah. ++ on affinity stuff. | |
| 20:01:06 | jaypipes | heh, much love. | |
| 20:04:24 | cburgess | dansmith mriedem Thinking fill shared storage and cinder ephemeral both probably depend on a re-write of the imagebackend interface and creating cleaner seperation and modularization of that area of the code. | |
| 20:05:44 | dansmith | that's at least a cycle of work probably | |
| 20:05:47 | dansmith | unless it's all we do | |
| 20:07:13 | cburgess | dansmith Probably right. Just trying to figure out how we could do shared storage or a cinder backend properly without fixing that dumpster fire induced miasma. | |
| 20:07:35 | dansmith | TIL a new word | |
| 20:07:57 | cburgess | lol | |
| 20:08:03 | cburgess | miasma? | |
| 20:08:37 | dansmith | no "shared" | |
| 20:08:39 | dansmith | yes miasma :P | |
| 20:09:00 | cburgess | I try and educate as well as provide bad ideas, and donuts, to the nova community. | |
| 20:09:13 | dansmith | and we thank you for it | |
| 20:09:13 | cburgess | Its just one of my many redeeming qualities. | |
| 20:10:42 | sean-k-mooney | jaypipes: do you think the user controlled port pair allocations will slip to queens? | |
| 20:11:10 | jaypipes | sean-k-mooney: considering it didn't land for Pike... yes. | |
| 20:11:13 | sean-k-mooney | jaypipes: i have been getting some question lately about doing bonding with sriov for ha and performacne which would need that | |
| 20:11:30 | sean-k-mooney | oh ya we are past spec deadline | |
| 20:15:35 | openstackgerrit | Rafael Folco proposed openstack/nova master: placement: Specific error for inventory in use https://review.openstack.org/460231 | |
| 20:29:41 | mriedem | jaypipes: i don't know what "port pair allocations with SR-IOV" means | |
| 20:30:44 | mriedem | dansmith: isn't gpu/sriov love in placement == nested resource providers == pike? | |
| 20:31:13 | dansmith | mriedem: there's virt driver work that needs to be done that I don't think is going to happen in pike | |
| 20:31:14 | dansmith | certainly for gpu | |
| 20:33:06 | mriedem | anything forward looking for cells v2 in queens? | |
| 20:33:19 | mriedem | besides fix all the bugs we introduce in pike? :) | |
| 20:33:30 | jaypipes | mriedem: https://review.openstack.org/#/c/182242/ | |
| 20:33:34 | mriedem | oh i suppose integration of limits in keystone | |
| 20:33:44 | jaypipes | ++ | |
| 20:33:57 | mriedem | jaypipes: oh right, that spec... | |
| 20:34:04 | dansmith | mriedem: well, yeah, all the bugs, but also presumably we'll have plenty of performance things to work out | |
| 20:34:30 | mriedem | jaypipes: that's the one where the "user" is a single tenant NFV machine right? | |
| 20:35:49 | mriedem | ok here is the list: "VGPU support, full shared storage/network reporting and affinity support in the placement service, volume multi-attach, Cinder as ephemeral backend, integration of limits in Keystone, Cells v2 hardening." | |
| 20:35:57 | mriedem | "+ delete shelve" | |
| 20:36:00 | mriedem | :) | |
| 20:36:11 | dansmith | you have my vote | |
| 20:36:29 | dansmith | HTTP 420: Sorry I dunno where I left that shelve feature.. please try again... uh, never. | |
| 20:36:45 | mriedem | stoner apis | |
| 20:36:46 | mriedem | psh | |
| 20:36:52 | dansmith | heh | |
| 20:37:47 | jaypipes | mriedem: yes. | |
| 20:40:05 | mriedem | (optional) Do you have questions for users that can help inform/guide your development teams? | |
| 20:40:10 | mriedem | oh boy | |
| 20:40:21 | mriedem | i can't remember what i asked about this for pike | |
| 20:41:05 | mriedem | "do you use any microversion > 2.1?" | |
| 20:41:09 | openstackgerrit | Abhishek Sharma M proposed openstack/nova master: Catching OverQuota Exception https://review.openstack.org/460305 | |
| 20:41:09 | mriedem | that could be a legit question | |
| 20:41:13 | dansmith | hah | |
| 20:41:52 | mriedem | b/c we're always coding 18 months ahead at least | |
| 20:41:54 | mriedem | and have no idea | |
| 20:51:57 | mriedem | "While the majority of users aren’t currently using Nova cells, about 146 deployments reported using cells, which is substantially more than the 67 users reporting using Nova cells one year ago. " | |
| 20:53:36 | abhi89 | johnthetubaguy: Hi | |