Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-09-29
19:19:47 mriedem single node devstack is probably not going to work there
19:19:52 mriedem you'd have to fake out like 1000 computes or something
19:19:56 mriedem something that placement has to chew on
19:20:13 superdan nah, just shut down scheduler so we block on making that decision and then try to list before the timeout
19:20:17 superdan but still, not great
19:21:33 mriedem true, set the rpc timeout to 10 minutes :)
19:22:22 superdan the problem is, because we don't have real threads, we're already pegging the cpu just processing the results from the DBs as it is, so, not likely the even smaller overlap of operations is really going to help
19:22:42 melwitt how can we get real threads
19:23:06 superdan I'm not sure eventlet is threadsafe, so I'm not sure we can without bigger changes
19:23:39 superdan that was the problem with us using real threads for db workers ages ago, and it was never fixed, AFAIK, we just worked around it by moving to an all-python db driver
19:24:34 superdan in aggregate it's not as big of a deal because multiple requests will be running at once in our workers, so more overall work gets done with more cores, just not in a single-request sort of environment
19:25:00 superdan any environment that only has one concurrent request ever is either (a) dan's test box or (b) probably not worried about listing thousands of instances at a time :)
19:25:15 melwitt heh
19:29:06 mriedem superdan: some comments/questions in https://review.openstack.org/#/c/498948/
19:29:27 openstackgerrit priyaduggirala proposed openstack/nova master: Rename parameters in call() of nova/image/glance.py https://review.openstack.org/508533
19:29:33 mriedem not trying to block, just want to make sure i know what's going on with this stuff before it merges and i'm lost later
19:31:57 superdan mriedem: I thought we didn't do _LI( but we don't do _( at all anymore?
19:32:13 mriedem we don't translate log messages at all anymore
19:32:21 superdan really thought I was getting pep8 fails
19:32:31 superdan christ, I can never keep it straight
19:33:53 mriedem https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html
19:34:02 mriedem i think that top paragraph is the new guideline
19:34:24 mriedem and https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#log-translation
19:37:17 superdan I believe you, I just can't keep track of it
19:37:51 mriedem i had to look it up too, wasn't sure about _() when you asked but was pretty sure
19:38:40 superdan I figure if I just pick some behavior I'll be right 20% of the time when we've circled back to that as the preferred one
19:39:10 mriedem depends on the current ibm corporate wide software guidelines at the time
19:39:43 superdan mriedem: so, this isn't in the gate yet so do you want me to fix the bottom patch or tack on to the end? this set is pretty fragile so if I do the bottom it'll likely percolate awesomeness up the stack pretty good
19:39:53 superdan and by "awesomeness" I mean "my tears"
19:40:16 mriedem was there anything major? the translation markers and simple logging stuff, plus docstring or whatever can all be done at the end
19:40:23 mriedem there was the one conditional block that i thought was dead code
19:40:43 superdan the else?
19:40:48 superdan the comment was just incorrect
19:40:52 mriedem oh
19:41:13 mriedem the bottom 2 are approved so i'd say just take a fixup change at the end of the series
19:41:38 superdan I really should just remove "on the source" from that one since it'll be used any time we need to revert the allocation regardless of why/where
19:42:13 mriedem the thing i wanted to be cautious of was overusing it like remove_provider_from_instance_allocation in the scheduler report client,
19:42:23 mriedem because remove_provider_from_instance_allocation was written really for cold migrate / resize + resize to same host,
19:42:41 mriedem but we've used it for other move operations, and there are some assumptions in there which don't always work for other move operatoins
19:42:49 mriedem i.e.
19:42:50 mriedem # allocation with our part subtracted
19:42:50 mriedem # are the only provider then we need to merge back in the doubled
19:42:50 mriedem # NOTE(danms): We are in a resize to same host scenario. Since we
19:43:05 superdan well, this really should apply to all of the types once we get it done because same/different, cold/live, they're all the same process since we don't have to worry about clashes since the migration uuid holds things
19:43:16 mriedem if so,
19:43:30 mriedem then heed my warning about assuming migration.source_node is set
19:43:32 mriedem HEED IT
19:43:44 mriedem because we don't even attempt to set that shit for live migration
19:43:45 superdan yeah, I saw, I need to go look
19:43:46 mriedem since there is no claim
19:43:56 superdan because I thought it was set there too
19:44:05 mriedem _live_migrate in conductor task manager
19:44:16 mriedem we set the compute attributes, but not the node ones
19:44:23 mriedem was just looking at using those yesterday for something else
19:44:51 mriedem if those do get set, it would happen later in the compute probably
19:44:58 superdan oh the hosts I see
19:46:32 superdan well, I expect that just means we don't have coverage for live migrations failing in a way that will result in this getting called,
19:46:44 superdan and/or I wasn't consistent in that patch at the top,
19:46:53 superdan but I haven't really gone over that one in as much detail
19:46:59 penick pino: There's a couple ways to do this, one is to completely generate a ssh host key and sign it, then stuff it into the instance through vendordata. Another is.. more complex.
19:47:01 superdan keeping these two sets straight has been hard
19:47:08 mriedem superdan: i bet
19:47:13 mriedem don't forget about evacuate and unshelve
19:47:46 mriedem oh yeah, plus the migration object has dest_compute, dest_host, and dest_node, where one of those is the hostname and one is the host IP
19:47:47 mriedem yay!
19:48:10 penick pino: the more complex version means running a whole AuthNG infrastructure to handle trusted boot of an instance and giving it one-time credentials which allow it to bootstrap on boot and get its key signed. Heavy to implement, useful long term for secure infrastructure. Probably way heavier than you need
19:48:32 superdan mriedem: not sure unshelve is a thing here.. once you're offloaded we shouldn't keep an allocation for you
19:49:04 mriedem yeah
19:49:04 mriedem oh right
19:49:29 superdan and for evacuate, we don't need to keep a claim on the old host, so ... also not sure that's a thing
19:49:52 superdan I originally had them on my radar, but thinking more I'm not sure that's the right thing to do
20:02:32 dims mriedem : if i can get to the browser based vnc for a nova vm ... is there a way to attach a real vncviewer instead of the browser? (is there enough information in the browser url?)
20:03:14 superdan dims: you need a websocket client proxy
20:03:29 superdan dims: the browser client isn't connecting to the vnc server, but getting the stream over a websocket
20:04:31 dims superdan : i see, i have to find a vnc client that supports that ..
20:04:38 dims thanks superdan. will look
20:04:39 superdan dims: it's called novnc :P
20:04:54 dims which is browser based :)
20:04:58 superdan right
20:05:13 superdan I expect you won't find another, but .. good luck :)
20:05:21 superdan you could build a ws proxy that listens on a tcp port
20:05:31 superdan client->proxy->proxy->server
20:07:04 dims hmm, i might just stick to the browser :)
20:07:13 superdan good plan :)
20:12:48 mriedem dims: are you trying to circumvent zones
20:13:28 dims mriedem : not really, i can use the browser on my mac already, its just not very convenient for long use
20:14:01 mriedem s/*/i can use my mac already, it's just not very convenient/
20:15:09 dims mriedem : bad regex? :)
20:15:15 mriedem yeah maybe
20:16:40 superdan dims: vnc consoles are for disaster recovery and windows boxes, neither of which should be prolonged use
20:17:22 dims right superdan. there's just no other way to get to these vm(s) currently (multi-level protected zones)
20:17:39 superdan ouch
20:17:47 dims y sigh
20:21:38 openstackgerrit Matt Riedemann proposed openstack/nova master: Refactor duplicate code for looking up the compute node name https://review.openstack.org/508604
20:21:38 openstackgerrit Matt Riedemann proposed openstack/nova master: Add hints to what the Migration attribute values are https://review.openstack.org/508603
20:25:35 cfriesen not strictly a nova question, but hoping someone knows. If I monkey_patch the world with eventlet, will threading.Thread() actually give me a real thread or will it give me a greenthread?
20:27:01 superdan a greenthread
20:29:46 cfriesen thanks. I was expecting a real thread and couldn't figure out why I didn't see it on the system.
20:32:01 cfriesen do Condition and RLock and friends all work properly with the greenthread?
20:32:21 superdan yes

Earlier   Later