Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-03-17
16:22:14 mriedem cburgess: oh that's even better b/c we cache the clients
16:22:21 johnthetubaguy yeah, what cburgess said
16:22:22 cdent figleaf: no, you just are trying to apply principles that might work elsewhere, here
16:22:34 cburgess cfriesen Right thats what I'm thinking. If we know the state is "we made the cast to run pre live migration" then we should be able to cancel that by making the right call to abort it.
16:22:36 johnthetubaguy cburgess: I am +1 increasing the timeout for the lots of volumes case
16:22:44 figleaf cdent: like I said, too idealistic
16:23:04 superdan mriedem: no caching of the client anymore right?
16:23:09 superdan well, for compute anyway
16:23:23 cburgess johnthetubaguy Some type of equation you thjink? Before call is made look at size of BDM and make some type of auto-adjust? Or just a new config value for pre-live timeout?
16:23:31 cfriesen mriedem: I think there's an example of the per-call timeout in nova/cells/rpcapi.py
16:23:34 superdan well, sometimes in compute actually
16:23:38 johnthetubaguy cburgess: just a large one I think
16:23:51 mriedem yeah i see the timeout kwarg to the client.prepare method
16:23:53 johnthetubaguy cburgess: its only there as a "oh dear, you must have given up" kind of timeout
16:23:58 johnthetubaguy mriedem: yeah, thats the one
16:24:04 cburgess johnthetubaguy Just an arbitrary "This call can suck, wait 10 minutes" type thing?
16:24:11 johnthetubaguy mriedem: I thought we used that already for something, I can't remember now
16:24:13 superdan it'd be nice if we had a per-call heartbeat mechanism,
16:24:17 superdan which o.msg could provide for us
16:24:27 superdan send a "still working" reply to the reply queue which would reset the timer
16:24:29 cburgess johnthetubaguy The timeout would be a stop gap really. I think mriedem and cfriesen idea about moving to cast is the right solution but probably going to take more effort.
16:24:31 johnthetubaguy superdan: thats the "task" framework in my head
16:24:37 cburgess superdan OMG yes
16:24:40 superdan johnthetubaguy: no, I mean something at the o.msg layer
16:24:47 superdan johnthetubaguy: we could solve it in nova with a task too
16:24:50 johnthetubaguy superdan: yeah, check if job is progressing, and you know the owner
16:24:54 mriedem cctxt = self.client.prepare(timeout=timeout)
16:24:54 mriedem arg_p = jsonutils.to_primitive(arg)
16:24:54 mriedem def ping(self, context, arg, timeout=None):
16:24:57 mriedem we do have something using it
16:25:00 superdan but it'd be nice for lots of things if o.msg would heartbeat a particular call
16:25:11 johnthetubaguy superdan: oh I see what you mean now
16:25:29 cfriesen mriedem: you can set it in the client init but also in the cctxt.call()
16:25:32 superdan johnthetubaguy: you could just send something to the reply queue like "this isn't the reply, but I ain't dead yet"
16:25:50 mriedem cfriesen: i don't actually see the timeout kwarg in the rpc client call() method in oslo.messaging
16:25:50 cburgess mriedem cfriesen johnthetubaguy superdan: So 1 bug and 1 spec here? bug = set a "large" timeout for the pre_live call today, spec = move to casting?
16:25:51 johnthetubaguy superdan: yeah, like a "continue" or whatever
16:25:53 mriedem it's in the prepare() method
16:26:18 superdan johnthetubaguy: or lease renewal
16:26:31 mriedem cfriesen: http://git.openstack.org/cgit/openstack/oslo.messaging/tree/oslo_messaging/rpc/client.py#n178
16:26:45 mriedem cfriesen: call() doesn't use a timeout kwarg http://git.openstack.org/cgit/openstack/oslo.messaging/tree/oslo_messaging/rpc/client.py#n151
16:27:16 cfriesen mriedem: we're using it in nova/cells/rpcapi.py, line 332
16:27:40 johnthetubaguy superdan: yeah, reset the timeout timer
16:27:45 mriedem cfriesen: that looks like a mistake
16:28:06 cburgess mriedem cfriesen Yeah the kwargs just gets pased all the way down and I think that one gets dropped.
16:28:08 mriedem cfriesen: the oslo.messaging rpc client call() method takes any **kwargs
16:28:16 mriedem right, the call() method doesn't use the timeout
16:28:18 mriedem the prepare() method does
16:28:32 mriedem which is nova/cells/rpcapi.py L326
16:28:48 cburgess mriedem Right.
16:29:08 cburgess mriedem I still suspect its doable... we just have to modify things a bit more to make it work.
16:29:18 cfriesen mriedem: I think maybe the call() routine used to take a timeout
16:29:23 mriedem maybe
16:29:27 mriedem anyway yeah this is easy
16:29:48 mriedem https://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L720
16:29:59 cburgess https://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L720
16:29:59 mriedem you use your nova config pre_lie_migration_timeout value there
16:30:01 cburgess LOL yeah
16:30:08 cburgess mriedem Just found that same line.
16:30:09 mriedem default to None to get the default from oslo.messaging
16:30:14 mriedem bump to 10 minutes if you need to
16:30:18 cburgess mriedem Yup
16:30:23 cburgess mriedem So fixed value or config option?
16:30:27 mriedem config
16:30:33 cburgess Where is sdague to yell at me about config options?
16:30:37 cburgess mriedem Bug? or Spec?
16:30:41 mriedem this is completely operator side config
16:30:48 mriedem this is a bug i think
16:31:01 mriedem the spec is changing the blocking call to two casts
16:31:07 cburgess mriedem Right
16:31:27 cburgess mriedem OK... I shall file bug with code since this is easy and allow me to get ATC again. :P
16:31:44 mriedem cburgess: ATC isn't a thing anymore
16:31:45 mriedem remember
16:31:50 mriedem your ATC pass is your PTG $100
16:32:49 openstackgerrit Chris Friesen proposed openstack/nova master: remove unused parameter in rpc call https://review.openstack.org/447055
16:33:07 openstackgerrit Sarafraj Singh proposed openstack/nova-specs master: Spec to add per instance timeout for LM https://review.openstack.org/442809
16:33:17 cburgess mriedem ATC = voting rights still
16:33:39 cburgess mriedem Semi-related... any idea who to contact about my ATC pass? I never got that email with details for Boston.
16:34:14 cburgess mriedem I can't vote in elections for TC, or nova unless I'm ATC. Or unless a PTL goes through the special ATC nomination process to make someone an ATC.
16:34:46 dtp johnthetubaguy and jroll, are you guys around?
16:35:20 raj_singh johnthetubaguy: I tried to address your concerns on https://review.openstack.org/#/c/442809/. Can you please have another look, when you get a chance?
16:35:28 johnthetubaguy dtp: I am around
16:35:39 cfriesen cburgess: I had an issue too...ended up emailing summit@openstack.org
16:35:42 johnthetubaguy raj_singh: cool
16:35:48 cburgess cfriesen Cool thanks.
16:35:59 dtp johnthetubaguy - would have have a look at https://blueprints.launchpad.net/nova/+spec/conf-network-metadata-always-static-ip when you have a few minutes, please?
16:36:07 cburgess cfriesen Weird because I recall them even asking me to confirm I was going to boston and my email when I checked in at PTG but nothing.
16:36:18 mriedem cburgess: i don't remember anything about ATC passes for boston
16:36:23 mriedem true for the elections though
16:36:32 mriedem cburgess: did you land any code changes in ocata?
16:36:45 cfriesen cburgess: whoops, make that summitreg@openstack.org
16:36:49 mriedem cburgess: otherwise there is an ATC nomination process but i think we're past that
16:36:52 cburgess mriedem No I haven't landed actual code in... a while...
16:37:07 cburgess mriedem I don't get to code anything. :(
16:37:26 sdague cburgess: we can totally find you some easy bugs to fix :)
16:37:42 cburgess I last landed code on 2/8/2016
16:37:52 cburgess sdague I just found one. It involves a new config option, you will hate me.
16:37:52 sdague cburgess: nova api-ref is in tree now, I'm sure you can tell us a ton of things we lie about with the API :)
16:38:03 cburgess sdague LOL
16:38:12 mriedem dtp: btw, now that you're around, we should talk about your alternative for https://blueprints.launchpad.net/nova/+spec/service-hyper-pci-uuid-in-api

Earlier   Later