Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-31
15:46:08 mriedem idk
15:46:10 sean-k-mooney *binding_failed
15:46:19 mriedem kind of throwing things at the wall at that point trying to see what works
15:46:23 mriedem w/o a real recreate i can tinker with
15:46:46 sean-k-mooney i guess we need to create a regression test for it to figure it out
15:46:53 sean-k-mooney or get logs
15:47:11 mriedem we can't really test this very easily...
15:47:14 mriedem tempest won't do this
15:47:25 mriedem and an in-tree regressoin test would just stub out neutron
15:48:00 mriedem this is why for now i just did handle the error like the original bug for the exact same thing
15:48:10 mriedem https://review.openstack.org/#/c/129158/
15:48:18 mriedem exact same issue, just pre-os-vif
15:49:52 sean-k-mooney well catching the error and preventing the agent from crashing is proably correct in any case.
15:56:11 cdent is there a tag for nova-api
15:56:49 cdent or does one just say "api"?
15:57:55 sean-k-mooney cdent: i had assumed [openstack-dev][nova][api] ...
15:58:16 sean-k-mooney im probably wrong however
15:58:24 mriedem i think he's talking about bug tags
15:58:37 sean-k-mooney oh ok
15:58:41 cdent sean-k-mooney: yeah, sorry, mriedem is right, I wasn't context switching well
15:58:50 cdent too many contexts at once
15:59:07 sean-k-mooney cdent: then yes its just api https://wiki.openstack.org/wiki/Nova/BugTriage
15:59:15 cdent thanks
16:01:22 openstackgerrit Jay Pipes proposed openstack/nova master: track down why inventory placement calls multiply https://review.openstack.org/587510
16:02:55 openstackgerrit Jay Pipes proposed openstack/nova master: track down why inventory placement calls multiply https://review.openstack.org/587510
16:03:59 tbachman !pspy
16:04:00 openstack tbachman: Error: "pspy" is not a valid command.
16:05:03 tbachman !pyspy
16:05:05 openstack tbachman: Error: "pyspy" is not a valid command.
16:05:54 dansmith tbachman: `less ~/.irclogs/...`
16:06:14 tbachman dansmith: thx!
16:06:22 dansmith :P
16:09:26 tbachman there is a command tho — can’t remember what it is/was
16:10:08 tbachman !?
16:10:09 openstack tbachman: Error: "?" is not a valid command.
16:10:11 tbachman heh
16:10:19 tbachman !help
16:10:20 openstack tbachman: (help [<plugin>] [<command>]) -- This command gives a useful description of what <command> does. <plugin> is only necessary if the command is in more than one plugin.
16:12:16 sean-k-mooney tbachman: generally i jsut install highmon in my client that said weechat highlights mentions by default even without it now
16:12:42 tbachman sean-k-mooney: thx!
16:15:59 jroll tbachman: https://anticdent.org/purple-irc-bot.html
16:16:20 tbachman p!spy
16:16:24 tbachman jroll: thx!
16:16:32 tbachman that was the one I was looking for :)
16:16:36 jroll :)
16:16:49 cdent dansmith, mriedem: rado filed a bug on that threading/eventlet thing: https://bugs.launchpad.net/nova/+bug/1784666
16:16:49 openstack Launchpad bug 1784666 in OpenStack Compute (nova) "The thread local which stores the request context is not green" [Medium,New]
16:16:50 melwitt cdent, mriedem: I have a newer change I'm experimenting with for the request_id's jumping across requests. some have reported that it's not only restricted to periodics and that it's happening in general with parallel requests. the patch I'm testing is https://review.openstack.org/582332 but it's not clear to me how to verify whether it fully solves the problem
16:17:17 cdent wow, major jinx with melwitt there
16:17:42 dansmith melwitt: it happens in compute, which never uses target_cell, so I wouldn't think that is related
16:18:02 cdent melwitt: rado is pretty sure it is "simply" an import ordering problem
16:18:22 cdent he can fix it by changing things around
16:18:23 dansmith like the comment on that patch says
16:18:24 melwitt oh, yeah okay. that makes more sense
16:19:03 melwitt ah, I missed that comment
16:19:28 mriedem dansmith: melwitt: cdent: yeah says the same as radu's bug
16:19:39 mriedem "In his patch, eventlet.monkey_patch() must before import oslo_log (actually before import oslo_context)."
16:19:43 dansmith yeah
16:22:28 melwitt unrelated to that, I'd think the cell targeting would cause request_id changes in logging in nova-api though. but I'm not sure I've observed that before. I've seen only the ones in compute
16:23:26 dansmith melwitt: but any targeting of a context should result in the same request id right?
16:23:45 dansmith so even if you did overwrite the logging context with the targeted one, it shouldn't matter I would think
16:24:23 melwitt dansmith: yeah, maybe that's why I haven't observed it
16:24:27 dansmith yeah :)
16:24:46 melwitt mriedem: do we want to dupe one of these to the other bug? https://bugs.launchpad.net/nova/+bug/1773102
16:24:46 openstack Launchpad bug 1773102 in OpenStack Compute (nova) "Abnormal request id in logs" [Medium,In progress] - Assigned to melanie witt (melwitt)
16:28:49 mriedem melwitt: yes done
16:28:55 melwitt thanks
16:30:03 mriedem cdent: i assume radu is pushing a patch?
16:30:36 cdent rado. checking
16:30:58 cdent ealier today he said he was
16:32:49 cdent mriedem: yeah, tomorrow
16:33:04 cdent (it's getting late where he is)
16:41:33 melwitt dansmith: I'm trying to determine if I could salvage this small part of the patch for the scheduler host manager, those could change logged request_id, I would think. do you disagree? https://review.openstack.org/#/c/582332/4/nova/scheduler/host_manager.py
16:42:29 dansmith melwitt: why do you want to use the current request's req-id for that?
16:42:34 dansmith or, meaningm,
16:42:50 dansmith why does it matter that we log the current request as the one that triggered the cache update?
16:42:55 melwitt want to avoid overwriting thread local context bc that's what's used in logging
16:43:41 dansmith that's the async_init_instance_info() part yeah?
16:44:02 melwitt creating the new RequestContext without overwrite=False will replace the thread local context with one that has a freshly generated request_id, so log messages that occur after it was overwritten could have the random request_id
16:44:35 melwitt yeah _async_init_instance_info and refresh_cells_caches
16:44:50 dansmith right, but, why not just create those with overwrite=False
16:45:06 melwitt dansmith: that's what my change does, uses a helper method that does that
16:45:41 melwitt it's the equivalent of get_admin_context except not admin
16:46:17 dansmith oh, okay I see, I thought that was getting it from the TLS for some reason
16:47:08 dansmith why is that not an admin context?
16:47:29 melwitt because the original wasn't? you think it should be admin?
16:48:06 dansmith yeah, I know, it wasn't.. but the reason I didn't match what that get_context() was, is because I normally think of get_admin_context() as "I need an internal context to do a DB thing"
16:48:15 dansmith like,
16:48:30 dansmith it's a pattern I recognize, but don't recognize the get_context() bit even though they're basically the same
16:48:42 melwitt oh, I see. yeah, it could be, but I wasn't thinking to change it from what it was before because I don't know if it was intentional
16:48:50 dansmith anyway, yes probably worth keeping that bit
16:49:01 dansmith yeah, I'm surprised it doesn't need to be admin because it's listing all instances right?
16:49:16 dansmith maybe it works because it has no project_id?
16:49:20 dansmith anyway, seems weird
16:49:40 dansmith those are clearly admin-y internal cache-fill ops, so I would think.. admin.
16:49:45 melwitt yeah, that's a good point. it would miss instances where project_id doesn't match, so maybe it misses instances
16:50:08 dansmith well, it's setting project_id=None, so presumably it's working, but weird
16:50:19 melwitt admin doesn't have to match project_id for instance get_all but non-admin does. I have learned that the hard way from func test writing
16:50:28 dansmith also, would it not make sense to make oslo_context not overwrite context in TLS by default if one is already set?
16:50:52 melwitt I dunno, tbh

Earlier   Later