| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-01 | |||
| 17:45:48 | dansmith | mriedem: that line you quote is intentional log.warning in the libvirt driver | |
| 17:46:02 | dansmith | mriedem: and it looks like we're getting a stack trace automagically from the logging layer (?) | |
| 17:46:05 | mriedem | i know, but the trace is coming from oslo.messaging isn't it? | |
| 17:46:11 | dansmith | no I don't think so | |
| 17:47:06 | dansmith | mriedem: see the line below.. you need exc_info=False maybe ? | |
| 17:47:09 | dansmith | maybe a default changed somehow? | |
| 17:47:12 | dansmith | oh | |
| 17:47:14 | dansmith | ooooh | |
| 17:47:17 | dansmith | you're hitting the lower one | |
| 17:47:19 | dansmith | with exc_info=True | |
| 17:47:25 | dansmith | because it is _Remote | |
| 17:47:52 | mriedem | dansmith: yeah there is that, | |
| 17:47:54 | dansmith | so maybe in that handler, check the exception name and if it looks remoteish then you can not pass the exc_info flag? | |
| 17:47:57 | mriedem | but that doesn't explain this one http://logs.openstack.org/54/487954/5/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/4412f5d/logs/screen-n-cpu.txt.gz?level=TRACE#_Jul_28_19_41_40_927881 | |
| 17:47:59 | dansmith | it's a little icky | |
| 17:48:33 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L6624 | |
| 17:48:40 | mriedem | oh well, | |
| 17:48:51 | mriedem | maybe we don't hit that code b/c of the same thing? | |
| 17:49:45 | dansmith | well, you log that message, so it seems like you are | |
| 17:49:50 | dansmith | not sure why you get a trace there then | |
| 17:49:53 | mriedem | but where does the traceback come from? | |
| 17:49:54 | mriedem | right | |
| 17:50:28 | dansmith | unless a default has changed and oslo now logs a trace if you're in exception context | |
| 17:51:22 | dansmith | boy that sure is ugly | |
| 17:51:51 | mriedem | been this way since ocata | |
| 17:52:13 | trinaths | hi, I'm hitting this error, when i start nova-compute, http://paste.openstack.org/show/617156/ | |
| 17:52:16 | openstackgerrit | Sumedh Degaonkar proposed openstack/nova master: Adds 'sata' as a valid disk bus for qemu and kvm hypervisors https://review.openstack.org/459794 | |
| 17:52:22 | trinaths | please help me resolve the same. | |
| 17:58:39 | sdague | mriedem: actually, looking at sfinucan's split up of docs, I'm really not convinced that https://review.openstack.org/#/c/478485/ is a good move | |
| 18:00:17 | mriedem | http://docs-draft.openstack.org/85/478485/16/check/gate-nova-docs-ubuntu-xenial/481415d//doc/build/html/ | |
| 18:00:17 | mriedem | sdague: ... i said it was a massive change | |
| 18:00:36 | mriedem | compared to how things used to be https://docs.openstack.org/nova/ocata/ | |
| 18:00:37 | sdague | mriedem: yeh | |
| 18:00:39 | mriedem | i can't even really review it | |
| 18:00:53 | mriedem | asettle is also working on that patch | |
| 18:00:59 | sdague | honestly, I mostly want to light docs like that on fire | |
| 18:01:17 | sdague | because you have to know what you are looking for before you can find what you are looking for | |
| 18:01:44 | mriedem | wait which thing do you want to light on fire? | |
| 18:02:29 | sdague | spliting up the process doc into 6 process docs | |
| 18:02:52 | sdague | http://docs-draft.openstack.org/85/478485/16/check/gate-nova-docs-ubuntu-xenial/481415d//doc/build/html/contributor/process.html | |
| 18:03:14 | mriedem | compared to https://docs.openstack.org/nova/ocata/process.html ? | |
| 18:03:24 | sdague | yes | |
| 18:06:23 | mriedem | idk if this is meant to be consistent with other projects, but it's not consistent with cinder's home page now | |
| 18:06:31 | mriedem | https://docs.openstack.org/cinder/latest/ | |
| 18:06:41 | mriedem | or glance https://docs.openstack.org/glance/latest/ | |
| 18:13:16 | sdague | it would be nice to figure out how we get in page toc into the sidebar instead of top of page | |
| 18:13:34 | mriedem | yes that's also been annoying me | |
| 18:14:28 | sdague | dhellman might know | |
| 18:14:34 | sdague | looks like glance does it | |
| 18:16:56 | mriedem | sdague: any idea how we figure out which log formatter/handler we're using in a devstack run? | |
| 18:17:05 | mriedem | like, are those configurable somewhere? | |
| 18:18:10 | sdague | mriedem: ?? | |
| 18:18:24 | mriedem | well there is a json formatter, and a context formatter | |
| 18:18:33 | sdague | yes, we're using the context formatter | |
| 18:18:38 | mriedem | and a color handler, and journal and syslog handler | |
| 18:18:43 | mriedem | but how would someone know? | |
| 18:18:47 | mriedem | config? | |
| 18:19:27 | sdague | it's config, but I think those are the defaults | |
| 18:20:55 | sdague | oh, you know what it is, it's actually python logging config | |
| 18:20:59 | sdague | outside of our config | |
| 18:21:10 | sdague | so use_syslog / use_journal will tweak those things | |
| 18:21:14 | sdague | https://docs.openstack.org/oslo.log/latest/configuration/index.html | |
| 18:21:27 | sdague | but on the formatter side it's all done as python logger setup | |
| 18:23:14 | mriedem | ok, because what i think we're seeing with a bunch of these random stacktraces related to ComputeHostNotFound_Remote is that we're logging with an exception context, | |
| 18:23:18 | mriedem | and oslo.log is logging the traceback | |
| 18:23:30 | mriedem | automagically | |
| 18:24:20 | mriedem | https://www.youtube.com/watch?v=iExgnVXSAuE | |
| 18:24:24 | mriedem | thanks oslo.log | |
| 18:26:11 | mriedem | gonna push a patch to test that theory | |
| 18:27:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: See if we can squash ComputeHostNotFound_Remote on startup https://review.openstack.org/489683 | |
| 18:27:55 | mriedem | sdague: dansmith: ^ is the canary | |
| 18:29:23 | mriedem | i think it's something to do with this: https://github.com/openstack/oslo.log/blob/3.30.0/oslo_log/formatters.py#L144 | |
| 18:36:45 | openstackgerrit | Merged openstack/nova master: Test resize with placement api https://review.openstack.org/487958 | |
| 18:36:58 | mriedem | yay ^ | |
| 18:37:13 | openstackgerrit | Jackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra https://review.openstack.org/457711 | |
| 18:54:24 | jaypipes | gibi: where do I change the notification sample test JSON files? | |
| 18:54:58 | jaypipes | gibi: nm, found it. | |
| 19:02:15 | openstackgerrit | Jay Pipes proposed openstack/nova master: remove provider allocs in confirm/revert resize https://review.openstack.org/488510 | |
| 19:02:15 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: remove existing allocs when set allocs https://review.openstack.org/489273 | |
| 19:02:16 | openstackgerrit | Jay Pipes proposed openstack/nova master: Additional assertions to resize tests https://review.openstack.org/489714 | |
| 19:05:41 | melwitt | this is a bug fix (data corruption possible) I think we'll need for rc1, if anyone can review please https://review.openstack.org/#/c/488545/ | |
| 19:07:33 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 19:22:38 | openstackgerrit | Eric Fried proposed openstack/nova master: nova.utils.get_endpoint_data() https://review.openstack.org/488137 | |
| 19:23:09 | mriedem | melwitt: question in there | |
| 19:23:16 | mriedem | the persistent vs live stuff always confuses me | |
| 19:25:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup unnecessary logic in os-volume_attachments controller code https://review.openstack.org/485823 | |
| 19:25:36 | cfriesen | mriedem: isn't "persisten" whether there is a persistent domain (unrelated to the device)? | |
| 19:26:14 | melwitt | mriedem: I'm not sure I understand the question. in my comment I tried to say "if DeviceNotFound was raised when we passed persistent=True, that means it wasn't found, so we should continue on to try a live detach". though I do notice I should have used "if persistent and live" | |
| 19:26:33 | melwitt | because if live isn't True then there's no need to continue | |
| 19:26:42 | mriedem | well there you go | |
| 19:27:08 | melwitt | is that what you were pointing out? sorry :) | |
| 19:27:15 | mriedem | no | |
| 19:27:39 | mriedem | i just wanted to ask a question to sound like i know 2 sh*ts about this code before +2ing it | |
| 19:27:59 | mriedem | <- professional | |
| 19:28:15 | melwitt | cfriesen: yeah, from what I understand there's effectively two copies of the VM config. one is "persistent" to affect upon next boot and the other is "live" which affects only the live VM | |
| 19:28:19 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 19:28:28 | cdent | jmlowe: I’m going to break the bourbon pattern to give you some early for the good advice of the day award | |
| 19:28:41 | mriedem | melwitt: the comment makes more sense now | |
| 19:29:13 | melwitt | so if the VM even has a persistent config, you have to detach the device from both configs to make sure the device is detached live and also won't show up again after a reboot | |
| 19:29:25 | melwitt | it's confusing | |