| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-01 | |||
| 17:48:51 | mriedem | maybe we don't hit that code b/c of the same thing? | |
| 17:49:45 | dansmith | well, you log that message, so it seems like you are | |
| 17:49:50 | dansmith | not sure why you get a trace there then | |
| 17:49:53 | mriedem | but where does the traceback come from? | |
| 17:49:54 | mriedem | right | |
| 17:50:28 | dansmith | unless a default has changed and oslo now logs a trace if you're in exception context | |
| 17:51:22 | dansmith | boy that sure is ugly | |
| 17:51:51 | mriedem | been this way since ocata | |
| 17:52:13 | trinaths | hi, I'm hitting this error, when i start nova-compute, http://paste.openstack.org/show/617156/ | |
| 17:52:16 | openstackgerrit | Sumedh Degaonkar proposed openstack/nova master: Adds 'sata' as a valid disk bus for qemu and kvm hypervisors https://review.openstack.org/459794 | |
| 17:52:22 | trinaths | please help me resolve the same. | |
| 17:58:39 | sdague | mriedem: actually, looking at sfinucan's split up of docs, I'm really not convinced that https://review.openstack.org/#/c/478485/ is a good move | |
| 18:00:17 | mriedem | sdague: ... i said it was a massive change | |
| 18:00:17 | mriedem | http://docs-draft.openstack.org/85/478485/16/check/gate-nova-docs-ubuntu-xenial/481415d//doc/build/html/ | |
| 18:00:36 | mriedem | compared to how things used to be https://docs.openstack.org/nova/ocata/ | |
| 18:00:37 | sdague | mriedem: yeh | |
| 18:00:39 | mriedem | i can't even really review it | |
| 18:00:53 | mriedem | asettle is also working on that patch | |
| 18:00:59 | sdague | honestly, I mostly want to light docs like that on fire | |
| 18:01:17 | sdague | because you have to know what you are looking for before you can find what you are looking for | |
| 18:01:44 | mriedem | wait which thing do you want to light on fire? | |
| 18:02:29 | sdague | spliting up the process doc into 6 process docs | |
| 18:02:52 | sdague | http://docs-draft.openstack.org/85/478485/16/check/gate-nova-docs-ubuntu-xenial/481415d//doc/build/html/contributor/process.html | |
| 18:03:14 | mriedem | compared to https://docs.openstack.org/nova/ocata/process.html ? | |
| 18:03:24 | sdague | yes | |
| 18:06:23 | mriedem | idk if this is meant to be consistent with other projects, but it's not consistent with cinder's home page now | |
| 18:06:31 | mriedem | https://docs.openstack.org/cinder/latest/ | |
| 18:06:41 | mriedem | or glance https://docs.openstack.org/glance/latest/ | |
| 18:13:16 | sdague | it would be nice to figure out how we get in page toc into the sidebar instead of top of page | |
| 18:13:34 | mriedem | yes that's also been annoying me | |
| 18:14:28 | sdague | dhellman might know | |
| 18:14:34 | sdague | looks like glance does it | |
| 18:16:56 | mriedem | sdague: any idea how we figure out which log formatter/handler we're using in a devstack run? | |
| 18:17:05 | mriedem | like, are those configurable somewhere? | |
| 18:18:10 | sdague | mriedem: ?? | |
| 18:18:24 | mriedem | well there is a json formatter, and a context formatter | |
| 18:18:33 | sdague | yes, we're using the context formatter | |
| 18:18:38 | mriedem | and a color handler, and journal and syslog handler | |
| 18:18:43 | mriedem | but how would someone know? | |
| 18:18:47 | mriedem | config? | |
| 18:19:27 | sdague | it's config, but I think those are the defaults | |
| 18:20:55 | sdague | oh, you know what it is, it's actually python logging config | |
| 18:20:59 | sdague | outside of our config | |
| 18:21:10 | sdague | so use_syslog / use_journal will tweak those things | |
| 18:21:14 | sdague | https://docs.openstack.org/oslo.log/latest/configuration/index.html | |
| 18:21:27 | sdague | but on the formatter side it's all done as python logger setup | |
| 18:23:14 | mriedem | ok, because what i think we're seeing with a bunch of these random stacktraces related to ComputeHostNotFound_Remote is that we're logging with an exception context, | |
| 18:23:18 | mriedem | and oslo.log is logging the traceback | |
| 18:23:30 | mriedem | automagically | |
| 18:24:20 | mriedem | https://www.youtube.com/watch?v=iExgnVXSAuE | |
| 18:24:24 | mriedem | thanks oslo.log | |
| 18:26:11 | mriedem | gonna push a patch to test that theory | |
| 18:27:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: See if we can squash ComputeHostNotFound_Remote on startup https://review.openstack.org/489683 | |
| 18:27:55 | mriedem | sdague: dansmith: ^ is the canary | |
| 18:29:23 | mriedem | i think it's something to do with this: https://github.com/openstack/oslo.log/blob/3.30.0/oslo_log/formatters.py#L144 | |
| 18:36:45 | openstackgerrit | Merged openstack/nova master: Test resize with placement api https://review.openstack.org/487958 | |
| 18:36:58 | mriedem | yay ^ | |
| 18:37:13 | openstackgerrit | Jackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra https://review.openstack.org/457711 | |
| 18:54:24 | jaypipes | gibi: where do I change the notification sample test JSON files? | |
| 18:54:58 | jaypipes | gibi: nm, found it. | |
| 19:02:15 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: remove existing allocs when set allocs https://review.openstack.org/489273 | |
| 19:02:15 | openstackgerrit | Jay Pipes proposed openstack/nova master: remove provider allocs in confirm/revert resize https://review.openstack.org/488510 | |
| 19:02:16 | openstackgerrit | Jay Pipes proposed openstack/nova master: Additional assertions to resize tests https://review.openstack.org/489714 | |
| 19:05:41 | melwitt | this is a bug fix (data corruption possible) I think we'll need for rc1, if anyone can review please https://review.openstack.org/#/c/488545/ | |
| 19:07:33 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 19:22:38 | openstackgerrit | Eric Fried proposed openstack/nova master: nova.utils.get_endpoint_data() https://review.openstack.org/488137 | |
| 19:23:09 | mriedem | melwitt: question in there | |
| 19:23:16 | mriedem | the persistent vs live stuff always confuses me | |
| 19:25:25 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup unnecessary logic in os-volume_attachments controller code https://review.openstack.org/485823 | |
| 19:25:36 | cfriesen | mriedem: isn't "persisten" whether there is a persistent domain (unrelated to the device)? | |
| 19:26:14 | melwitt | mriedem: I'm not sure I understand the question. in my comment I tried to say "if DeviceNotFound was raised when we passed persistent=True, that means it wasn't found, so we should continue on to try a live detach". though I do notice I should have used "if persistent and live" | |
| 19:26:33 | melwitt | because if live isn't True then there's no need to continue | |
| 19:26:42 | mriedem | well there you go | |
| 19:27:08 | melwitt | is that what you were pointing out? sorry :) | |
| 19:27:15 | mriedem | no | |
| 19:27:39 | mriedem | i just wanted to ask a question to sound like i know 2 sh*ts about this code before +2ing it | |
| 19:27:59 | mriedem | <- professional | |
| 19:28:15 | melwitt | cfriesen: yeah, from what I understand there's effectively two copies of the VM config. one is "persistent" to affect upon next boot and the other is "live" which affects only the live VM | |
| 19:28:19 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 19:28:28 | cdent | jmlowe: I’m going to break the bourbon pattern to give you some early for the good advice of the day award | |
| 19:28:41 | mriedem | melwitt: the comment makes more sense now | |
| 19:29:13 | melwitt | so if the VM even has a persistent config, you have to detach the device from both configs to make sure the device is detached live and also won't show up again after a reboot | |
| 19:29:25 | melwitt | it's confusing | |
| 19:29:48 | mriedem | melwitt: that would be good doc to put in that code | |
| 19:29:57 | melwitt | will do | |
| 19:31:02 | cfriesen | melwitt: so we try first calling detach_device() with both persistant and live but libvirt raises an exception since it doesn't find it in the persistent config? | |
| 19:31:50 | mriedem | so the bug is we tried to detach from persistent config and it wasn't there, so we gave up, but it was in the live config, and then the user reboots the guest and now the volume is attached to both? | |
| 19:32:00 | cfriesen | melwitt: so we have to retry just the live detach without the persistant | |
| 19:32:05 | jmlowe | cdent: what advise was that? "never shit in your hat" "money talks and bullshit walks" "shit in one hand wish in the other and see which one fills up first" | |
| 19:32:19 | melwitt | cfriesen: right. this can happen in the scenario where the guest is busy (e.g. file open) and the guest ignores the ACPI request to detach from live. so what happens there is the detach from the persistent config succeeds but the live fails and so the overall detach fails | |
| 19:32:54 | mriedem | can qemu / libvirt just add a "seriously_please_detach_this_thing_at_some_point" API? | |
| 19:33:02 | jmlowe | cdent: it appears that most of my advise is scatological in nature | |
| 19:33:06 | cdent | jmlowe: ceph your glance and your disk | |
| 19:33:11 | cfriesen | melwitt: I take it libvirt chokes if you specify both VIR_DOMAIN_AFFECT_CONFIG and VIR_DOMAIN_AFFECT_LIVE and it's not in the persistant? | |
| 19:33:12 | melwitt | cfriesen: later on, if the guest is all done and the file is closed, the user wants to detach the volume again, they will issue the detach command and we'll need to detach it from only the live config bc it's already gone from the persistent config | |
| 19:33:16 | cdent | which may be scat | |
| 19:33:20 | melwitt | cfriesen: tes | |
| 19:33:22 | melwitt | *yes | |
| 19:33:56 | melwitt | libvirt will raise a "no device found" type error due to the AFFECT_CONFIG flag | |
| 19:34:21 | cfriesen | melwitt: almost seems safer to just handle persistent and live separately rather than trying to do them both and have to clean up if it fails | |