Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-02
21:17:45 dansmith because those weren't in there a few days ago
21:17:48 mriedem https://github.com/openstack/oslo.log/commit/a5ee482ceeee78cc801b3419b9bc3efae04d0f26
21:17:57 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/461446
21:18:52 mriedem oslo.log 3.26.0 came into upper-constraints on 4/24
21:19:34 openstackgerrit Abhishek Sharma M proposed openstack/nova master: Catching OverQuota Exception https://review.openstack.org/460305
21:21:42 dansmith mriedem: sdague is pinging me through another channel whilst elevating the temperature of dead animal flesh exterior to his home
21:22:03 dansmith mriedem: about the possibility of an interaction between the journal logging calls when we're using eventlet
21:22:07 dansmith like blocking the process
21:22:11 dansmith which is kindof interesting
21:22:27 mriedem ok
21:22:30 mriedem that sounds super fun
21:22:39 dansmith not so much
21:22:52 mriedem also, these giant journal zips produced in the jobs now, how does one read those?
21:23:08 dansmith with journalctl Iimagine
21:23:12 mriedem oh journalctl
21:23:13 mriedem yeah
21:35:23 mriedem dansmith: so you're hitting this race and can't debug it b/c of the systemd thing?
21:35:41 mriedem or you think the systemd thing is causing the problem?
21:36:19 dansmith mriedem: I can't repro it locally, and the gate was hitting it on the base job repeatedly until I started adding debug to chase it dwn
21:36:42 dansmith now only one of the jobs seems to hit it, but yeah, I'm wondering why my messages aren't showing up to figure out how to debug further
21:37:35 melwitt sdague, mriedem, sfinucan: fyi, neither LC_ALL nor PYTHONIOENCODING are helping the issue with the log traces. back to the drawing board
21:39:08 mriedem what else doesn't work?
21:39:14 mriedem right now
21:41:32 mriedem given this is the first day i've seen the sun in about a week i think i'm just going to go outside and grill too
21:43:11 melwitt enjoy the outdoors, vampire
21:49:11 openstackgerrit melanie witt proposed openstack/nova master: WIP Use six.text_type() when logging Instance object https://review.openstack.org/461912
22:00:01 stvnoyes summit
22:00:07 stvnoyes boston
22:00:30 stvnoyes pls ignore^^^ ;-)
22:38:44 openstackgerrit melanie witt proposed openstack/nova master: WIP Use six.text_type() when logging Instance object https://review.openstack.org/461912
23:45:53 mriedem melwitt: i tried the six.text_type thing in a python shell earlier today and that won't solve it, like you said
23:46:14 mriedem at least i don't think so
23:47:05 mriedem melwitt: what i was going to try was doing encodeutils.safe_encode(value, incoming='utf-8') here https://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/fields.py#L269
23:47:29 mriedem because i think that's where the unicodedecodeerror starts
23:48:09 melwitt mriedem: yeah, I just tried the same thing with a WIP patch in the gate and it didn't help. I'm digging into what's going on. the __repr__ function for ovo returns a bytes string for python2
23:49:11 melwitt and I can get the same error by just doing a unicode(instance object) or unicode(str(instance object)). I don't yet understand the connection to the logger
23:50:12 mriedem maybe it's something with the systemd log handler stuff that was in the latest oslo.log that started getting used in upper-constraints yesterday
23:50:13 melwitt it's blowing up during the emit() where it does a self.format(record), whatever format is doing causes it to blow up if there's unicode in the instance string
23:54:02 mriedem melwitt: umm, doesn't this look odd? https://github.com/openstack/oslo.log/blob/master/oslo_log/handlers.py#L95
23:54:11 mriedem note that before the py2 conditional, it formats the record w/o encoding it
23:54:39 melwitt yeah, I was just looking at the same thing
23:55:10 mriedem it also seems the OSJournalHandler should be doing something similar
23:55:11 melwitt but, it's not that handler that I saw blowing up in the gate, it's the OSJournalHandler at L120
#openstack-nova - 2017-05-03
00:00:32 melwitt the instance string repr is a bytes string in python2, that's what the safe_encode does in the __repr__ function in ovo. the instance.display_name is of type unicode. I don't know if that's relevant to what's happening here
00:05:15 mriedem ooo fun i pulled the journal file down from a failed job and i'm poking through the nova-compute logs using journalctl
00:06:59 melwitt super fun
00:07:22 melwitt the OSJournalHandler is new as of about a month ago, fwiw
00:07:33 mriedem yeah
00:08:08 melwitt if the syslog one was the old default, it seems like it should have been making the same traces. I don't know what the old handler was
01:27:54 openstackgerrit melanie witt proposed openstack/nova master: WIP Use six.text_type() when logging Instance object https://review.openstack.org/461912
01:37:07 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Support tag instances when boot(3/4) https://review.openstack.org/459593
02:19:03 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for limits endpoint instead of stevedore https://review.openstack.org/458805
02:19:04 openstackgerrit jichenjc proposed openstack/nova master: use plain routes list for hypervisor endpoint instead of stevedore https://review.openstack.org/459235
02:19:04 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for hosts endpoint instead of stevedore https://review.openstack.org/459231
02:21:40 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for os-instance-usage-audit-log endpoint instead of stevedore https://review.openstack.org/461647
02:34:58 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for os-quota-classes endpoint instead of stevedore https://review.openstack.org/461641
02:36:46 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for os-migrations endpoint instead of stevedore https://review.openstack.org/461628
02:38:26 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for os-quota-sets endpoint instead of stevedore https://review.openstack.org/461630
02:45:26 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Use plain routes list for server-metadata endpoint instead of stevedore https://review.openstack.org/461703
03:19:34 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: Encode ovo String field type values in __repr__ https://review.openstack.org/461975
03:19:38 mriedem melwitt: ^ trying something since we can't test this easily in ovo itself
03:48:34 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Support tag instances when boot(3/4) https://review.openstack.org/459593
03:57:04 openstackgerrit Chris Friesen proposed openstack/python-novaclient master: 2.46: Show flavor info in server details https://review.openstack.org/435141
04:21:36 openstackgerrit falseuser proposed openstack/nova master: [BugFix] Change the parameter of the exception error message https://review.openstack.org/461985
04:24:42 openstackgerrit melanie witt proposed openstack/nova master: WIP Use six.text_type() when logging Instance object https://review.openstack.org/461912
04:26:21 openstackgerrit guang-yee proposed openstack/nova master: libvirt: Reconnect volumes and encryptors during a hard reboot https://review.openstack.org/400384
05:51:11 openstackgerrit Alex Xu proposed openstack/nova master: Add policy description for extensions https://review.openstack.org/461995
05:51:12 openstackgerrit Alex Xu proposed openstack/nova master: Add policy description for os-quota-classes https://review.openstack.org/461996
06:14:41 openstackgerrit Alex Xu proposed openstack/nova master: Adjust the order of server actions in api-ref https://review.openstack.org/461998
06:14:41 openstackgerrit Alex Xu proposed openstack/nova master: Move all the deprecated server actions together into the bottom of doc https://review.openstack.org/458738
06:32:51 openstackgerrit Andrey Volkov proposed openstack/nova master: [placement] Add api-ref for DELETE RP inventory https://review.openstack.org/455215
06:32:52 openstackgerrit Andrey Volkov proposed openstack/nova master: [placement] Add api-ref for GET RP inventory https://review.openstack.org/455213
06:32:52 openstackgerrit Andrey Volkov proposed openstack/nova master: [placement] Add api-ref for PUT RP inventory https://review.openstack.org/455214
06:32:53 openstackgerrit Andrey Volkov proposed openstack/nova master: [placement] Add api-ref for DELETE RP inventories https://review.openstack.org/452708
06:32:53 openstackgerrit Andrey Volkov proposed openstack/nova master: [placement] Add api-ref for GET RP inventories https://review.openstack.org/449765
06:32:54 openstackgerrit Andrey Volkov proposed openstack/nova master: [placement] Add api-ref for PUT RP inventories https://review.openstack.org/452707
06:32:54 openstackgerrit Andrey Volkov proposed openstack/nova master: [WIP] [placement] Add api-ref resource provider https://review.openstack.org/456176
06:50:35 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Send out notifications when instance tags changed https://review.openstack.org/459923
07:04:14 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Support tag instances when boot(3/4) https://review.openstack.org/459593
07:12:10 bauzas p!spy
07:12:14 bauzas good morning Nova
07:23:32 openstackgerrit Huan Xie proposed openstack/nova master: WIP: Fix live migration on XS7.1 https://review.openstack.org/462017
07:25:05 openstackgerrit Alex Xu proposed openstack/nova master: Add policy description for os-quota-classes https://review.openstack.org/461996
07:25:06 openstackgerrit Alex Xu proposed openstack/nova master: Add policy description for extensions https://review.openstack.org/461995
08:39:55 openstackgerrit Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
09:04:44 openstackgerrit jichenjc proposed openstack/nova master: Use plain routes list for os-instance-usage-audit-log endpoint instead of stevedore https://review.openstack.org/461647
09:19:35 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Use plain routes list for server-password endpoint instead of stevedore https://review.openstack.org/462049
09:27:29 openstackgerrit Alex Xu proposed openstack/nova master: Add policy description for os-quota-classes https://review.openstack.org/461996
09:27:30 openstackgerrit Alex Xu proposed openstack/nova master: Add policy description for extensions https://review.openstack.org/461995
09:37:02 johnthetubaguy gmann: alex_xu: just looking at this ips API, seems like we should have deprecated that? https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/ips.py
09:41:55 openstackgerrit Alex Szarka proposed openstack/nova master: Transform instance-live_migration_abort notification https://review.openstack.org/420453
09:42:14 asettle Good morning \o/ it's been a whole week since I've been here
09:42:31 asettle I've got a quick nova doc patch ifs omeone could take a wee lookie
09:42:34 asettle https://review.openstack.org/#/c/462009/2
09:47:19 johnthetubaguy asettle: seems like a good add to me
09:47:44 asettle Thanks johnthetubaguy :) I just did my little edit, and +2
09:48:41 openstackgerrit jichenjc proposed openstack/nova master: [placement] Add doc for resource provider usages https://review.openstack.org/450105
09:52:30 alex_xu johnthetubaguy: I remember we talk about in the before
09:53:34 johnthetubaguy alex_xu: ah, I forgot about that one completely
09:54:54 alex_xu johnthetubaguy: I can't remeber the detail also, try to search the log

Earlier   Later