Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-05-29
09:09:09 lyarwood frickler: morning, looking
09:20:26 lyarwood I'm seeing this logged alot in there, this isn't the issue but still something we should fix
09:20:29 lyarwood 50459 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: Exception ignored in: <function _after_fork at 0x7f5217bc6b80>
09:20:31 lyarwood 50460 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: Traceback (most recent call last):
09:20:33 lyarwood 50461 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork
09:20:35 lyarwood 50462 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: assert len(_active) == 1
09:20:37 lyarwood 50463 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: AssertionError:
09:26:48 frickler lyarwood: didn't coreycb have some patches up for that? saw that at least in a couple of other projects
09:28:42 frickler hmm, that has long been merged in nova https://review.opendev.org/707474 , maybe it needs some more work
09:49:51 frickler coreycb: seems oslo_concurrency/processutils.py would need to get that fix, too
10:01:47 openstackgerrit Liang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch https://review.opendev.org/731388
10:09:35 lyarwood frickler: sorry had a downstream ping I had to jump on
10:09:51 lyarwood frickler: it looks like we've missed the detach here somehow
10:10:00 lyarwood frickler: does this reproduce easily for you?
10:20:17 frickler lyarwood: well I failed to reproduce it locally so far, but on that patch it seems consistent through the last rechecks
10:21:15 frickler I'm still not 100% convinced that this is not related to the oslo_concurrency issue, trying to patch that one now
10:25:14 lyarwood frickler: yeah I don't think it is
10:26:55 lyarwood frickler: it looks like libvirt is raising a new exception when we can't find the device here https://github.com/openstack/nova/blob/323339236517b00ee2495572cfb2540b9e72ce98/nova/virt/libvirt/guest.py#L393-L414
10:27:12 lyarwood frickler: I can see the error in libvirtd but n-cpu just continues trying to detach the device
10:27:50 lyarwood frickler: I'll add some LOG.exception lines to prove it now
10:34:36 openstackgerrit Lee Yarwood proposed openstack/nova master: DNM libvirt: Add exception logging when attempting to detach device https://review.opendev.org/731721
10:43:59 openstackgerrit Liang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch https://review.opendev.org/731388
12:58:42 gmann gibi: thanks, let's wait for bnemec when is best fit time for him. I am ok anytime after 15 UTC
12:58:52 gibi gmann: Ok
13:21:23 openstackgerrit Lee Yarwood proposed openstack/nova master: DNM libvirt: Add exception logging when attempting to detach device https://review.opendev.org/731721
14:09:00 frickler lyarwood: o.k., I managed to trigger some of your logging code, anything in particular you would want to see?
14:10:01 lyarwood frickler: `Unable to find device` or `Found device` would be a useful start
14:12:22 lyarwood frickler: this smells like two issues btw, a latent bug in Nova within our device lookup code for disks and a new issue in libvirt where it isn't raising the exception.
14:19:26 frickler lyarwood: http://paste.openstack.org/show/794150/ , the first one didn't give a tempest failure, the second did
14:39:51 spotz Hey all, this still the right place for placement questions? Might have a docs bug but not sure
15:24:43 openstackgerrit Liang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch https://review.opendev.org/731388
15:41:23 beekneemech gmann: gibi: I've tentatively scheduled the Oslo-specific topics for our first hour and left the second hour (starting 1600) for the Nova cross-project discussions.
15:41:26 beekneemech Does that work for you?
16:30:57 openstackgerrit Lee Yarwood proposed openstack/nova master: WIP compute: Handle MessagingTimeout during reserve_block_device_name https://review.opendev.org/731804
20:43:07 mriedem spotz: #openstack-placement
20:43:48 spotz Thanks mriedem wasn't sure if they'd made a channel or not
#openstack-nova - 2020-05-30
04:09:44 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.PY2 and six.PY3 https://review.opendev.org/726512
04:10:46 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.add_metaclass https://review.opendev.org/726565
04:11:09 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.reraise https://review.opendev.org/726898
04:11:36 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.moves https://review.opendev.org/727224
04:12:16 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757
04:13:29 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777
04:14:07 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.binary_type/integer_types/string_types https://review.opendev.org/728094
04:28:03 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.text_type (1/2) https://review.opendev.org/728109
04:28:48 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.text_type (2/2) https://review.opendev.org/728117
06:55:19 openstackgerrit Merged openstack/nova master: zuul: Make devstack-plugin-ceph-tempest-py3 a voting check job again https://review.opendev.org/730320
14:44:32 born2bake Hi folks... I ve added resume_guests_state_on_host_boot = True to nova.conf in config folder... each nova.conf file has it. However, when I created an instance and go to libvirt container and do virsh dominfo instance-00000001 - Autostart is disabled. Does anybody know why and how to resolve?
#openstack-nova - 2020-05-31
03:24:43 openstackgerrit norman shen proposed openstack/nova master: Check stack trace legnth https://review.opendev.org/732003
03:25:56 openstackgerrit norman shen proposed openstack/nova master: Check stack trace legnth https://review.opendev.org/732003
03:28:17 openstackgerrit norman shen proposed openstack/nova master: Check stack trace legnth https://review.opendev.org/732003
#openstack-nova - 2020-06-01
07:13:12 bauzas gibi: hola
07:13:23 bauzas I forgot that today is an holiday for me :(
07:14:26 bauzas gibi: I'll try to be around for the Oslo/Nova meeting today but in case I'm not there, I don't have any opinions about it
10:58:19 brinzhang bauzas, gibi: where can get the meetting info, such as the meeting tools(zoom or irc), in http://ptg.openstack.org/ptg.html I dont find there is nova slot in, how to get the PTG meeting info ?
11:00:35 gibi bauzas: ack
11:00:43 gibi bauzas: I re-schedule dth holiday :D
11:01:16 gibi brinzhang: everything is here https://etherpad.opendev.org/p/nova-victoria-ptg
11:01:51 brinzhang gibi:ack
11:02:36 gibi brinzhang: and also if you click the slots here http://ptg.openstack.org/ptg.html then it will bring you to the video room
11:03:52 brinzhang ok, thanks gibi
12:01:29 johnthetubaguy gibi: realised I never updated after we went virtual, I got back from paternity leave last week, so I hope to be around
12:14:03 gibi johnthetubaguy: cool.
12:45:50 johnthetubaguy gibi: I am finding my hayfeaver might simulating it a little bit, have you tried rubbing your eyes after eating some crisps? :P
12:52:29 openstackgerrit John Garbutt proposed openstack/nova-specs master: Re-propose Unified Limits in Nova https://review.opendev.org/732173
13:13:33 openstackgerrit John Garbutt proposed openstack/nova master: Add stub unified limits driver https://review.opendev.org/712137
13:13:33 openstackgerrit John Garbutt proposed openstack/nova master: Assert quota related API behavior when noop https://review.opendev.org/712140
13:13:34 openstackgerrit John Garbutt proposed openstack/nova master: Make unified limits APIs return reserved of 0 https://review.opendev.org/712141
13:13:34 openstackgerrit John Garbutt proposed openstack/nova master: Add logic to enforce local api and db limits https://review.opendev.org/712139
13:13:35 openstackgerrit John Garbutt proposed openstack/nova master: Enforce api and db limits https://review.opendev.org/712142
13:13:36 openstackgerrit John Garbutt proposed openstack/nova master: Update quota_class APIs for db and api limits https://review.opendev.org/712143
13:13:36 openstackgerrit John Garbutt proposed openstack/nova master: Update limit APIs https://review.opendev.org/712707
13:13:37 openstackgerrit John Garbutt proposed openstack/nova master: Update quota sets APIs https://review.opendev.org/712749
13:13:37 openstackgerrit John Garbutt proposed openstack/nova master: Tell oslo.limit how to count nova resources https://review.opendev.org/713301
13:13:38 openstackgerrit John Garbutt proposed openstack/nova master: Enforce resource limits using oslo.limit https://review.opendev.org/615180
13:13:38 openstackgerrit John Garbutt proposed openstack/nova master: Add legacy limits and usage to unified limits https://review.opendev.org/713498
13:13:39 openstackgerrit John Garbutt proposed openstack/nova master: Update quota apis with keystone limits and usage https://review.opendev.org/713499
13:13:39 openstackgerrit John Garbutt proposed openstack/nova master: Add reno for unified limits https://review.opendev.org/715271
13:28:29 openstackgerrit Andreas Jaeger proposed openstack/os-traits master: Switch to newer openstackdocstheme and reno versions https://review.opendev.org/732355
13:57:52 gibi johnthetubaguy: :)
14:01:11 gibi I might need some conference type coffee
14:04:51 dansmith gibi: where are the zoom passwords?
14:05:10 gibi dansmith: it was in a mail. all zoom rooms has the same PTG2020
14:05:27 dansmith gibi: shame accepted, thanks
14:06:14 gibi no shaming intended
14:06:37 dansmith shame self-applied anyway
14:06:43 gibi :)
14:07:26 gibi jitsi works pretty OK so far (I'm in the opendev room)
14:44:13 gibi artom, bauzas: FYI, today at 16:00 UTC in the sdk-cli room there will be discussion about making OSC as the primary client instead of project specific CLI clients (https://etherpad.opendev.org/p/sdk-victoriaptg-planning ). Unfortunately it will happen in paralell with the oslo discussion.
14:44:37 artom gibi, thanks for the heads up
14:47:48 artom France is on holiday (as usual >;) but I'll be there
14:48:50 gibi artom: thanks, I will be in oslo at that time
14:52:14 jsuchome dansmith: hey, any chance for review of that direct-rbd download patch?
14:52:26 dansmith jsuchome: yeah dude, I haven't forgotten about it I promise
14:52:52 dansmith jsuchome: I've got my own patches up without review too.. and PTG this week :)
15:00:13 jsuchome dansmith: oh, there's PTG right now, sorry, I didn;t realize
15:02:21 dansmith gibi: I'm just getting a spinning "connecting" dialog trying to join the oslo room
15:02:26 dansmith gibi: are you connected?
15:02:36 gibi dansmith: I'm in
15:02:40 dansmith argh
15:02:49 dansmith I've used zoom before no problem, but it made me upgrade this morning

Earlier   Later