Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-06-13
09:41:12 lyarwood https://review.openstack.org/#/q/project:openstack/nova+topic:bug/1764883 - looking for additional reviews on this bugfix for an evacuation race if anyone has time this morning.
10:01:18 openstackgerrit Vishakha Agarwal proposed openstack/python-novaclient master: No requirement of –all-tenants while listing servers https://review.openstack.org/569090
10:08:05 openstackgerrit jiang wei proposed openstack/nova master: Add action initiator attribute to the payload https://review.openstack.org/536243
10:43:03 openstackgerrit Andrey Volkov proposed openstack/nova master: Update nova network info when doing rebuild for evacuate operation https://review.openstack.org/382853
10:52:16 openstackgerrit Balazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler https://review.openstack.org/567268
10:52:17 openstackgerrit Balazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding https://review.openstack.org/569459
10:52:18 openstackgerrit Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317
12:05:35 openstackgerrit Balazs Gibizer proposed openstack/nova master: Remove network info stubbing in functional test https://review.openstack.org/570018
12:05:36 openstackgerrit Balazs Gibizer proposed openstack/nova master: Add request_spec.RequestGroup versioned object https://review.openstack.org/568840
12:05:37 openstackgerrit Balazs Gibizer proposed openstack/nova master: Add requested_resources field to RequestSpec https://review.openstack.org/567267
12:05:38 openstackgerrit Balazs Gibizer proposed openstack/nova master: Add bandwidth related standard resource classes https://review.openstack.org/570847
12:05:39 openstackgerrit Balazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler https://review.openstack.org/567268
12:05:40 openstackgerrit Balazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding https://review.openstack.org/569459
12:05:41 openstackgerrit Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317
12:27:41 efried naichuans: Done.
12:43:45 naichuans efried: Thank you very much, Eric
12:59:05 openstackgerrit jiang wei proposed openstack/nova master: Add action initiator attribute to the payload https://review.openstack.org/536243
13:04:35 openstackgerrit Eric Fried proposed openstack/nova master: Validate transport_url in nova-manage cell_v2 commands https://review.openstack.org/569858
13:21:24 openstackgerrit Eric Fried proposed openstack/os-traits master: normalize_name helper https://review.openstack.org/560107
13:28:07 openstackgerrit Dan Smith proposed openstack/nova stable/queens: [Stable Only] Initialise failed_builds in IronicNodeState https://review.openstack.org/574997
13:28:55 openstackgerrit Dan Smith proposed openstack/nova stable/queens: [Stable Only] Initialise failed_builds in IronicNodeState https://review.openstack.org/574997
13:36:14 dansmith mriedem: noice (re: live migration heartbeat test)
13:46:39 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Add status field to version document at / https://review.openstack.org/575117
13:46:44 mriedem stephenfin: if you agree with this, i'll rev that change quick and just fast approve it https://review.openstack.org/#/c/571375/4/api-ref/source/servers-admin-action.inc@70
13:47:32 stephenfin mriedem: Fine by me
13:49:03 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix bug to api-ref https://review.openstack.org/571375
13:49:06 mriedem thanks
13:51:17 SpamapS I'm sitting here watching my nova-compute process wait for input on stdin and I'm wondering what I've done wrong here...
13:52:10 mriedem nova-compute is now a vending machine program from cs 201
13:52:15 dansmith sudo?
13:52:15 SpamapS http://paste.openstack.org/show/723383/
13:52:22 SpamapS That's all it logs.
13:52:46 SpamapS there's no sudo running no.. the nova-compute process itself is looping on select(0,....)
13:53:38 SpamapS http://paste.openstack.org/show/723384/
13:53:39 SpamapS stracing it
13:55:21 dansmith get gmr and see where it's sitting
13:55:25 dansmith I've never seen that before, fwiw
13:55:30 SpamapS doesn't log anything
13:55:36 dansmith gmr doesn't?
13:55:43 SpamapS no.. sent it USR1 and USR2
13:56:12 dansmith wow, must be stuck super early
13:56:59 SpamapS http://paste.openstack.org/show/723385/
13:57:08 SpamapS it actually did log that when I sent it USR1 just now
13:57:10 SpamapS just that
13:57:30 SpamapS I wonder if I send it a SIGINT if it will at least give me a keyboard interrupt stack trace
13:59:19 SpamapS http://paste.openstack.org/show/723386/ ooo... SIGINT did actually make a clue
14:00:46 SpamapS but .. .there's no amqp sockets open and no traffic trying so..... still so confused
14:01:18 dansmith SpamapS: is fd 0 really stdin for that? maybe stdin is closed and 0 was used for a socket or something?
14:02:05 dansmith ls -l /proc/$pid/fd/0
14:02:39 SpamapS nova-comp 7 nova 0u CHR 136,2 0t0 5 /dev/pts/2
14:02:51 dansmith bizarre
14:02:54 SpamapS yeah
14:03:11 SpamapS I wonder if it's some bug in kolla-ansible
14:03:13 SpamapS made bad containers
14:04:21 SpamapS ohh! gmr is actually spraying, to stdout
14:04:23 SpamapS which is the docker logs
14:05:33 dansmith it goes to stderr, which is probably the same for the container
14:05:39 dansmith you were looking in the logs?
14:06:21 SpamapS http://paste.openstack.org/show/723389/
14:06:32 SpamapS Yeah I thought the log I was looking at was stderr
14:06:49 SpamapS there's a greenthread in there with no traceback
14:07:04 SpamapS but I think what's going on is, rmq tries, and fails, and is sleeping a bit?
14:07:24 SpamapS and that select on 0 is actually just eventlet's "got nothin to do" loop
14:08:00 dansmith makes sense
14:08:20 SpamapS yep that's it
14:08:24 SpamapS it backs off quick
14:08:34 SpamapS my tcpdump wasn't running before I restarted nova-compute
14:08:52 SpamapS figured it'd still be doing stuff immediately after startup...
14:09:19 mriedem odyssey4me: osa doesn't run the nova-status upgrade check during upgrades huh?
14:09:32 mriedem odyssey4me: are you aware of any existing patches for that? if not, i'll see if i can hack something in
14:10:45 SpamapS dansmith: thanks for the hints
14:10:59 dansmith I didn't do anything, but sure
14:13:24 openstackgerrit Dan Smith proposed openstack/nova master: Implement file backed memory for instances in libvirt https://review.openstack.org/567876
14:19:52 jaypipes mnaser: hyper scale.
14:21:16 openstackgerrit Eric Fried proposed openstack/nova master: Enforce placement minimum in nova.cmd.status https://review.openstack.org/568965
14:26:54 openstackgerrit Chris Dent proposed openstack/nova master: [placement] replace deprecated accept.best_match https://review.openstack.org/575127
14:27:27 mriedem FFU bombs away! https://review.openstack.org/#/c/575125/
14:28:02 efried dansmith: cellsv1 job gate failure: testtools.matchers._impl.MismatchError: set(['VM-2', 'VM-1']) != set([u'VM']) <== known?
14:28:27 dansmith not to me
14:28:29 mriedem no, efried link
14:28:46 efried http://logs.openstack.org/75/486475/38/gate/nova-cells-v1/2c2a566/job-output.txt.gz#_2018-06-13_13_26_17_111992
14:29:54 mriedem http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22self.assertEqual(set(%5B'VM-1'%2C%20'VM-2'%5D)%2C%20server_names)%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d
14:30:21 efried same hit http://logs.openstack.org/40/568840/14/check/nova-cells-v1/1f17903/job-output.txt.gz#_2018-06-13_13_18_29_085017 and http://logs.openstack.org/67/567267/15/check/nova-cells-v1/0717ac7/job-output.txt.gz#_2018-06-13_13_27_34_890271
14:30:39 efried okay, yah
14:30:42 mriedem hmm, check queue only
14:30:43 mriedem 4 changes
14:30:52 mriedem last 24 hours
14:30:58 mriedem is it a specific change related to multiple create?
14:31:14 mriedem https://review.openstack.org/#/c/570847/
14:31:23 mriedem if it's all in that series, then it's a bug / race / leak in that series
14:31:33 efried No, the first link was in the xen series
14:31:43 mriedem yeah they are different
14:31:47 mriedem hmm, tempest change in the last 24 hours?
14:32:11 mriedem DING DING DING https://github.com/openstack/tempest/commit/306be6ce35652ca6273745b692aed3e681b30d8c
14:32:25 mriedem https://review.openstack.org/#/c/569199/
14:33:16 mriedem gmann: fyi ^
14:33:22 mriedem i guess we could just blacklist this test for cells v1....
14:33:37 dansmith why does it fail for cellsv1?
14:33:58 mriedem well it's intermittent, so assuming some kind of time delay
14:34:14 dansmith but

Earlier   Later