Earlier  
Posted Nick Remark
#openstack-nova - 2017-02-09
12:12:05 EmilienM yes, it was a pain in the *** for us, we couldn't release and I think at some point we might be able to release very soon (early next week maybe)
12:12:16 EmilienM cdent: I'll let you know if something changes
12:12:33 EmilienM johnthetubaguy, cdent, dansmith (and matt but offline): thx for your help, appreciated
12:13:19 cdent Always happy to help EmilienM
12:17:54 openstackgerrit Wang Qing wu proposed openstack/nova master: PowerVM Driver: spawn/destroy https://review.openstack.org/391288
12:38:10 gabor_antal johnthetubaguy: thanks for the review, it was missed from the commit, I added it!
12:38:57 gabor_antal johnthetubaguy: could you please recheck my patch? https://review.openstack.org/#/c/431030/
12:39:23 johnthetubaguy gabor_antal: did you send the update already?
12:39:40 openstackgerrit Gábor Antal proposed openstack/nova master: Enable global hacking checks and removed local checks https://review.openstack.org/431030
12:39:48 gabor_antal yeah, just my ssh was a bit slow
12:40:08 gabor_antal looks like it needed some time to finish the review command
12:41:21 johnthetubaguy gabor_antal: no worries, looks good, thanks!
12:41:39 gabor_antal great, thanks for the wuick review! :)
12:42:16 gabor_antal Would you please also check https://review.openstack.org/#/c/389338/5 if you have some time? this is a dependency for my patch
12:45:48 alex_xu johnthetubaguy: re: https://review.openstack.org/#/c/424931/4, if we raise exception instead of just pick the first one, is that kind of API behaviour change?
12:46:07 johnthetubaguy alex_xu: yeah, I think the best thing is just not changing it really
12:46:27 johnthetubaguy gabor_antal: yeah, I wasn't sure about till we actually release ocata
12:47:25 alex_xu johnthetubaguy: yeah, i'm ok without change. it is just reduce some useless loop.
12:51:17 gabor_antal johnthetubaguy: okay, maybe it should wait, you know that better tahn me. btw fyi the hacking changes are not proposed automatically by the Proposal bot. We have speaked about it with mriedem, here's some proof: https://review.openstack.org/#/c/409626/
12:53:31 gabor_antal https://github.com/openstack/requirements/blob/master/blacklist.txt maybe thats better link
13:03:57 openstackgerrit Gábor Antal proposed openstack/nova master: Removed unnecessary parantheses and fixed formation https://review.openstack.org/370549
13:05:36 gabor_antal johnthetubaguy: would you please recheck https://review.openstack.org/#/c/370549/ too? I only resolved merge conflict
13:49:38 openstackgerrit Jeremy Liu proposed openstack/nova master: [3/3]Replace six.iteritems() with .items() https://review.openstack.org/415080
13:52:14 openstackgerrit John Garbutt proposed openstack/nova master: Add device_id when creating ports https://review.openstack.org/431545
14:13:14 jaypipes lyarwood: lol, yeah :)
14:19:40 gibi johnthetubaguy: as you requested on the nova meeting, here is a bp that needs re-approve for pike https://blueprints.launchpad.net/nova/+spec/versioned-notification-transformation-pike
14:20:04 gibi johnthetubaguy: also if you have rights could you change the assignee on that bp to me?
14:20:37 gibi johnthetubaguy: it seems that the launchpad id 'gibi' is somebody else than me :)
14:20:46 johnthetubaguy gibi: seems lp thinks other people are gibi, yeah
14:20:53 openstackgerrit Vladik Romanovsky proposed openstack/nova master: libvirt: set vlan tag for macvtap on SR-IOV VFs https://review.openstack.org/421186
14:21:05 johnthetubaguy gibi: does that look better now?
14:21:30 gibi johnthetubaguy: thanks a lot! it is perfect now
14:29:03 johnthetubaguy gibi: cool, no problem
14:29:30 openstackgerrit John Garbutt proposed openstack/nova master: Stop leaking ports on port_create timeout https://review.openstack.org/347183
14:30:22 vladikr sfinucan, johnthetubaguy, I've added a todo to remove the ip link command after the libivrt version bump. https://review.openstack.org/421186
14:30:34 sfinucan vladikr: (y)
14:30:37 sfinucan I'll review now
14:30:43 vladikr sfinucan, thanks
14:31:27 johnthetubaguy vladikr: thanks, lgtm
14:33:01 openstackgerrit John Garbutt proposed openstack/nova master: Stop leaking ports on port_create timeout https://review.openstack.org/347183
14:34:57 johnthetubaguy BobBall: I am just imagining lots of -1s from the XenServer CI again?
14:35:43 EmilienM cdent: do we managed to promote TripleO CI with new nova but somehow we're unable to promote Puppet CI (which are btw much simpler to debug than tripleo)
14:36:03 EmilienM cdent: s/do we/we/
14:36:59 EmilienM cdent: http://logs.openstack.org/15/424915/13/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/42fa9b9/logs/nova/nova-placement-api.txt.gz#_2017-02-09_12_29_24_213
14:37:01 cdent EmilienM: hrmm. Any clues?
14:37:02 jaypipes johnthetubaguy, bauzas: easy +2 here: https://review.openstack.org/#/c/430399/2
14:37:09 sfinucan Sorry for what I'm about to do, but it should be the last time I've to do it (all the "risky" stuff is at the end and can bitrot for a while)
14:37:15 EmilienM cdent: I don't think placement api service is ready, I don't see the POST
14:38:51 EmilienM cdent: we're running nova from rc1
14:39:34 openstackgerrit Stephen Finucane proposed openstack/nova master: netutils: Ignore 'use_ipv6' for network templates https://review.openstack.org/430910
14:39:34 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'use_ipv6' https://review.openstack.org/430911
14:39:35 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Ignore 'use_ipv6' for port filters https://review.openstack.org/430909
14:39:35 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'ipv6_backend' https://review.openstack.org/430716
14:39:36 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Fix indentation in conf/netconf https://review.openstack.org/430717
14:39:36 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate various 'dns_', 'dhcp_' options https://review.openstack.org/431018
14:39:37 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'ebtables_' options https://review.openstack.org/431019
14:39:37 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate various 'dns_' options https://review.openstack.org/431017
14:39:38 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'allow_same_net_traffic' https://review.openstack.org/430913
14:39:38 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate remaining options in conf/network https://review.openstack.org/431022
14:39:39 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Final cleanups in conf/network https://review.openstack.org/431023
14:39:39 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Ignore 'allow_same_net_traffic' for port filters https://review.openstack.org/430912
14:39:40 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate various 'iptables_' options https://review.openstack.org/430915
14:39:40 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'fake_network' https://review.openstack.org/431020
14:39:41 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'firewall_driver' https://review.openstack.org/412390
14:39:41 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'metadata_host', 'metadata_port' https://review.openstack.org/430914
14:39:42 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'send_arp_for_ha', 'send_arp_for_ha_count' https://review.openstack.org/431021
14:40:18 cdent EmilienM: Looks like a similar situation to what we were seeing before: n-cpu is not finding the placement endpoint, never records inventory for the compute-node. When n-scheduler starts talking to the placement-api that's the first time it is being talked to, but there's no data there
14:40:43 EmilienM cdent: what is weird is that it passed on the other scenario
14:40:48 EmilienM cdent: which means we're racy now
14:41:04 EmilienM cdent: all our scenarios deploy nova/placement the same way
14:41:15 sfinucan johnthetubaguy, sdague: I think the first 10 or so of those are noddy but helpful nonetheless. Would appreciate ye taking a glance and (hopefully) sending them on their way
14:41:17 EmilienM owalsh: ^ fyi
14:41:42 EmilienM cdent: i'm currently investigating puppet catalog and resources orchestration
14:42:12 sfinucan They should clean up and mark virtually all of the nova-network opts for future removal, while the latter ones untangle the repurposed nova-net opts
14:42:14 cdent EmilienM: it would be great if we could get a summary, with times, of when things happen, because trying to inspect multiple logs at once is proving rather difficult
14:42:37 johnthetubaguy cdent: I wondering if we only try fetching the service catalog once? would we log something for that?
14:43:12 EmilienM cdent: sure let me prepare that: etherpad.openstack.org/p/puppet-nova-trunk-debug
14:43:40 cdent johnthetubaguy: I don't know. This is part of why I've been saying that I think our understanding of keystoneauth1 is incomplete. I personally haven't had a chance to fix that for myself yet.
14:44:43 johnthetubaguy cdent: yeah, I have been digging in there recently for the service token stuff, I know I keep getting lost in there still
14:46:40 EmilienM cdent: I tried to grep useful things : http://paste.fedoraproject.org/551748/66515901/
14:47:22 EmilienM cdent: if it's too noisy I can reduce the file
14:47:36 EmilienM cdent: so that file is the catalog of a job that failed
14:49:39 openstackgerrit Takashi NATSUME proposed openstack/nova master: Enable cold migration with target host(1/2) https://review.openstack.org/408955
14:50:20 openstackgerrit Takashi NATSUME proposed openstack/nova master: Enable cold migration with target host(2/2) https://review.openstack.org/408964
14:50:34 EmilienM cdent: I reduced the catalog: http://paste.fedoraproject.org/551757/86651825/
14:50:39 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add functional tests for cold migration to same host https://review.openstack.org/414926
14:50:57 openstackgerrit Takashi NATSUME proposed openstack/nova master: api-ref: Add parameters in cold migrate action https://review.openstack.org/410042
14:51:26 cdent EmilienM: I'm not sure that providing a log of the puppet activity (that is what that is isn't it?0 really tells us what we need to know. We need to understand the interactions of the various services, in a timeline, which means extracting signal from the logs of the various services, unforutnately. What I mean is:
14:52:50 EmilienM cdent: what gives the catalog is timestamps of when service run
14:52:56 cdent at time X something is registered in the catalog, at time Y something starts, at time Z n-cpu tries to talk to placement for the first time, at time A the placement service records its first hit, at time B n-cpu asks for a service catalog
14:52:59 EmilienM cdent: for ex, you can see when we configure cells, etc
14:53:01 cdent when the services _run_ is not enough
14:53:27 cdent it's when the actions happen that we're trying to detangle, isn't it? I'm not entirely sure.
14:53:36 EmilienM right
14:57:37 sdague the service catalog is embedded in the keystone token
14:58:03 mordred keypairs are per-AZ?
14:58:39 sdague johnthetubaguy / cdent perhaps there is a placement service call before it's registered?
14:58:45 EmilienM cdent: damn, now we promoted tripleo CI, I also see jobs flapping with placement service
14:58:53 EmilienM cdent: it's working sometimes but failing very often
14:59:25 nicolasbock Hi, I am trying to setup the placement-api

Earlier   Later