Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-01
20:46:30 dansmith did we deprecate and not also change devstack to not deploy?
20:46:37 dansmith deploy consoleauth I mean
20:46:43 melwitt yeah
20:46:46 dansmith I would think that would have caught this
20:46:48 dansmith ack
20:46:50 melwitt aye
20:47:14 melwitt yeah, it'd have to do a "if cells v1" before deploy/not deploy but I think that would be fine
20:47:56 melwitt and maybe I could rig up a DNM test for the workaround being enabled to test that in the gate too
20:48:44 dansmith we have cellsv1 conditions
20:52:35 sean-k-mooney melwitt: you could make the rpc conditional and then make a devstack change that depends on it that disables deploying the consoleauth service to test it
20:53:12 melwitt yeah, that's what we're talking about doing
20:54:05 sean-k-mooney cool and then the DNM test change is to reassert that if consoleauth is deploy the workaround still well works
20:56:15 melwitt yeah, that's what I was thinking
21:39:02 openstackgerrit Chris Dent proposed openstack/nova master: Clean up header encoding handling in compute API https://review.openstack.org/606967
21:44:13 openstackgerrit Merged openstack/nova stable/queens: Revert "Make host_aggregate_map dictionary case-insensitive" https://review.openstack.org/605266
21:44:19 openstackgerrit Merged openstack/nova stable/queens: Enforce case-sensitive hostnames in aggregate host add https://review.openstack.org/605267
21:59:15 openstackgerrit Merged openstack/nova stable/pike: Fix instance evacuation with PCI devices https://review.openstack.org/590059
22:04:16 openstackgerrit Merged openstack/nova stable/pike: Update nova network info when doing rebuild for evacuate operation https://review.openstack.org/590070
22:37:26 mriedem so has the great consoleauth undeprecafacation been sorted out?
22:39:20 melwitt mriedem: yeah, I'm working on an alternative approach dan suggested, to add the [workarounds] option condition to the compute/api calls, and see if that will make it possible not to run nova-consoleauth if backported to rocky. I can test it with a devstack change stacked on top. and I think I can use a DNM devstack change to verify that things also work with the [workarounds] option enabled
22:40:20 mriedem so default to workaround enabled and people can disable that when they have drained their existing tokens in the consoleauth service?
22:40:31 mriedem wonder if there is a nova-status check we can add for this...
22:41:17 melwitt I was thinking not to change the default, because only people with long TTL will potentially care about avoiding an instant reset of consoles
22:41:35 melwitt and the workaround is for those peeps
22:41:46 melwitt default TTL is 10 min
22:47:21 sean-k-mooney i think 3 hours of debuging the neutron sriov nic agent is enough for tonight
23:00:24 sean-k-mooney ... so i decided to restack and the thing i was debugging for the last 3 hours is now working and the only codecages i have done is add logging
23:48:52 openstackgerrit melanie witt proposed openstack/nova master: WIP Use nova-consoleauth only if workaround enabled https://review.openstack.org/607068
23:49:41 mriedem melwitt: dansmith: thanks for hitting those pike backports in https://etherpad.openstack.org/p/nova-ocata-em
23:50:38 melwitt thanks for etherpadding them
23:50:52 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Cross-cell resize https://review.openstack.org/603930
#openstack-nova - 2018-10-02
00:04:33 openstackgerrit Matt Riedemann proposed openstack/nova master: Handle missing marker during online data migration https://review.openstack.org/605164
00:05:01 melwitt takashin: hi, could you please check the reply to your -1 on this change? https://review.openstack.org/582147
00:05:38 takashin melwitt: I will check it.
00:05:45 melwitt thank you
00:07:12 takashin melwitt: I would like to know other cores' opinions about https://review.openstack.org/#/c/582147/ .
00:07:46 melwitt takashin: ack. will try to look later
00:14:18 openstackgerrit Merged openstack/nova stable/pike: unquiesce instance after quiesce failure https://review.openstack.org/581454
00:53:38 openstackgerrit Takashi NATSUME proposed openstack/python-novaclient master: Fix test_instance_action functional test failure https://review.openstack.org/606947
01:34:25 openstackgerrit Brin Zhang proposed openstack/nova master: Add compute version 36 to support ``volume_type`` https://review.openstack.org/579360
02:29:39 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Remove more code related to extensions and testing https://review.openstack.org/607088
02:32:26 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Remove more code related to extensions and testing https://review.openstack.org/607088
02:33:42 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Remove more code related to extensions and testing https://review.openstack.org/607088
02:51:10 openstackgerrit Merged openstack/nova stable/pike: Update RequestSpec.flavor on resize_revert https://review.openstack.org/605879
03:12:45 openstackgerrit Ghanshyam Mann proposed openstack/nova master: [WIP]Remove extensions loading framework from wsgi.py https://review.openstack.org/607092
03:21:54 openstackgerrit Merged openstack/nova stable/rocky: nova-manage - fix online_data_migrations counts https://review.openstack.org/605828
03:22:01 openstackgerrit Merged openstack/nova stable/pike: Filter deleted computes from get_all_by_uuids() https://review.openstack.org/604449
03:24:16 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge image_size extension response into image view builder https://review.openstack.org/606845
03:49:07 openstackgerrit Takashi NATSUME proposed openstack/python-novaclient master: Fix test_instance_action functional test failure https://review.openstack.org/606947
04:08:01 openstackgerrit Brin Zhang proposed openstack/nova master: Add compute API version for when a ``volume_type`` is requested https://review.openstack.org/605573
04:24:54 openstackgerrit Merged openstack/nova stable/pike: Use ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa in tree https://review.openstack.org/581444
06:22:03 openstackgerrit OpenStack Proposal Bot proposed openstack/nova stable/rocky: Imported Translations from Zanata https://review.openstack.org/604260
07:11:42 openstackgerrit Radoslav Gerganov proposed openstack/nova master: VMware: Live migration of instances https://review.openstack.org/270116
07:54:40 bauzas good morning nova
08:05:01 gryf morning guys
08:11:03 bhagyashris Hi all, I am facing one issue on current master while live migration of the instance , I am trying to live migrate the instance but it's not migrating that instance on given destination node and that instance remains on source node with the status as active, checked n-cpu, n-condunctor logs but didn't get any error at first time.
08:12:06 bhagyashris And if I again try to live migrate that instance then it's giveing me Migration pre-check error: Binding failed for port <port-id> please check neutron logs for more information. (HTTP 400
08:12:58 bhagyashris Can anyone please help me
08:39:44 takashin Nova cores, could you review https://review.openstack.org/#/c/606947/ ? It is a fix for a gate failure in python-novaclient.
08:48:40 gibi bhagyashris: what do you see in the neutron log about the bind failure?
08:52:52 gibi bhagyashris: have you specified target host for the migration ? Have you use force=True in the migration request?
08:54:42 bhagyashris gibi: this is log I get at neutron server side http://paste.openstack.org/show/731235/
08:55:16 bhagyashris gibi: yes I have specified target host in request but not force=True
08:57:08 openstackgerrit Merged openstack/nova master: Fix neutron-tempest-linuxbridge irrelevant-files https://review.openstack.org/606616
08:57:17 openstackgerrit Merged openstack/nova master: Add hide server address tests in test_serversV21.py https://review.openstack.org/605435
08:57:27 openstackgerrit Merged openstack/nova stable/ocata: [Stable Only] Remove soft-deleted instances from quota_usages https://review.openstack.org/570653
08:57:38 openstackgerrit Merged openstack/nova master: Follow up for Ib6f95c22ffd3ea235b60db4da32094d49c2efa2a https://review.openstack.org/604743
08:57:40 gibi bhagyashris: if force is False then nova checks the host with the scheduler. did you looked at the scheduler log for the first migration attempt?
08:57:45 openstackgerrit Merged openstack/nova master: Raise error on timeout in wait_for_versioned_notifications https://review.openstack.org/604859
09:01:24 bhagyashris gibi: checked the n-sch log but there I didn't get any logs
09:01:53 gibi bhagyashris: it is soo strange that nothing in the nova-conductor log
09:02:27 bhagyashris gibi: n-cond or n-sch?
09:02:34 gibi nova-cond
09:02:40 bhagyashris At n-cond I got the logs
09:02:59 gibi bhagyashris: could you post the n-cond log?
09:06:35 bhagyashris gibi: #link: http://paste.openstack.org/show/731237/
09:16:49 gibi bhagyashris: are you sure that force wasn't set to True in the live migration request? the last log line in http://paste.openstack.org/show/731237/ shows that claim_resources_on_destination was called to make the allocation for the instance on the dest host but that function only called for force live migrate and force evacuate
09:17:41 bhagyashris gibi: I am sure I havent set force to true
09:19:26 bhagyashris gibi: openstack server migrate <instance-id> --live <destination-host> --block-migration this command I used for live migration
09:26:18 gibi bhagyashris: as you did not specify any api microverion openstack client used the minimum and in there specifying a host means force by default
09:26:21 gibi bhagyashris: https://developer.openstack.org/api-ref/compute/#live-migrate-server-os-migratelive-action
09:26:43 gibi bhagyashris: this explains why you don't see anything in the nova-sch log
09:30:14 bhagyashris gibi: ohh ok
09:33:22 gibi bhagyashris: could you check what is the status of the first live migration attempt with nova migration-list command?
09:34:25 bhagyashris gibi: it's showing me the status as error
09:35:58 gibi bhagyashris: and nothing relevan on the source compute or dest compute n-cpu log?
09:38:35 bhagyashris gibi: at destination compute it' check destination check data is LibvirtLiveMigrateData
09:43:14 gibi bhagyashris: I'm clueless how that migration failed without leaving any trace of information anywhere what was the reason fo the failure. I think when it failed it failed to clean up some port binding stuff in Neutron and that causes the second migration attempt to fail at port binding
09:45:12 bhagyashris gibi: yeah... but at very first why it's not migrating that instance on destination node with any failure :(
09:52:34 openstackgerrit Brin Zhang proposed openstack/nova master: Add microversion 2.67 to support volume_type https://review.openstack.org/606398
10:18:27 openstackgerrit Sam Morrison proposed openstack/python-novaclient master: Fix up userdata argument to rebuild. https://review.openstack.org/605341
10:32:53 openstackgerrit Merged openstack/nova stable/pike: Fix nova-status "_check_resource_providers" check https://review.openstack.org/600113
10:34:15 openstackgerrit Merged openstack/python-novaclient master: Fix test_instance_action functional test failure https://review.openstack.org/606947
10:42:16 sean-k-mooney gibi: that livemigration error bhagyashris had is bizarr i have never seen neutron trigger FlushError i would have liked to see the n-cpu log on the source and dest node i suspect that there was a libvirt/qemu error that resulted in the first failure but not sure what would cause the second.
10:46:35 gibi sean-k-mooney: bhagyashris stated that there was no relevant log in the nova-cpu services :/
10:46:54 gibi sean-k-mooney: but I share your feeling about the weiredness of the fault
10:49:33 sean-k-mooney ya i would still be interested in seeing the libvirt qemu instance log to confirm there were no qemu/libvirt errors but it is strange. im wonder if the neutron issue could have something to do with multiple port bindings. e.g. we know 1 or 2 bindings now works but did anyone ever test with 3. with out a way to reproduce however its really just guess work on my part
10:51:24 openstackgerrit Matthew Booth proposed openstack/nova master: Fix a race evacuating instances in an anti-affinity group https://review.openstack.org/605436
12:03:36 openstackgerrit Takashi NATSUME proposed openstack/python-novaclient master: Update the contributor guide https://review.openstack.org/606926
12:04:08 openstackgerrit Takashi NATSUME proposed openstack/python-novaclient master: Update the CLI reference https://review.openstack.org/606871

Earlier   Later