Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-04-05
21:05:00 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: test live_migration_wait_for_vif_plug=True https://review.openstack.org/558006
21:05:39 melwitt you were probably like, "what ... link the same bug you told me about?"
21:06:04 melwitt haha
21:17:35 arvindn05 i am having issues running tox -epep8. Was working fine unti it wasnt...
21:17:54 arvindn05 gettting the error netifaces.c:1:20: fatal error: Python.h: No such file or directory
21:18:04 arvindn05 anyone saw this error before?
21:20:28 melwitt we recently switched it to run with python3 by default
21:20:49 melwitt so maybe you need the python3-dev package, or the like
21:21:47 openstackgerrit Arvind Nadendla proposed openstack/nova master: Update ImageMetaProp object to expose traits https://review.openstack.org/557795
21:24:15 arvindn05 melwitt: thanks...i tried that based on similar recommendation
21:24:18 arvindn05 The following packages have unmet dependencies:
21:24:18 arvindn05 python3-dev : Depends: libpython3-dev (= 3.5.1-3) but it is not going to be installed
21:24:19 arvindn05 Depends: python3.5-dev (>= 3.5.1-2~) but it is not going to be installed
21:24:19 arvindn05 E: Unable to correct problems, you have held broken packages.
21:24:55 arvindn05 unable to install them because it says i have unmet dependencies...i looked for broken packages...but no luck
21:29:01 arvindn05 melwitt: nevermind got it to work. Thanks for the hint
21:29:20 arvindn05 had to use a different package manager aptitude to resolve the dependencies
21:29:45 melwitt k, cool
21:32:56 openstackgerrit Merged openstack/nova master: [placement] Fix incorrect exception import https://review.openstack.org/558916
21:47:43 melwitt looks like limestone hosts still timing out
22:27:53 openstackgerrit Michael Still proposed openstack/nova master: Move xenapi disk resizing to privsep. https://review.openstack.org/552242
22:27:54 openstackgerrit Michael Still proposed openstack/nova master: Sync xenapi and libvirt on what flags to pass e2fsck. https://review.openstack.org/554078
22:27:54 openstackgerrit Michael Still proposed openstack/nova master: Move xenapi partition copies to privsep. https://review.openstack.org/553605
22:27:55 openstackgerrit Michael Still proposed openstack/nova master: Move image conversion to privsep. https://review.openstack.org/554437
22:27:55 openstackgerrit Michael Still proposed openstack/nova master: We don't need utils.trycmd any more. https://review.openstack.org/554439
22:27:56 openstackgerrit Michael Still proposed openstack/nova master: We no longer need rootwrap. https://review.openstack.org/554438
22:31:40 logan- melwitt: would you mind linking me to the job log? im not seeing the fail in logstash. we (limestone) are actively monitoring and fixing the timeouts noticed yesterday and the failure rate has been much lower today.
22:33:44 openstackgerrit Michael Still proposed openstack/nova master: Use a pythonic delete, with a retry. https://review.openstack.org/554793
22:51:55 cfriesen_ eandersson: I was off working other stuff today, didn't see your messages. If you have questions abut numa stuff, feel free to ping me. Just wondering, do you have the NUMATopologyFilter enabled in the "scheduler_default_filters" config option in nova.conf?
23:01:42 openstackgerrit Eric Fried proposed openstack/nova master: WIP: placement: Granular GET /allocation_candidates https://review.openstack.org/517757
23:04:13 mriedem cfriesen_: i think the net is https://bugs.launchpad.net/nova/+bug/1417667/comments/45
23:04:13 openstack Launchpad bug 1417667 in OpenStack Compute (nova) "migration/evacuation/resize/unshelve of instance with NUMA topology needs to recalculate NUMA topology" [Medium,In progress] - Assigned to sahid (sahid-ferdjaoui)
23:04:38 mriedem cfriesen_: ah you replied
23:04:48 mriedem ok, so i think we should probably close that bug
23:08:03 mriedem mikal: fyi, i'm cleaning up https://review.openstack.org/#/c/324720/
23:11:43 eandersson cfriesen_, yea it's enabled
23:15:15 openstackgerrit Arvind Nadendla proposed openstack/nova master: Update ImageMetaProp object to expose traits https://review.openstack.org/557795
23:16:14 openstackgerrit Merged openstack/nova master: Add --enable and --disable options to nova-manage update_cell https://review.openstack.org/555416
23:16:25 openstackgerrit Merged openstack/nova master: Update the cells FAQs and scheduler maintenance docs. https://review.openstack.org/556459
23:16:53 arvindn05 mriedem: added the check to unit test for the image traits review
23:20:37 Spaz-Home Morning folks
23:27:55 openstackgerrit Merged openstack/nova master: update_provider_tree devref and docstring updates https://review.openstack.org/553476
23:30:18 cfriesen_ mriedem: I guess we've got https://bugs.launchpad.net/nova/+bug/1289064 to track the live migration issues, so yeah I'm okay with closing that one.
23:30:18 openstack Launchpad bug 1289064 in OpenStack Compute (nova) "live migration of instance should claim resources on target compute node" [Medium,In progress] - Assigned to sahid (sahid-ferdjaoui)
23:33:42 cfriesen_ eandersson: in that case I'm a bit surprised. Do you see resource claims being made on the destination?
23:52:40 Spaz-Home Interesting.. does it only do those claims if a host is specified?
23:55:30 eandersson cfriesen_, > During the sync_power process the instance has moved from host X to host Y
23:55:36 eandersson Is all I see in the normal logs
23:55:55 cfriesen_ eandersson: mitaka, right?
23:55:58 eandersson yea
23:56:04 eandersson > CPUPinningInvalid: Cannot pin/unpin cpus [1, 21] from the following pinned set [0, 1, 20, 5, 25, 21]
23:57:15 eandersson It's complaning about that after it was moved
#openstack-nova - 2018-04-06
00:00:33 cfriesen_ eandersson: you might try enabling debug logs and redoing the test. I'd expect that on the dest nova-compute rebuild_instance() would call ResourceTracker.rebuild_claim()
00:00:48 cfriesen_ eandersson: that in turn should emit logs like "Attempting claim: ....."
00:02:10 cfriesen_ Spaz-Home: no, it should claim resources for scheduler-chosen destinations as well since even now placement doesn't do cpu pinning, hugepages, etc.
00:02:52 cfriesen_ eandersson: just to be clear, the "Attempting claim" log should come out at "info" level
00:03:45 eandersson Is that in the compute logs?
00:03:54 cfriesen_ eandersson: yes, on the dest node
00:04:42 eandersson I only see three of those, and none from the evacuated nodes
00:05:08 cfriesen_ eandersson: that claim is what calls _test_numa_topology(), which is what handles setting up the CPU pinning for the instance that is being evacuated.
00:06:11 openstackgerrit Michael Still proposed openstack/nova master: Rename network.utils to network.linux_utils. https://review.openstack.org/559197
00:07:33 cfriesen_ eandersson: I'd expect to see the "Attempting claim" log on instance boot as well.
00:09:55 eandersson > Creating Image
00:09:56 eandersson > Instance spawned successfully.
00:10:12 eandersson That is it for INFO logging
00:11:13 eandersson I'll enable debug logging and see in a bit
00:11:14 cfriesen_ do you see a "Rebuilding instance" log on the dest compute node when doing the evacuate?
00:16:14 eandersson Yea - missed that line > [instance: xxxx] Rebuilding instance
00:16:35 eandersson > disk not on shared storage, rebuilding from: 'yyyy'
00:16:53 cfriesen_ okay, so pretty much right after that I'd expect to see an "Attempting claim" log, followed by a "Claim successful" log
00:19:38 cfriesen_ if you want you could add extra logging in nova.compute.manager.ComputeManager.rebuild_instance() around where it sets "rebuild_claim" and then calls it.
00:21:08 cfriesen_ eandersson: crazy question maybe, but did you live-migrate any of these pinned instances?
00:21:26 eandersson nope
00:21:34 eandersson I created a lab and 5 new VMs.
00:21:41 eandersson with two computes
00:22:07 eandersson evacuated the two vms on compute2
00:24:37 eandersson > Lock "compute_resources" acquired by "nova.compute.resource_tracker.rebuild_claim"
00:28:26 cfriesen_ eandersson: that's promising, it corresponds to the locking decorator on ResourceTracker.rebuild_claim
00:31:10 cfriesen_ eandersson: just had a thought...what virt driver are you using?
00:32:16 cfriesen_ eandersson: do you see a log like "Virt driver does not support 'get_available_resource'. Compute tracking is disabled." ?
00:34:07 eandersson nope
00:35:54 eandersson cfriesen_, default kvm
00:47:59 eandersson libvirt, qemu-kvm
01:31:37 melwitt logan-: here's the link http://logs.openstack.org/84/534384/24/check/tempest-full/ae551f8/
01:58:28 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove non-libguestfs file injection for libvirt https://review.openstack.org/324720
02:34:26 gmann alex_xu_: ping
02:45:22 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/virt/xenapi/test_vm_utils.py (3) https://review.openstack.org/559258
04:36:59 homeski is osprofiler working in Nova --Newton?
04:37:09 homeski can't find any docs on it ..
04:53:30 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in test_user_data.py https://review.openstack.org/559264
05:23:50 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/api/openstack/compute/test_hosts.py https://review.openstack.org/559266
05:45:57 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/api/*/test_instance_actions.py https://review.openstack.org/559269
06:53:42 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/548772
07:11:34 kashyap zigo: Thanks. Will look in a few.
07:19:18 openstackgerrit Roman Dobosz proposed openstack/nova master: Pass hosts or nodes to HostStates update method depending on a driver. https://review.openstack.org/556761
07:19:18 openstackgerrit Roman Dobosz proposed openstack/nova master: Added weight for aggregate soft (anti) affinity. https://review.openstack.org/556762
07:19:19 openstackgerrit Roman Dobosz proposed openstack/nova master: Added node field for InstanceGroup objects https://review.openstack.org/559284
08:32:00 kaisers1 mikal_: ping
08:46:27 kashyap It's often best to just ask the question -- when the person wakes up, they could respond directly to the question.
09:27:45 openstackgerrit Surya Seetharaman proposed openstack/nova master: Cleanup patch for the cell-disable series https://review.openstack.org/559299

Earlier   Later