Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-12-19
04:52:34 openstackgerrit Takashi NATSUME proposed openstack/nova master: api-ref: Verify parameters in servers.inc https://review.openstack.org/528201
05:21:21 openstackgerrit Nakanishi Tomotaka proposed openstack/nova master: Use Placement API to check resource usage https://review.openstack.org/528953
06:05:58 openstackgerrit Rajesh Tailor proposed openstack/nova master: Host addition host-aggregate should be case-sensitive https://review.openstack.org/498334
06:06:33 openstackgerrit Rajesh Tailor proposed openstack/nova master: Fix case-sensitivity for metadata keys https://review.openstack.org/504885
07:00:31 openstackgerrit jichenjc proposed openstack/nova master: Remove 'nova-manage shell' command https://review.openstack.org/521835
07:00:32 openstackgerrit jichenjc proposed openstack/nova master: Remove 'nova-manage account' and 'nova-manage project' https://review.openstack.org/521833
07:00:32 openstackgerrit jichenjc proposed openstack/nova master: Remove 'nova-manage logs' command https://review.openstack.org/522133
07:12:56 Guest72028 Hello All
07:19:17 Guest72028 I have a questions for nova specialists about nova ressources: is it possible to have compute nodes spares ? is it possible to reserve compute ressources , prioritize rebuild order of VM while evacuation process ?
08:25:12 openstackgerrit 龚肖 proposed openstack/nova stable/pike: compute: Catch binding failed exception while init host https://review.openstack.org/528985
08:35:02 openstackgerrit TommyLike proposed openstack/nova master: Remove redundant try/except block when authorize https://review.openstack.org/528991
08:35:38 openstackgerrit Mr Rambo proposed openstack/nova master: Fix the problems that volume-backed server rebuild https://review.openstack.org/528994
09:05:26 lyarwood mdbooth: thanks for the reviews yesterday btw, is your uuid series ready for review?
09:07:00 mdbooth lyarwood: Mostly, yes.
09:07:17 mdbooth lyarwood: Well, most of it
09:08:22 mdbooth lyarwood: I'd very much like your input on this one: https://review.openstack.org/#/c/528363/
09:08:41 mdbooth But I'm about to rebase that into the main series, because I need it for the next patch
09:09:33 mdbooth The series is here: https://review.openstack.org/#/q/topic:bp/local-disk-serial-numbers+(status:open+OR+status:merged)
09:10:36 lyarwood mdbooth: ack, looking
09:13:27 mdbooth lyarwood: Thanks
09:37:46 openstackgerrit Mr Rambo proposed openstack/nova master: Fix the problems that volume-backed server rebuild https://review.openstack.org/528740
09:41:28 maciejjozefczyk gibi jaypipes: Hello, thanks for your comments :) Please check my response https://review.openstack.org/#/c/520024/ Thank you!
10:02:49 lyarwood mdbooth: so the DriverVolumeBlockDevice change LGTM, could we set self.connection_info earlier and just pass that around within the method?
10:05:03 mdbooth lyarwood: Which function are you referring to specifically?
10:05:51 lyarwood mdbooth: _legacy_volume_attach or _volume_attach in block_device.py
10:06:28 mdbooth Sec, just reading and digesting your review comment
10:07:07 mdbooth lyarwood: Ah, you're talking about *not* changing the interface?
10:07:17 mdbooth And continuing to pass connection info explicitly?
10:07:46 lyarwood mdbooth: no, just setting self.connection_info when we actually fetch it from cinder
10:07:56 mdbooth Ah...
10:08:45 mdbooth Yep, that would be cleaner.
10:08:54 mdbooth It would make the patch a bit noisier, though...
10:08:59 mdbooth And it's already pretty noisy.
10:09:01 mdbooth Hmm...
10:09:06 lyarwood yeah I assumed that's why you didn't do it tbh
10:16:06 norman hi all, anyone know why domainxml of the live-migrated instance has <features> under <cpu> section, but new booted instances not
10:17:04 norman I'd trying go through the code ,failed to find clues. I am still using the Mitaka, not sure the new version is Ok or not
10:17:56 Tahvok Hey guys, I'm unable to find a good example of ComputeCapabilitiesFilter. Do you apply it on the flavor's metadata or on the compute host somehow? If it should me on the compute host, so where exactly do I specify my 'capabilities' filter?
10:28:25 openstackgerrit Merged openstack/nova stable/ocata: Make request_spec.spec MediumText https://review.openstack.org/528332
10:28:31 openstackgerrit Merged openstack/nova master: Fix the formatting for 2.56 in the compute REST API history doc https://review.openstack.org/528114
10:41:02 lyarwood kashyap: http://logs.openstack.org/38/528338/4/check/legacy-tempest-dsvm-multinode-live-migration/d867726/logs/subnode-2/screen-n-cpu.txt.gz#_2017-12-18_20_20_13_894 - seeing this on stable/newton, LM failure for a single paused instance, looks like the remote libvirtd didn't respond in time, have you seen this before?
10:41:09 lyarwood kashyap: http://logs.openstack.org/38/528338/4/check/legacy-tempest-dsvm-multinode-live-migration/d867726/job-output.txt.gz#_2017-12-18_20_20_56_230121 is the tempest failure
10:42:04 kashyap lyarwood: Is this only stable/newton?
10:42:29 lyarwood kashyap: that's the only place I've seen it thus far
10:50:20 kashyap lyarwood: So I looked at all the logs, one thing that potentially jumps out at me is in the source QEMU log:
10:50:23 kashyap [...]
10:50:24 kashyap [...]
10:50:27 kashyap warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
10:50:30 kashyap main-loop: WARNING: I/O thread spun for 1000 iterations
10:50:52 kashyap Now, that warning isn't really an egregious error (& upstream QEMU is aware of it; it's a hard thing to fix), but that might be contributing to it
10:51:04 kashyap Where "it" being the timeout we see:
10:51:05 kashyap 2017-12-18 20:20:13.880+0000: 16820: error : virKeepAliveTimerInternal:143 : internal error: connection closed due to keepalive timeout
10:51:08 kashyap 2017-12-18 20:20:13.881+0000: 16825: error : virKeepAliveTimerInternal:143 : internal error: connection closed due to keepalive timeout
10:51:11 kashyap (From the source)
10:52:10 lyarwood kashyap: kk, so the dest was stuck, didn't send a keepalive and everything dies?
10:52:32 kashyap lyarwood: Yeah, from the destination livirtd log:
10:52:35 kashyap 2017-12-18 20:20:19.394+0000: 23816: error : qemuMonitorIOWrite:545 : Unable to write to monitor: Broken pipe
10:53:01 kashyap The above means libvirt lost access to the QMP socket connection, i.e. VM died
10:53:39 lyarwood kashyap: it's paused, that shouldn't cause the QMP socket to die however right?
10:55:27 kashyap lyarwood: Yeah, clearly something is wonky. I'll check this w/ Dave Gilbert as he meditates on migration
10:55:53 kashyap But the process _is_ killed, as we see from the destination (http://logs.openstack.org/38/528338/4/check/legacy-tempest-dsvm-multinode-live-migration/d867726/logs/subnode-2/libvirt/qemu/instance-00000004.txt.gz):
10:55:58 kashyap 2017-12-18T20:17:51.612132Z qemu-system-x86_64: terminating on signal 15 from pid 16820
10:58:06 openstackgerrit Chen Hanxiao proposed openstack/nova master: libvirt: don't call sync_guest_time if qga is not enabled https://review.openstack.org/524836
10:58:26 lyarwood kashyap: kk, thanks, I'm going to recheck this change and see if we can hit it again
10:58:37 kashyap So it is the live block migration, right
10:58:43 kashyap (To pause it)
10:58:47 kashyap LiveMigrationTest.test_live_block_migration_paused
10:59:10 lyarwood kashyap: yup, LM of a paused instance without shared storage
11:01:22 openstackgerrit Matthew Booth proposed openstack/nova master: Rename block_device_info_get_root https://review.openstack.org/529028
11:01:23 openstackgerrit Matthew Booth proposed openstack/nova master: Add local_root to block_device_info https://review.openstack.org/529029
11:03:00 openstackgerrit Matthew Booth proposed openstack/nova master: Expose driver_block_device fields as attributes https://review.openstack.org/528362
11:03:01 openstackgerrit Matthew Booth proposed openstack/nova master: Pass DriverBlockDevice to driver.attach_volume https://review.openstack.org/528363
11:13:27 openstackgerrit Merged openstack/nova master: Implement query param schema for migration index https://review.openstack.org/518644
11:14:16 kashyap lyarwood: So a couple of things from interacting w/ Dan & Dave from QEMU:
11:14:54 kashyap (1) You see the _later_ log messages (on destination) are 3 before the earlier log message.
11:15:23 kashyap So that's some weird timestamps there.
11:17:08 kashyap (2) We're not the first to hit this case; there's this existing bug https://bugzilla.redhat.com/show_bug.cgi?id=1367620 ("storage migration fails due to keepalive timeout")
11:17:08 openstack bugzilla.redhat.com bug 1367620 in libvirt "storage migration fails due to keepalive timeout" [High,Assigned] - Assigned to jdenemar
11:18:59 openstackgerrit Matthew Booth proposed openstack/nova master: Expose BDM uuid to drivers https://review.openstack.org/529037
11:19:24 kashyap lyarwood: Wonder if you have a link to how many times this was hit in the past / this week?
11:21:52 lyarwood kashyap: I don't have one to hand now but I can create one, and an upstream bug for this
11:22:04 mdbooth lyarwood: Don't know if you're still looking at it, but I've been messing with that series this morning.
11:22:06 lyarwood assuming we've seen it more than once
11:22:09 mdbooth Not quite finished yet.
11:22:32 lyarwood mdbooth: kk I stopped after the earlier change sorry
11:23:00 mdbooth lyarwood: NP. I'd have been messing you about anyway.
11:23:13 kashyap lyarwood: I'm creating a potential reproducer, and can file one with that
11:23:23 kashyap lyarwood: But if you've already drafted a bug / issue, go ahead & submit it
11:27:58 openstackgerrit Merged openstack/nova master: Remove 'nova-manage shell' command https://review.openstack.org/521835
11:32:15 openstackgerrit Matthew Booth proposed openstack/nova master: Give volume DriverBlockDevice classes a common prefix https://review.openstack.org/526346
11:32:16 openstackgerrit Matthew Booth proposed openstack/nova master: Add DriverLocalImageBlockDevice https://review.openstack.org/526347
11:34:34 openstackgerrit Matthew Booth proposed openstack/nova master: Use real block_device_info data in test_blockinfo https://review.openstack.org/527916
11:34:34 openstackgerrit Matthew Booth proposed openstack/nova master: Rename block_device_info_get_root https://review.openstack.org/529028
11:34:35 openstackgerrit Matthew Booth proposed openstack/nova master: Add local_root to block_device_info https://review.openstack.org/529029
11:34:35 openstackgerrit Matthew Booth proposed openstack/nova master: Expose driver_block_device fields as attributes https://review.openstack.org/528362
11:34:36 openstackgerrit Matthew Booth proposed openstack/nova master: Pass DriverBlockDevice to driver.attach_volume https://review.openstack.org/528363
11:53:45 openstackgerrit Stephen Finucane proposed openstack/nova master: Change 'InstancePCIRequest' spec field https://review.openstack.org/449257
11:53:46 openstackgerrit Stephen Finucane proposed openstack/nova master: Add Neutron port capabilities to devspec in request https://review.openstack.org/451777
11:53:46 openstackgerrit Stephen Finucane proposed openstack/nova master: Format NIC features using os-traits definitions https://review.openstack.org/466051
12:00:28 stephenfin ralonsoh: If you have the time to address it, I rebased and left a question on https://review.openstack.org/#/c/449257/

Earlier   Later