Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-24
21:53:50 sean-k-mooney it seam like the image just got truncated but i dont know why that would happen
21:55:13 spatel http://paste.openstack.org/show/732990/
21:55:19 spatel this is destination compute node logs
21:57:00 spatel I am using swap disk it shouldn't be an issue
21:57:33 sean-k-mooney swap will be copied also so ya it shoudl be fine
21:57:48 openstackgerrit Vladyslav Drok proposed openstack/nova stable/pike: Fix resize_instance rpcapi call https://review.openstack.org/603439
21:58:07 spatel let me try again and see
21:59:01 sean-k-mooney this is a little suspicious Unknown base file: /var/lib/nova/instances/_base/68e4d13dacff5cffeaacecf533afab659ec3e170
22:00:11 spatel hmm!
22:00:21 spatel let me try again can capture fresh log
22:00:49 spatel i have spun up new vm and it has 98Mdisk file
22:00:57 openstackgerrit Matt Riedemann proposed openstack/nova master: Migrate old style volume attachments on nova-compute startup https://review.openstack.org/549130
22:01:52 spatel This is what i am doing for migration Horizon > live migration > select block migration
22:03:35 sean-k-mooney any luck reporducing?
22:03:53 sean-k-mooney by the way im assuming you are not using sriov interface on that vm right
22:04:00 spatel no SR-IOV
22:04:20 sean-k-mooney ok good because that does not wrok :)
22:04:25 sean-k-mooney at least not yet
22:04:31 spatel with in 30 second nova said migration completed and i can see host also updated in horizon
22:05:04 spatel here is the fresh log http://paste.openstack.org/show/732991/
22:05:12 spatel now going to reboot my instance
22:06:23 spatel failed to boot
22:06:29 spatel Entering emergency mode. Exit the shell to continue.
22:06:54 sean-k-mooney hum if you log in to host ostack-compute-27.v1v0x.net does its n-cpu log have any errors
22:06:56 spatel disk size is 3.3M
22:07:07 spatel n-cpu log?
22:07:12 spatel is that a log file?
22:07:19 sean-k-mooney nova compute agent
22:07:56 sean-k-mooney in the developement installer devstack its refered to as n-cpu for short
22:08:16 sean-k-mooney so same log but for source node
22:08:41 spatel This is the log of compute-27 http://paste.openstack.org/show/732992/
22:09:04 spatel no error there
22:09:27 spatel what is this ? [instance: 2ad8cdf5-4db7-4024-a35f-343340ad27ee] Instance not resizing, skipping migration.
22:11:06 sean-k-mooney we use the same code path for migration and resizing vms since it basically the same thing
22:11:31 spatel hmm
22:12:45 sean-k-mooney its being logged from here https://github.com/openstack/nova/blob/12fcfc5e2b51b563529a5bc0b2990816bbbda80b/nova/compute/resource_tracker.py#L1127
22:12:45 spatel Do you think my nova config is wrong?
22:13:16 sean-k-mooney no i think your nova config is fine that said we might need to enable dbug loggin to get to the bottom of what is going on
22:13:47 spatel So i have manually copy disk file from source to destination and VM is up
22:14:33 spatel look like compute node not not copying full disk file or source just deleting instance before copy finish
22:14:36 openstackgerrit Merged openstack/nova master: Add debug logs for when provider inventory changes https://review.openstack.org/597560
22:14:44 sean-k-mooney ya that should not be required
22:14:57 sean-k-mooney ya can you enable debug logs on both nodes and try again
22:15:13 spatel Let me go home and collect all nova debug and file BUG
22:15:40 spatel it seems source just deleting instance before nova migration complete...
22:16:18 sean-k-mooney spatel: sure its late here too so i was going to be droping ofline soon once my pizzia is readdy
22:16:49 sean-k-mooney if you can file a bug and attach some debug logs for the source and destination we can see why it is not copying the disk
22:17:45 spatel i will
22:17:48 spatel thanks!
22:18:06 sean-k-mooney spatel: by the way what release are you running
22:18:36 spatel queens
22:18:50 sean-k-mooney there was a error we caught just as Rocky was shipping where we treated migration compution as a success without checking if there was an error
22:18:59 sean-k-mooney ah so ya you might be hitting that bug
22:19:09 spatel damn it!! !
22:19:13 spatel really!!
22:19:29 sean-k-mooney basically if libvirt through an error after migrtion started we did not handel it properly
22:19:29 spatel send me that BUG and i will check code
22:20:40 spatel I am leaving and need to shutdown my pc but you can send me email on satish.txt@gmail.com
22:35:35 openstackgerrit Merged openstack/nova stable/pike: Revert "Make host_aggregate_map dictionary case-insensitive" https://review.openstack.org/605268
22:35:41 openstackgerrit Merged openstack/nova stable/pike: Enforce case-sensitive hostnames in aggregate host add https://review.openstack.org/605269
#openstack-nova - 2018-10-25
00:18:29 openstackgerrit melanie witt proposed openstack/nova stable/rocky: Add rolling upgrade info to enable_consoleauth workaround reno https://review.openstack.org/613174
00:54:18 openstackgerrit Tao Li proposed openstack/python-novaclient master: Deprecate the unused instance-name https://review.openstack.org/602520
01:11:46 openstackgerrit melanie witt proposed openstack/nova master: Add nova-status upgrade check for consoles https://review.openstack.org/611214
02:51:04 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add a hacking rule for deprecated assertion methods https://review.openstack.org/613184
03:03:37 openstackgerrit Fan Zhang proposed openstack/nova master: Retry after hitting libvirt error VIR_ERR_OPERATION_INVALID in live migration. https://review.openstack.org/612272
04:29:29 openstackgerrit Merged openstack/python-novaclient master: Deprecate the unused instance-name https://review.openstack.org/602520
05:38:09 openstackgerrit yenai proposed openstack/nova master: Delete redundant spaces https://review.openstack.org/613199
07:40:22 openstackgerrit Takashi NATSUME proposed openstack/nova master: api-ref: Remove unnecessary minimum microversion https://review.openstack.org/613226
07:46:31 openstackgerrit Takashi NATSUME proposed openstack/nova master: api-ref: Add descriptions of error cases https://review.openstack.org/610429
07:46:49 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add API ref guideline for examples https://review.openstack.org/604060
07:47:03 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add API ref guideline for body text https://review.openstack.org/605628
08:36:55 openstackgerrit Tetsuro Nakamura proposed openstack/nova-specs master: Spec: Support filtering by forbidden aggregate https://review.openstack.org/603352
08:38:27 openstackgerrit Seyeong Kim proposed openstack/nova master: Enable connection_info refresh for new-style attachments https://review.openstack.org/579004
09:37:34 pooja_jadhav hi team, I facing issue on nova-compute : (1054, u"Unknown column 'block_device_mapping.volume_type' in 'field list'"). For that I have did nova-manage db sync and api_db sync as well. still facing same issue. Can anyone help me to resolve this issue?
10:09:15 openstackgerrit Jan Gutter proposed openstack/os-vif master: Extend port profiles with datapath offload type https://review.openstack.org/572081
10:48:08 openstackgerrit Radoslav Gerganov proposed openstack/nova stable/pike: Don't persist zero allocation ratios in ResourceTracker https://review.openstack.org/613263
10:50:46 openstackgerrit Radoslav Gerganov proposed openstack/nova stable/pike: Update resources once in update_available_resource https://review.openstack.org/612295
11:02:03 openstackgerrit Radoslav Gerganov proposed openstack/nova stable/queens: Don't persist zero allocation ratios in ResourceTracker https://review.openstack.org/613271
11:04:32 openstackgerrit Radoslav Gerganov proposed openstack/nova stable/queens: Update resources once in update_available_resource https://review.openstack.org/612294
12:21:52 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: Consider root id is None in the database case https://review.openstack.org/613305
12:21:52 openstack bug 1799892 in OpenStack Compute (nova) "Placement API crashes with 500s in Rocky upgrade with downed compute nodes" [Undecided,New] https://launchpad.net/bugs/1799892
12:21:52 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: Add recreate test for bug 1799892 https://review.openstack.org/613304
12:34:54 cdent mnaser gets the gold star award for finding good bugs
12:48:18 openstackgerrit Jan Gutter proposed openstack/os-vif master: Extend port profiles with datapath offload type https://review.openstack.org/572081
13:15:08 openstackgerrit Adam Spiers proposed openstack/nova-specs master: Add spec for libvirt driver launching AMD SEV-encrypted instances https://review.openstack.org/609779
13:22:54 openstackgerrit Merged openstack/nova master: Remove the CachingScheduler https://review.openstack.org/611723
13:24:51 spatel sean-k-mooney: morning
13:25:13 sean-k-mooney spatel: hi o/
13:25:13 spatel I did debug and here is the source machine log http://paste.openstack.org/show/733042/
13:26:45 spatel There are some ERROR
13:27:09 spatel something like this - MessagingTimeout: Timed out waiting for a reply to message ID 80fcdfa3d90d4c128284822024982706
13:27:19 sean-k-mooney the messaging timeout is strange but may just meean the compute node was not back up after your config change
13:27:49 spatel Let me tell you what happened...
13:27:53 spatel 1. build foo instance
13:28:06 spatel 2. did block migration - and it failed with error
13:28:28 spatel 3. tried again and succeeded but vm failed to boot on destination compute node
13:29:23 spatel if you see my log i have break them in two section first one is for first attempt and second block is retried
13:29:55 sean-k-mooney right
13:30:20 sean-k-mooney the second migration looks ok bu i take it you had the same truncated disk on the dest
13:30:44 spatel yes it had only 1MB disk on destination, it didn't copy full disk

Earlier   Later