Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-12-12
22:08:34 Kevin_Zheng mriedem: just shelve the vm and all the volume will be in reserved status and cannot be detached
22:08:36 mriedem yeah that's normal
22:08:37 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Handle tags in _bury_in_cell0 https://review.openstack.org/624813
22:08:40 mriedem b/c there is no image ref if you bfv
22:08:59 mriedem Kevin_Zheng: we have a tempest test that does exactly that
22:09:31 melwitt yeah, I am so rusty on this. that's embarrassing
22:09:39 Kevin_Zheng I didn’t check yet but that’s the result I got
22:09:51 mriedem good thing i don't have a twitter account to let everyone know about this
22:09:52 Kevin_Zheng And also the code does it
22:09:56 melwitt I was thinking that sentence was the fault field, but the fault field wasn't even showing because I wasn't authed as admin. lots of good stuff going on here
22:10:20 melwitt haha yup
22:10:40 mriedem Kevin_Zheng: is this vanilla devstack or something with your detach boot volume changes applied?
22:10:47 melwitt failure to prep block device, which is probably my conf limit code firing
22:11:12 mriedem melwitt: yes probably, and we don't reschedule on failures from _prep_block_device i don't think
22:11:19 mriedem so we just crap out
22:11:46 brandor5 mriedem: i found the problem, the resize call was firing too fast... I put a "sleep" in for a few seconds and it worked
22:11:51 Kevin_Zheng DevStack
22:12:27 mriedem brandor5: hmm, that's weird, because as soon as the server status is VERIFY_RESIZE is should be ok to call confirmResize
22:12:35 mriedem especially because i'm sure tempest does exactly the same thing,
22:12:42 mriedem poll for VERIFY_RESIZE and then immediately confirm
22:13:02 brandor5 yeah, that's what I'm doing
22:13:12 brandor5 if I sleep for 3 seconds after that it works
22:13:46 Kevin_Zheng I did not change the shelve flow, I only change the detach attach flow and it will not affect as it was blocked in the very early stage calling begin-detaching
22:13:57 brandor5 unless my polling isn't working the way I think
22:14:02 brandor5 I'll double check that really quick
22:14:57 mriedem this is where nova-compute changes the status https://github.com/openstack/nova/blob/newton-eol/nova/compute/manager.py#L3943
22:15:16 mriedem this is the api code for the status field https://github.com/openstack/nova/blob/newton-eol/nova/api/openstack/common.py#L79
22:15:32 mriedem so once compute does the update to the db, and the api returns VERIFY_RESIZE is should be ok to confirm
22:15:52 mriedem Kevin_Zheng: did you look at the tempest test i pointed out?
22:16:40 Kevin_Zheng Not yet, I will check it today
22:16:52 mriedem good morning btw
22:17:03 mriedem 6am what are you doing
22:17:35 Kevin_Zheng My kid fall down the bed so everyone is up:)
22:18:35 mriedem ha
22:18:39 mriedem time for guard rails
22:18:46 mriedem or strapts
22:18:47 mriedem *straps
22:18:54 brandor5 mriedem: found the problem... it was an error on my part... if you look at the paste I linked the register,until, retries, and delay options were indented incorrectly and not being honored....... so it was just running the cehck once and then moving on through
22:19:26 mriedem brandor5: ah ok
22:19:33 mriedem i knew it couldn't be nova's fault, nova is never at fault
22:19:38 brandor5 hehe
22:19:44 Kevin_Zheng :) go back to sleep now
22:19:45 brandor5 it's dns
22:19:45 mriedem it probably worked in the cli just b/c of a time delay
22:19:51 brandor5 or the network
22:19:53 mriedem building the client and such
22:19:54 mriedem or that
22:19:54 brandor5 those are my favorits :D
22:21:20 openstackgerrit Matt Riedemann proposed openstack/nova stable/queens: Handle tags in _bury_in_cell0 https://review.openstack.org/624824
22:22:39 melwitt confirmed, it was my limit code. so it does work for bfv. and it didn't create a new volume before it failed. I will test it more, maybe it's too good to be true
22:22:57 mriedem we should get that fault message cleaned up to actually have the error message in it
22:23:02 melwitt aye
22:23:22 mriedem could have sworn i did something in that code for that, but might have been logging
22:24:02 mriedem yeah https://review.openstack.org/#/q/I20a5e8e5e10dd505c1b24c208f919c6550e9d1a4
22:25:02 mriedem maybe "Failure prepping block device" is just your error message
22:25:50 melwitt indeed in the log: Failure prepping block device: TooManyDiskDevices: The maximum allowed number of disk devices to attach to a single instance has bee
22:25:50 melwitt n exceeded
22:26:08 melwitt le sigh, newline
22:36:55 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Handle tags in _bury_in_cell0 https://review.openstack.org/624825
22:48:13 openstackgerrit Matt Riedemann proposed openstack/nova master: Replace ThreadPoolExecutor with GreenThreadPoolExecutor https://review.openstack.org/624740
22:48:14 openstackgerrit Matt Riedemann proposed openstack/nova master: Add python 3.7 unit and functional tox jobs https://review.openstack.org/624055
22:48:15 openstackgerrit Matt Riedemann proposed openstack/nova master: Add py36/py37 functional jobs to the experimental queue https://review.openstack.org/624826
22:48:23 melwitt found that the error message gets squashed here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2341-L2343 and that's what ends up in the instance fault
22:49:39 melwitt feels like a separate patch for improving the information there, I can try it and see how it looks, going to be a much longer string
23:15:53 openstackgerrit Matt Riedemann proposed openstack/os-resource-classes master: Sync STANDARDS with placement/nova rc_fields https://review.openstack.org/624370
23:15:54 openstackgerrit Matt Riedemann proposed openstack/os-resource-classes master: Tune up documentation to be more useful https://review.openstack.org/624384
23:21:44 mriedem melwitt: oh ok, we should just do similar in https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2341 then and probably include the underlying reason
23:21:54 mriedem also, duh duh duh nova-lvm job is busted (privsep) http://logs.openstack.org/43/619143/12/check/nova-lvm/786180c/logs/screen-n-cpu.txt.gz?level=TRACE#_Dec_12_12_35_39_607002
23:22:24 melwitt mriedem: yeah, doing it now. figured I should write up a bug too
23:23:57 mriedem https://bugs.launchpad.net/nova/+bug/1808247
23:23:57 openstack Launchpad bug 1808247 in OpenStack Compute (nova) "nova-lvm job failing with ArgsAlreadyParsedError (privsep)" [Medium,Confirmed]
23:24:02 mriedem if only mikal were on irc
23:28:49 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/os-vif master: Create iproute.IPRoute() inside a context https://review.openstack.org/624831
23:29:05 openstackgerrit Merged openstack/nova master: Make compute rpcapi version calculation check all cells https://review.openstack.org/623284
23:29:16 openstackgerrit Merged openstack/nova master: Remove allocations before setting vm_status to SHELVED_OFFLOADED https://review.openstack.org/623596
23:31:11 openstackgerrit melanie witt proposed openstack/nova master: Propagate exception message from _prep_block_device https://review.openstack.org/624832
23:33:57 melwitt ah, I guess I need a test to show it going to the fault message. or maybe just that the message from _prep_block_device gets included
23:41:00 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/os-vif master: Create iproute.IPRoute() inside a context https://review.openstack.org/624831
#openstack-nova - 2018-12-13
00:01:19 openstackgerrit melanie witt proposed openstack/nova master: Propagate exception message from _prep_block_device https://review.openstack.org/624832
00:16:50 openstackgerrit melanie witt proposed openstack/nova master: Add method to generate device names universally https://review.openstack.org/573066
00:16:50 openstackgerrit melanie witt proposed openstack/nova master: WIP Add configuration of maximum disk devices to attach https://review.openstack.org/616777
00:20:23 openstackgerrit Merged openstack/nova master: refactor get_console_output() for console logfiles https://review.openstack.org/575735
01:18:38 openstackgerrit sean mooney proposed openstack/nova master: Libvirt: do not set mac when unplugging macvtap vf https://review.openstack.org/624842
02:06:39 openstackgerrit Brin Zhang proposed openstack/nova-specs master: Support admin to specify project to create snapshot https://review.openstack.org/616843
03:27:36 openstackgerrit zhangyujun proposed openstack/nova master: ESFIX: Should not raise while restore power on failed https://review.openstack.org/624854
03:30:36 openstackgerrit zhangyujun proposed openstack/nova master: Should not raise while restore power on failed https://review.openstack.org/624854
07:35:55 openstackgerrit Andrey Volkov proposed openstack/nova master: Exclude build request instance with marker from listing https://review.openstack.org/624870
07:50:37 openstackgerrit Takashi NATSUME proposed openstack/python-novaclient master: Fix flavor keyerror when nova boot vm https://review.openstack.org/582147
08:11:28 openstackgerrit Takashi NATSUME proposed openstack/os-resource-classes master: Update author-email in setup.cfg https://review.openstack.org/624885
10:45:25 openstackgerrit Merged openstack/nova stable/rocky: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/624677
11:07:53 frickler lyarwood: could I ask you to place https://review.openstack.org/619254 onto your review list? that's the last one in my stable backport series, all the others got merged
11:11:31 lyarwood frickler: ack
11:12:15 openstackgerrit Merged openstack/os-resource-classes master: Initial automated testing for os-resource-classes https://review.openstack.org/624366
11:12:16 openstackgerrit Merged openstack/os-resource-classes master: Rename ORDERED_CLASSES to STANDARDS https://review.openstack.org/624369
11:12:17 openstackgerrit Merged openstack/os-resource-classes master: Sync STANDARDS with placement/nova rc_fields https://review.openstack.org/624370
11:18:50 openstackgerrit Tom Barron proposed openstack/nova stable/rocky: Do Not Merge, testing libvirt-python requirements https://review.openstack.org/624958
11:33:55 openstackgerrit Zhenyu Zheng proposed openstack/nova master: WIP don't call begin_detaching when detaching volume from sheved vm https://review.openstack.org/624959
11:49:30 openstackgerrit Chris Dent proposed openstack/os-resource-classes master: Tune up documentation to be more useful https://review.openstack.org/624384
12:14:01 cdent stephenfin: see my comment on https://review.openstack.org/624826 for something that might be of interest to you. Also I fixed the problems you identified in ^
12:14:58 stephenfin cdent: Hmm, the cookiecutter? Weird. Maybe something's changed that I wasn't aware of

Earlier   Later