Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-03-12
13:42:10 lyarwood kashyap: thanks
13:43:11 gibi masterpe: yeah so the evacuation was successful. I have no further idea how https://github.com/openstack/nova/blob/stable/rocky/nova/compute/manager.py#L729 is not triggered.
13:44:20 gibi masterpe: if you find the offending allocations against compute2 in allocations table of the nova_api db then you can deleted them manually. But I have no idea how to prevent them to happen
13:44:22 dansmith brinzhang: so, the thing I don't get about what he said is the "pass a format_exc_val kwarg" which isn't really an option here
13:44:48 masterpe I can not find that code in the file
13:45:04 dansmith brinzhang: but we can make serialize_args pass the cleaned-up exception message-or-type thing for exc_val
13:48:28 masterpe gibi: In my manager.py the commit https://opendev.org/openstack/nova/commit/09de94e39bbcfc7f8130638e73a8248e49cb6ab7 is missing, so probebly the file is from before that.
13:50:05 brinzhang dansmith: what do you mean? Can you elaborate? I still don't quite understand. In other words, I don't know what I should do about @serialize_args
13:50:14 gibi masterpe: ahh. you have 18.2.3 and that patch is in 18.3
13:50:17 dansmith brinzhang: I'm cooking up a patch for you
13:50:23 gibi masterpe: that explains what you see
13:50:44 brinzhang dansmith: thanks
13:51:28 masterpe I see 4 enty's
13:51:28 masterpe when I do select * from allocations where consumer_id = 'b4a3a998-e8b4-482c-ae4d-dddaee707ff1';
13:51:45 masterpe the collum resource_provider_id is the compute node?
13:52:35 gibi masterpe: yes, the resource_provider_id is the compute node uuid
13:52:55 gibi masterpe: most probably you have allocation for that instance on two computes, compute2 and compute3
13:53:13 gibi the bugfix that is missing from your rocky version is fixing that
13:53:37 gibi if you could upgrade to 18.3 and restart the compute that would also fix your deployment as well
14:03:22 dansmith gibi: fwiw, normally I can make the 1600 meeting but not today because I already have a thing I have to do
14:03:31 dansmith er, s/fwiw/fyi/
14:03:43 gibi dansmith: ack, thanks for the heads up
14:04:23 masterpe The id in resource_providers is linked with resource_provider_id of allocations
14:05:15 gibi masterpe: yes sorry, the resource_provider_id is an internal id pointing to the resource_providers table and there is an uuid in that table that points to the compute node
14:05:49 masterpe resource_provider_id has id 9
14:06:24 masterpe so I can delete all records in allocations with resource_provider_id = 9?
14:08:51 gibi masterpe: if you evacuated all the servers from compute2 then yes
14:09:37 masterpe yes compute2 is emty
14:23:05 dSrinivas Hi all, while attaching the volume to the Nova intance it is failing with this errorhttp://paste.openstack.org/show/790596/ can anyone please me
14:25:11 dSrinivas For FC not for iscsi
14:31:18 sean-k-mooney so that look like an os_brick issue but have you check if the FC deivce is mounted properly on the host at the expected path
14:31:54 dSrinivas FC devices are Online
14:32:23 dSrinivas It is attaching the volume and again it is deleting after 20seconds
14:32:45 sean-k-mooney and if you run the grep command yourself does it find the deivice
14:32:51 dSrinivas Cinder volume logs it is showing attachment updated successful
14:32:58 sean-k-mooney hum well the deleing it proably part of the clean up
14:33:47 dSrinivas Systool -c fc_host -v | grep Online it is showing the ports as online
14:34:04 sean-k-mooney on the storage backend
14:34:32 sean-k-mooney that does not mean that the compute node was able to connect to it and mount it right?
14:34:46 sean-k-mooney it just means the sotrage backend thing it available to be mounted?
14:34:51 dSrinivas At storage side it is showing attached again after 20se
14:34:52 sean-k-mooney FC is not my thing
14:35:00 lyarwood dSrinivas: which version is this?
14:35:01 dSrinivas Deleting
14:35:07 dSrinivas Master
14:35:49 openstackgerrit Dan Smith proposed openstack/nova master: Make serialize_args handle exception messages safely https://review.opendev.org/712697
14:35:52 lyarwood yeah sorry no idea then, can you raise this in an os-brick bug and ask for help in #openstack-cinder ?
14:35:53 sean-k-mooney dSrinivas: the only thing that comes to mind is maybe it is timing out waithg for it to be avialble
14:35:56 dansmith brinzhang: mriedem ^
14:36:27 mriedem ooo
14:36:31 dSrinivas How to increase the retries
14:38:04 sean-k-mooney dSrinivas: im not sure.
14:38:29 dSrinivas It is attaching and then deleting immediately
14:38:54 sean-k-mooney and you can see that on the compute node not the storage backend
14:39:54 lyarwood dSrinivas: ignore the volume state, the underlying block devices aren't present.
14:40:18 lyarwood dSrinivas: raise an os-brick bug for this and reach out to the folks in #openstack-cinder
14:40:31 lyarwood dSrinivas: we don't directly support the os-brick code in this channel
14:40:34 openstackgerrit sean mooney proposed openstack/nova master: Provider Config File: Functions to merge provider configs to provider tree https://review.opendev.org/676522
14:41:05 dSrinivas I have check the n-cpu logs and pasted the link above
14:41:40 lyarwood dSrinivas: yes, n-cpu is calling os-brick and it is failing to connect the volume here
14:41:44 sean-k-mooney dSrinivas: which is showing an error form os-bricks which is a lib that is maintained by both cinder and nova
14:42:05 lyarwood sean-k-mooney: well we chip in with the odd bugfix ;)
14:42:16 sean-k-mooney yes
14:42:42 sean-k-mooney well you chip in with the odd bugfix i avoid it
14:44:17 openstackgerrit sean mooney proposed openstack/nova master: Provider Config File: Enable loading and merging of provider configs https://review.opendev.org/693460
14:44:53 kplant anyone know what would cause "No valid host was found. No valid host found for resize " when trying to resize an instance?
14:45:05 kplant i have plenty of resources available and can deploy new instances with the same flavor i'm trying to resize to
14:45:36 mriedem dansmith: left some comments, thanks
14:47:02 dansmith mriedem: thanks
14:50:09 openstackgerrit Takashi NATSUME proposed openstack/python-novaclient master: Microversion 2.82 - nova cyborg interaction https://review.opendev.org/712700
14:51:08 jkulik kplant, if you only have one host, you'd explicitly have to allow resize on the same host, iirc
14:51:22 kplant i did see that, i have more than one host though
14:51:44 jkulik then you'll have to look into the logs of nova-scheduler I guess
14:51:54 kplant unfortunately it gives me the same vague error message
14:53:23 jkulik enabling DEBUG logging always helps me understanding why the scheduler couldn't find an appropriate host. you need to search the logs for the request-id, not the instance uuid though.
14:53:34 kplant aah debug is a good idea
14:53:36 kplant ty
14:57:59 openstackgerrit sean mooney proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341
14:58:00 openstackgerrit sean mooney proposed openstack/nova master: Provider Config File: Functions to merge provider configs to provider tree https://review.opendev.org/676522
14:58:00 openstackgerrit sean mooney proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029
14:58:01 openstackgerrit sean mooney proposed openstack/nova master: Provider Config File: Enable loading and merging of provider configs https://review.opendev.org/693460
14:58:20 sean-k-mooney ^ that is just a clean rebase against master to allo the test to be run locally
14:58:37 sean-k-mooney it was complaing about moxs stubs
15:06:42 mriedem smcginnis: i hope my characterization of you being fluent in executive is accurate https://review.opendev.org/#/c/712498/
15:14:42 iurygregory Hi nova team, is this a know issue in the grenade tests http://paste.openstack.org/show/790610/ we started to see this in the ironic-grenade job while running the Smoke Tests http://zuul.openstack.org/build/3b20be2558c044628361f34eae06d520
15:15:01 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Update limit APIs https://review.opendev.org/712707
15:15:06 iurygregory ops not with grenade tests/ smoke tests =)
15:18:53 lyarwood iurygregory: http://status.openstack.org/elastic-recheck/#1844929 - I can't tell if it's the same issue but I've seen this recently in the grenade job
15:21:42 prometheanfire I think the mock update is causing some pain
15:21:46 prometheanfire https://1b8c54b5d412a3af0fef-40bd60678638a1db566d5d37b438f20d.ssl.cf1.rackcdn.com/712615/2/check/cross-nova-py36/8e15404/testr_results.html
15:21:54 prometheanfire https://review.opendev.org/712615
15:22:28 iurygregory well I see some Timed out waiting for response from cell in the logs also...
15:22:42 prometheanfire woo
15:22:52 prometheanfire fun time all around
15:23:06 iurygregory lyarwood, maybe .-. it's since the is the same message in logs
15:23:18 iurygregory but seems "random" so far in the ironic jobs
15:23:46 iurygregory some times the tests in grenade just works .-.
15:24:18 lyarwood yup that's what we are seeing with this issue
15:25:47 openstackgerrit Lee Yarwood proposed openstack/nova master: workarounds: Add option to disable native LUKSv1 decryption by QEMU https://review.opendev.org/708030
15:25:48 openstackgerrit Lee Yarwood proposed openstack/nova master: workarounds: Connect RBD volumes to the compute host as block devices https://review.opendev.org/708029

Earlier   Later