Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-26
12:29:03 sean-k-mooney why is jsonutils.dumps being used instead of to_primitive?
12:29:55 openstackgerrit Marek Lyčka proposed openstack/nova-specs master: Adds spec infrastructure for Victoria https://review.opendev.org/710023
12:30:36 sean-k-mooney oh i guess you are trying to convert it to a dictionay
12:30:56 sean-k-mooney by round tripping the data through json
12:31:01 gibi sean-k-mooney: the legacy_spec is a dict form of a RequestSpec object
12:31:12 sean-k-mooney ya so we cant just use obj_to_primitive
12:31:15 gibi but it is a special dict form
12:31:22 sean-k-mooney becasue that wont give us what we want
12:32:09 gibi it would give us a different dict
12:32:44 sean-k-mooney yep so it wont work here
12:32:59 sean-k-mooney why do we need to use legacy requests specs in this case?
12:33:38 sean-k-mooney i taught we had already moved to ovo form in rocky but i guess not fully
12:36:32 sean-k-mooney oh its becasue of the workaround for bug #1529084
12:36:32 openstack bug 1529084 in oslo.messaging "RPC fake driver should accept datetime items for data" [Undecided,Fix released] https://launchpad.net/bugs/1529084 - Assigned to Balazs Gibizer (balazs-gibizer)
12:37:19 sean-k-mooney that intoduced the seriasiation and deserialisation but missed the fact it wont recurse to the nested ovos which is what you are fixing
12:39:03 sean-k-mooney another fix would be to fix the rpc fake dirver to accept datimes in dicst and remvoe the jsonutils calls entirely
12:39:47 gibi sean-k-mooney: since stein we doesn't use the legacy dict but pass around the ovo, but that is an RPC change it is not backportable
12:40:33 gibi sean-k-mooney: but yeah, we we can somehow remove the need of the json.loads(json.dumps(..) stuff that would also help
12:41:09 sean-k-mooney well the comment says its done beacase teh fake rpc driver does not support datetimes in dicts
12:41:16 sean-k-mooney so if we fix that then we could remove it right
12:41:31 sean-k-mooney i assume there was a reason bauzas didnt do that orgininally
12:41:43 sean-k-mooney other then this was faster
12:42:35 gibi I'm not sure what will happen if the dump an load is removed and then the rpc will try to serialize the dict again with ovos inside
12:42:48 sean-k-mooney gibi: anway i have not checked that setting default=... will work but your change looks resonable to me
12:43:43 gibi sean-k-mooney: the solution is basically change how the to_primitive works during dumps to trigger the ovo serialization instead of stuck in an infinite loop
12:44:09 sean-k-mooney ya i was assuming "default=jsonutils.to_primitive" if not set
12:44:14 gibi I also filed a bug to oslo.serialization https://bugs.launchpad.net/oslo.serialization/+bug/1864678
12:44:14 openstack Launchpad bug 1864678 in oslo.serialization "jsonutils.to_primitive does not follow the protocol required by json.dump" [Medium,Triaged]
12:44:24 sean-k-mooney so you are just passing an extra argment to it with functools.partial
12:44:32 gibi sean-k-mooney: yeas, that is the default by default (sic)
12:44:38 gibi sean-k-mooney: yes
12:45:18 sean-k-mooney yep so that all makes sense to me. its not the cleanest thing long term but its the minimal backportable change which is more important
12:46:03 gibi I'm not sure that the fix of https://bugs.launchpad.net/oslo.messaging/+bug/1529084 can be backported to pike
12:46:03 openstack Launchpad bug 1529084 in oslo.messaging "RPC fake driver should accept datetime items for data" [Undecided,Fix released] - Assigned to Balazs Gibizer (balazs-gibizer)
12:46:35 gibi and the the we can bump oslo.messaging version on the stable branches
12:47:03 gibi then remove the loads(dumps(..)) part from nova, and backport it to pike
12:47:05 sean-k-mooney you mean to rocky
12:47:17 gibi sean-k-mooney: rocky is the first stable that is affected
12:47:25 gibi I have to backport the whole thing to pike
12:47:28 sean-k-mooney ah you have other cherry picks
12:47:32 gibi as we have a failure downstream
12:47:40 gibi in pike
12:47:47 sean-k-mooney gotch ya
12:48:16 gibi so my target is to fix pike, and sure I will fix every broken stable along the way but I wan't to do this with minimal change
12:48:31 gibi a new oslo.messaging version would be a lot harder business
12:48:32 sean-k-mooney not a core but i +1'd it anyway since it makes sense after our disucsstion stephenfin its pretty quick if you can take a look
12:48:42 gibi sean-k-mooney: thanks!
12:53:25 sean-k-mooney hehe i some how dont think the intel pmem ci is going to work on rocky
12:53:40 sean-k-mooney they might want to fix that at some point
12:55:27 brinzhang gmann: is that true of bug 1864776?
12:55:27 openstack bug 1864776 in OpenStack Compute (nova) "os-volumes-attachments API policy is allowed for everyone even policy defaults is admin_or_owner" [Undecided,In progress] https://launchpad.net/bugs/1864776 - Assigned to Brin Zhang (zhangbailin)
13:01:43 brinzhang gmann: I was pushed the fixed patch https://review.opendev.org/#/c/709955/, and rebase this patch I was tested https://review.opendev.org/#/c/709929/1/nova/tests/unit/policies/test_volumes.py, it also need everyone contexts to authorize the admin_or_owner role
13:02:13 brinzhang gmann: I am not sure this bug is true, while you are free, pls review, thanks
13:48:21 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: Don't print user_data for 'nova show' https://review.opendev.org/708850
13:50:17 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Add framework for extra spec validation https://review.opendev.org/704643
13:50:18 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Add microversion 2.82, extra spec validation https://review.opendev.org/708436
13:50:18 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add documentation for flavor extra specs https://review.opendev.org/710037
13:54:48 openstackgerrit Balazs Gibizer proposed openstack/nova master: WIP: Hey let's support routed networks y'all! https://review.opendev.org/656885
13:55:55 brinzhang__ stephenfin: do you have time the check this bug? That prevent my next work https://bugs.launchpad.net/nova/+bug/1864776
13:55:55 openstack Launchpad bug 1864776 in OpenStack Compute (nova) "os-volumes-attachments API policy is allowed for everyone even policy defaults is admin_or_owner" [Undecided,In progress] - Assigned to Brin Zhang (zhangbailin)
13:56:21 brinzhang__ that's the default policy refresh patch
13:57:50 brinzhang__ stephenfin: if it's true, I will rebase others on the fixed patch, otherwise I will abandon this fix, and push new patc to go continue
14:00:12 openstackgerrit Lee Yarwood proposed openstack/nova master: WIP libvirt: Reintroduce volume based LM tests https://review.opendev.org/536105
14:05:21 canori01 hey guys, is it possible to update the root disk for an in-use flavor in the database (such that new instances using that flavor take the change)or would that break things?
14:09:54 brinzhang__ canori01: Which things would you like to change of the root disk?
14:11:29 brinzhang__ canori01:It is not recommended to change the configuration of the root disk directly in db.
14:12:16 brinzhang__ May cause strange things due to incomplete modification.
14:21:26 canori01 brinzhang: Only thing I want to change is the root disk size from 0 to something like 20 or 50G
14:33:09 openstackgerrit Stephen Finucane proposed openstack/nova master: Unplug VIFs as part of cleanup of networks https://review.opendev.org/663382
14:33:09 openstackgerrit Stephen Finucane proposed openstack/nova master: Fix incorrect vm and task state after build failure race https://review.opendev.org/689388
14:50:42 openstackgerrit Lee Yarwood proposed openstack/nova master: api: Introduce microverion 2.82 allowing boot from volume rescue https://review.opendev.org/701430
14:50:43 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Extract _get_bdm_image_metadata into nova.utils https://review.opendev.org/705212
14:50:43 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Support boot from volume stable device instance rescue https://review.opendev.org/701431
15:01:08 openstackgerrit Lee Yarwood proposed openstack/nova master: DNM - Test stable device rescue tests with BFV instances https://review.opendev.org/710050
16:06:35 efried sean-k-mooney: Re token expiring on that cyborg job: is the conf set up with a service token?
16:12:22 sean-k-mooney i am not sure but the confs are in the job logs
16:12:28 sean-k-mooney it should not fail in either case
16:12:47 sean-k-mooney once it start happening you have to restart all the nova services to fix it
16:22:50 sean-k-mooney efried: so no https://48ef08cde8cc22034a1d-8011a2266d21f0c09baf1c83d6d5002e.ssl.cf5.rackcdn.com/709641/5/check/cyborg-multinode-tempest-full/e4d260f/controller/logs/etc/nova/nova_conf.txt
16:23:12 sean-k-mooney the service user section https://docs.openstack.org/nova/latest/configuration/config.html#service-user
16:24:14 sean-k-mooney is not configured but it should not need to be confitured for it to work
16:24:44 sean-k-mooney it might allow you to mask the issue
16:27:35 efried sean-k-mooney: so the failure is happening from a later operation, not from the middle of a long-running operation?
16:46:05 gmann brinzhang__: sure, sorry for late response. I will review your patches today
16:50:40 efried sean-k-mooney: can I get your nod on https://review.opendev.org/#/c/709902/ (rocky EM patch) for os-vif please?
17:06:32 efried lyarwood: https://review.opendev.org/#/c/709902/1/deliverables/rocky/nova.yaml lgty?
17:12:08 lyarwood efried: ack yes, elod ^?
17:33:03 sean-k-mooney efried: it can start failing with a 401 in the middel of an operation and once it has it will continue to fail for seperate operations
17:33:41 sean-k-mooney and yes ill look at the em patch now
17:35:28 sean-k-mooney there is one pending bugfix i want to back port to all affected branch in os-vif but rocky predates the issue so yes that commit looks correct
17:35:32 sean-k-mooney ill +1 the review
17:38:23 sean-k-mooney efried: basicaly the way i first hit the cybrog issue was i booted a vm. did a bunch of life cycle operation on it and then tried to delete it and that failed befaue the token was rejected.
17:38:36 sean-k-mooney when i tried to do the operation (deleting the arq) myself it worked
17:39:25 sean-k-mooney if i listed device profiles myslef it also worked but when i tried to boot anotuher vm after that point it failed beacuse the nova api was not able to retive the device profile info
17:40:44 sean-k-mooney so if i use osc to query cyborg directly everthing is fine. if i use osc to boot a vm and nova tires to query cyborg on my behalf it was failing but only after the services had been running for a while like an hour or so
18:24:17 lucidguy Anyone recall me asking for assistance with >1tb ram instances? I FIGURED IT OUT!!! only took days.
18:25:23 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Provide the backing file format when creating qcow2 disks https://review.opendev.org/708745
18:27:10 lyarwood lucidguy: what was it?
18:27:11 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Provide the backing file format when creating qcow2 disks https://review.opendev.org/708745
18:32:44 sean-k-mooney updating other people code when you have never reviewed it and are just following gerrit comments is hard

Earlier   Later