| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-01-13 | |||
| 15:19:08 | bauzas | woah | |
| 15:19:08 | dansmith | we log it even in production, tens of thousands of times | |
| 15:19:24 | dansmith | and by "we" I mean "we on behalf of o.msg" | |
| 15:19:34 | sean-k-mooney | so any reason fro me to not +2w this | |
| 15:19:35 | bauzas | oh wait | |
| 15:19:38 | sean-k-mooney | it looks ok to me | |
| 15:19:42 | bauzas | this isn't a singleton | |
| 15:19:44 | sean-k-mooney | it has the requiremtn bump | |
| 15:20:02 | dansmith | https://imgur.com/a/0Vvw1ss | |
| 15:20:02 | sean-k-mooney | ci will kick it back if it fails again on the recheck | |
| 15:20:04 | bauzas | sean-k-mooney: sent to the gate too, for the gosh sake we could merge it eventually | |
| 15:20:22 | dansmith | note the scroll bar showing all the matches | |
| 15:20:28 | bauzas | dansmith: yeah, look at https://review.opendev.org/c/openstack/nova/+/869900/6/nova/rpc.py | |
| 15:20:38 | sean-k-mooney | what was the nova-next failure | |
| 15:20:39 | dansmith | yup | |
| 15:20:42 | bauzas | everytime we call get_client, we instantiate a RPCService | |
| 15:20:50 | dansmith | yup | |
| 15:21:22 | bauzas | I would have preferred this being a singleton, but meh noxw | |
| 15:21:23 | bauzas | now | |
| 15:21:28 | dansmith | well, | |
| 15:21:35 | gmann | dansmith: bauzas dansmith : for functional test testing placement new defaults. I will switch it once placement new defaults are merged, otherwise we need to add system scope token to make placement exiting defaults - https://review.opendev.org/c/openstack/placement/+/865618 | |
| 15:21:39 | dansmith | it can't always be because of cell stuff so we need to pool | |
| 15:21:51 | bauzas | dansmith: ah true | |
| 15:22:01 | bauzas | gmann: wfm | |
| 15:23:11 | gmann | dansmith: bauzas: sean-k-mooney: as you were on rbac things, can you check this which was missed in original change. keeping legacy admin for 'os-tenant-networks' policy https://review.opendev.org/c/openstack/nova/+/865071 | |
| 15:25:04 | sean-k-mooney | oh thats projec treader of gloabl admin | |
| 15:25:15 | sean-k-mooney | where as the curren tbehavior woudl requrie you to have admin on the current project | |
| 15:26:01 | gmann | yeah keeping legacy admin unimpacted | |
| 15:26:15 | sean-k-mooney | i.e. admin implies member implies reader but the project reader will also check the project in your token matches | |
| 15:27:09 | sean-k-mooney | i guess we dont have any tempest tests covering that | |
| 15:27:37 | sean-k-mooney | or it would have blocked eitehr enablign the new default or where we broke it | |
| 15:31:54 | gmann | sean-k-mooney: this ADMIN is just role:admin not just project admin. so rule is "role:admin or project-reader" where we do not check project_id for admin role token | |
| 15:45:48 | sean-k-mooney | yep that is what i was expecting | |
| 16:47:32 | opendevreview | Sofia Enriquez proposed openstack/nova master: WIP: Implement encryption on backingStore https://review.opendev.org/c/openstack/nova/+/870012 | |
| 17:03:34 | sean-k-mooney | i guess sofia has got time to start on ^ again but that spec is not appvoed for this cycle | |
| 17:15:34 | sean-k-mooney | on a related note to the other warning it looks like we are spaming the functionl logs with an sqlachemy deprecation warning too | |
| 17:15:43 | sean-k-mooney | https://paste.opendev.org/show/bWlWeI8pWyiwWyvJ7NtC/ ^ dansmith | |
| 17:16:18 | sean-k-mooney | im not seeign a nova line there | |
| 17:16:28 | sean-k-mooney | so i guess this needs to be fixed in oslo_db | |
| 17:25:27 | dansmith | sean-k-mooney: yeah I've seen that too | |
| 17:25:40 | dansmith | it's annoying, just less annoying :) | |
| 17:26:09 | sean-k-mooney | ya os i just mentioned it on the oslo channel and look at code search | |
| 17:26:25 | sean-k-mooney | other then in some puppet code this is off everywhere | |
| 17:26:38 | sean-k-mooney | so i think oslo.db just need to drop the kwarg and do a release | |
| 17:27:05 | kashyap | Is this Oslo DB error know issue in the "nova-tox-functional-py38" job? | |
| 17:27:22 | sean-k-mooney | its just a log message | |
| 17:27:26 | sean-k-mooney | its not breakign anything | |
| 17:27:35 | kashyap | {0} nova.tests.functional.libvirt.test_vgpu.VGPUTests.test_resize_servers_with_vgpu [6.373304s] ... FAILED() | |
| 17:27:36 | sean-k-mooney | but it makes runnign the test more annrying | |
| 17:27:46 | kashyap | (From here: https://zuul.opendev.org/t/openstack/build/a229b41daba64b6f8dfdeca8c839e9f7) | |
| 17:28:35 | sean-k-mooney | thats a diffent oslo.db thing then i was talkign about | |
| 17:29:19 | sean-k-mooney | kashyap: that actully looks like a real bug | |
| 17:29:32 | kashyap | Again: it was not hit in the previous 3 runs :-( | |
| 17:29:36 | sean-k-mooney | im not sure why we woudl get a db conflict like that in a fucntional test | |
| 17:30:16 | kashyap | sean-k-mooney: But I agree - it "looks" on the surface like a real bug, but I'm not confident if it's _really_ a DB conflict, or a PEBKAC in the test or ... env snafu | |
| 17:30:41 | sean-k-mooney | so there are two tracebacks there | |
| 17:30:44 | sean-k-mooney | sqlite3.InterfaceError: Cursor needed to be reset because of commit/rollback and can no longer be fetched from. | |
| 17:30:57 | sean-k-mooney | and | |
| 17:30:59 | sean-k-mooney | Traceback (most recent call last): | |
| 17:31:01 | sean-k-mooney | File "/home/zuul/src/opendev.org/openstack/nova/.tox/functional-py38/lib/python3.8/site-packages/urllib3/connectionpool.py", line 440, in _make_request | |
| 17:31:03 | sean-k-mooney | httplib_response = conn.getresponse(buffering=True) | |
| 17:31:05 | sean-k-mooney | TypeError: getresponse() got an unexpected keyword argument 'buffering' | |
| 17:31:26 | sean-k-mooney | so it looks likethere si an issue wit urllib3 | |
| 17:31:28 | sean-k-mooney | as well | |
| 17:31:31 | kashyap | Right, the first one is the cause of the 2nd one | |
| 17:31:52 | kashyap | (If I'm parsin it correctly) | |
| 17:32:44 | sean-k-mooney | i dont see how urllib3 is for http requests not the db unless this is form parsing the db connection url | |
| 17:35:25 | sean-k-mooney | oh look urllib3 had a release 2 days ago... | |
| 17:37:03 | sean-k-mooney | hum ok but we have not change uc to allow it in 4 months | |
| 17:38:39 | kashyap | You mean upper-constraints? | |
| 17:38:54 | kashyap | Thanks for digging that. | |
| 17:39:19 | sean-k-mooney | ya so there has been a release but i dont think its in use | |
| 17:39:34 | sean-k-mooney | upper-constraits is still clamping to an older one on master | |
| 17:39:37 | kashyap | sean-k-mooney: As of now, just to show the "randomness" of the failures: all the jobs that failed in the previous run succeeded now - nova-live-migration, nova-multi-cell, nova-ovs-hybrid-plug, and nova-grenade-multinode jobs | |
| 17:39:51 | kashyap | (Except the new one above in the tox-functional-py38) | |
| 17:40:02 | kashyap | sean-k-mooney: Should we bump it? | |
| 17:40:09 | kashyap | Does it make sense to do so? | |
| 17:40:28 | sean-k-mooney | we have automation to bump it | |
| 17:40:48 | sean-k-mooney | that runs a set of jobs to check compatiablity with most of the projects | |
| 17:41:32 | kashyap | Oh, right; I forgot the bot; that's much safer | |
| 17:45:11 | sean-k-mooney | it ran with urllib3-1.26.12 | |
| 17:45:27 | sean-k-mooney | which is what i have locally and that works | |
| 17:45:44 | sean-k-mooney | my guess isthe two exceptions are somehow related | |
| 17:45:58 | sean-k-mooney | but i can see show directly | |
| 18:51:52 | opendevreview | Dan Smith proposed openstack/nova master: Persist existing node uuids locally https://review.opendev.org/c/openstack/nova/+/863918 | |
| 18:51:53 | opendevreview | Dan Smith proposed openstack/nova master: Make resource tracker use UUIDs instead of names https://review.opendev.org/c/openstack/nova/+/863919 | |
| 18:51:53 | opendevreview | Dan Smith proposed openstack/nova master: WIP: Detect host renames and abort startup https://review.opendev.org/c/openstack/nova/+/863920 | |
| 20:07:26 | opendevreview | Artom Lifshitz proposed openstack/nova master: Microversion 2.94: FQDN in hostname https://review.opendev.org/c/openstack/nova/+/869812 | |
| 20:37:44 | gmann | sean-k-mooney: bauzas: gibi: reminder for osc-placement gate fixes review https://review.opendev.org/q/I4e3e5732411639054baaa9211a29e2e2c8210ac0 | |
| 21:34:49 | sean-k-mooney[m] | gmann im not sure that is what we should be doing | |
| 21:35:39 | sean-k-mooney[m] | i was suggesting continng to use master on master and using the stable branch release or stable | |
| 21:36:21 | gmann | sean-k-mooney[m]: ok for stable. I was thinking we do the same way the Nova testing on master | |
| 21:37:02 | sean-k-mooney[m] | im not sure what that iss of the top of my head but ill take a look | |
| 21:37:27 | gmann | ok. let me know and I can update those accordingly. | |
| 21:38:21 | sean-k-mooney[m] | i guess if it works as it says in the comment that is ok | |
| 21:38:36 | sean-k-mooney[m] | i dont know how that works however | |
| 21:39:02 | sean-k-mooney[m] | i would not expect that to be how it works unless we are doing something special with tox siblings in the jobs | |
| 21:39:41 | sean-k-mooney[m] | the tox ini does not hve the described behavior on its own | |
| 21:39:55 | clarkb | siblings only take effect if you set the project as a required project on the zuul job (that pulls in the git repo and the tox role uses it as a single to install it from source) | |
| 21:40:45 | sean-k-mooney[m] | right so osc-placemnt would need to declarae placemnt as a required project | |