| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-12-14 | |||
| 21:34:37 | sean-k-mooney | melwitt: i shoudl proably an addtional unit test or two also so ill proably respin it tomorrow but if there is anything else you think i shoudl add let me know. im going to call it a day there o/ | |
| 21:35:18 | melwitt | cool sean-k-mooney, will do. seeya later o/ | |
| 21:47:39 | jawad_axd | Hi! Can someone push me on this one, with newly added compute . I can see in 'openstack compute service list' but not in 'openstack hypervisor list'. This is nova-compute log http://paste.openstack.org/show/801032/ | |
| 21:50:42 | openstackgerrit | Ghanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict https://review.opendev.org/c/openstack/placement/+/766994 | |
| 22:03:40 | openstackgerrit | Ghanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict https://review.opendev.org/c/openstack/placement/+/766994 | |
| 22:39:12 | openstackgerrit | Ghanshyam proposed openstack/nova stable/rocky: DNM: testing gate https://review.opendev.org/c/openstack/nova/+/767027 | |
| #openstack-nova - 2020-12-15 | |||
| 00:24:25 | openstackgerrit | Lance Bragstad proposed openstack/placement master: Pass context objects to oslo.policy directly https://review.opendev.org/c/openstack/placement/+/766785 | |
| 00:24:26 | openstackgerrit | Lance Bragstad proposed openstack/placement master: Implement secure RBAC for resource providers https://review.opendev.org/c/openstack/placement/+/760240 | |
| 00:25:26 | openstackgerrit | Lance Bragstad proposed openstack/placement master: Implement secure RBAC for resource providers https://review.opendev.org/c/openstack/placement/+/760240 | |
| 00:54:56 | openstackgerrit | Ghanshyam proposed openstack/placement master: Fix l-c job for pyflakes, request version conflict https://review.opendev.org/c/openstack/placement/+/766994 | |
| 00:55:16 | openstackgerrit | Wenping Song proposed openstack/nova master: Remove redundant code in instance delete https://review.opendev.org/c/openstack/nova/+/759577 | |
| 01:24:15 | gmann | gibi: melwitt lyarwood this fixes the placement gate (l-c job )- https://review.opendev.org/c/openstack/placement/+/766994 | |
| 01:25:05 | gmann | gibi: melwitt lyarwood I have moved it to latest hacking version which is easy than solving the old hacking + old pyflake compatible versions. | |
| 03:06:20 | openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List SG API https://review.opendev.org/c/openstack/nova/+/766726 | |
| 06:38:08 | openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List SG API https://review.opendev.org/c/openstack/nova/+/766726 | |
| 07:56:28 | gibi | sean-k-mooney: I do see that these test are slower then the others in the CI, but I don't see any timeout in nova. When it fails it fails from an exception from the db driver | |
| 08:25:28 | gibi | gmann: I've left comment in https://review.opendev.org/c/openstack/placement/+/766994 | |
| 08:37:02 | bauzas | good morning Nova | |
| 08:38:34 | hemanth_n | sean-k-mooney: hey regarding backports which we discussed yesterday, I am afraid it requires more patches to backport to ussuri. For now i found these 3 bfe7b7f3ee340012b3a9f1294ce86b07456ab206, 6c4d1a6aabb101466d2a2d05ed4c6240425b76b5, 0ab8a03c782851d4019c90e6159bd1f83c0658e6 and tests are failing still which implies more patches are needed (placement fixtures related ones are needed for sure). I want to take a pause and see if this is the | |
| 08:38:34 | hemanth_n | direction we want given lot many patches are required to make the patch i submitted clean cherrypick.. thoughts? | |
| 08:56:15 | bauzas | gmann: gibi: is https://review.opendev.org/c/openstack/placement/+/766994 helping the placement CI ? | |
| 08:56:55 | bauzas | we have a lof of open reviews from lbragstad about secure-rbac https://review.opendev.org/q/topic:%22secure-rbac%22+(status:open%20OR%20status:merged) , and I want to make sure that it would help | |
| 08:57:31 | bauzas | whoops, bad gerrit query https://review.opendev.org/q/topic:%2522secure-rbac%2522+(status:open+OR+status:merged)+project:openstack/placement | |
| 09:35:39 | openstackgerrit | Elod Illes proposed openstack/nova stable/pike: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/766497 | |
| 10:37:25 | gibi | bauzas: regarding the rbac thing I need to rely on gmann but as far as I know we did similar things in nova in Victoria | |
| 10:37:43 | bauzas | yup for sure | |
| 10:37:46 | gibi | bauzas: regarding the lower constraint fix I that is needed to unblock the placement gate | |
| 10:37:49 | bauzas | let's wait for gmann's reply | |
| 10:57:11 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add requested_networks field to RequestSpec object https://review.opendev.org/c/openstack/nova/+/749977 | |
| 10:57:11 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Add a routed networks scheduler pre-filter https://review.opendev.org/c/openstack/nova/+/749068 | |
| 11:50:00 | gibi | does somebody already looked into the volume handling timeout happening fairly frequently on master in tempest? there is a long timeout (196s) that runs out in various volume handling operations | |
| 11:50:04 | gibi | e.g. detach | |
| 12:01:00 | lyarwood | gibi: I have not but I can if you point me at an example | |
| 12:01:18 | elod | lyarwood: I'm back from PTO and I can work with the 'make LC non-voting' patch if you need help with that | |
| 12:01:20 | gibi | lyarwood: no started looking at https://zuul.opendev.org/t/openstack/build/fce6478a6842495f851b8c88336f0049 | |
| 12:01:33 | gibi | now | |
| 12:01:47 | lyarwood | elod: cool, as above I haven't had much time to push things along | |
| 12:02:32 | lyarwood | elod: if you want to try get https://review.opendev.org/c/openstack/nova/+/766738 passing that would be awesome | |
| 12:04:04 | elod | lyarwood: sure, I'll look at that patch too! thanks :) | |
| 12:21:47 | openstackgerrit | Elod Illes proposed openstack/nova stable/ussuri: [stable-only] lower-contraints: Bump various packages https://review.opendev.org/c/openstack/nova/+/766738 | |
| 12:31:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: add openstack-python3-wallaby-jobs-arm64 job https://review.opendev.org/c/openstack/nova/+/742094 | |
| 12:32:07 | lyarwood | gibi: so I think this is just a race between the instance delete and following volume delete | |
| 12:32:26 | lyarwood | gibi: we need the volume to be listed as available before we try to delete it | |
| 12:32:41 | gibi | so we need to wait for the instance to be deleted? | |
| 12:32:42 | lyarwood | gibi: trying to write this up in a tempest bug now | |
| 12:32:52 | lyarwood | gibi: it's technically async to that | |
| 12:33:09 | lyarwood | gibi: n-cpu deletes the volume attachment and that eventually leads c-vol to mark the volume as available again | |
| 12:33:32 | lyarwood | gibi: I just need to trace the cinder side through to confirm the race but n-cpu hasn't logged the call to delete the volume attachment for some reason | |
| 12:33:37 | gibi | ohh, so we would neet to wait for the volume become available before we call delete on it | |
| 12:33:42 | lyarwood | yeah indeed | |
| 12:33:54 | lyarwood | that's awkward but that's life with async APIs | |
| 12:34:06 | gibi | yeah, I think that is OK to add as a fix | |
| 12:36:19 | gibi | lyarwood: so the tearDownClass() needs to be smarter or we need to extend a specific test case? | |
| 12:37:31 | lyarwood | oh wait, actually it looks like n-cpu didn't delete the attachment | |
| 12:45:51 | teoobo_ | stephenfin: thanks for the code review | |
| 12:55:16 | lyarwood | gibi: ah ha! it's actually slow snapshot creation in c-vol that's causing this | |
| 12:55:20 | lyarwood | gibi: that's also async | |
| 12:55:43 | lyarwood | gibi: there's a TODO from Matt in the test that we should sort out to handle this, adding a waiter on the snapshot state etc | |
| 12:56:04 | gibi | lyarwood: I admire you that you was able to found this, I alway lost in the cinder logs | |
| 12:58:51 | gibi | is there a way I can help fixing this in tempest? | |
| 13:00:56 | gibi | I mean is it easy (for somebody like me without much tempest knowledge) to fix in tempest? | |
| 13:04:11 | lyarwood | gibi: haha thanks, I didn't have to sacrifice anything or anyone this time ;) | |
| 13:04:22 | lyarwood | gibi: so I think we just need to wait until the volume snapshot is ready | |
| 13:04:42 | gibi | I can hack on that if you have better things to do | |
| 13:04:45 | lyarwood | gibi: the issue here is that we are using the Nova imageCreate API that creates the volume snapshot for us indirectly | |
| 13:04:57 | lyarwood | gibi: no I can write this up and fix it | |
| 13:05:03 | gibi | cool, thanks | |
| 13:05:15 | lyarwood | gibi: just need to extract the volume snapshot details from the image metadata | |
| 13:08:03 | gibi | so it is not _that_ simple :) | |
| 13:10:21 | lyarwood | https://bugs.launchpad.net/tempest/+bug/1908269 | |
| 13:10:21 | openstack | Launchpad bug 1908269 in tempest "test_snapshot_volume_backed_multiattach not waiting for volume snapshot to become available" [Undecided,New] | |
| 13:10:26 | lyarwood | haha no nothing to do with bdms is | |
| 13:25:15 | gibi | lyarwood: thanks for the bugreport | |
| 13:28:31 | lyarwood | gibi: np, just rebuilding a devstack env, the fix should be trivial | |
| 13:28:44 | lyarwood | famous last words and all | |
| 13:31:19 | gibi | :) | |
| 13:57:05 | gmann | gibi: bauzas checking | |
| 14:01:43 | openstackgerrit | Merged openstack/nova master: Remove outdated comment from tox.ini https://review.opendev.org/c/openstack/nova/+/765534 | |
| 14:10:12 | gmann | gibi: bauzas on policy stuff in placement, I am waiting for the test case to be added and then we can start review. anyways I will check and leave the comment there. | |
| 14:10:29 | bauzas | all cool | |
| 14:11:44 | gibi | gmann: thanks for following that | |
| 14:12:17 | gmann | gibi: I will update the l-c fix comments after QA office hour. | |
| 14:12:26 | gibi | gmann: cool | |
| 14:14:01 | sean-k-mooney | gibi: did you see my ping yesterday | |
| 14:14:55 | sean-k-mooney | oh you did | |
| 14:15:08 | sean-k-mooney | it still might be worht a try setting the timeout | |
| 14:26:48 | gibi | sean-k-mooney: you want get the test to time out if it slow? or you want to prevent the test to time out when it is slow? | |
| 14:28:47 | sean-k-mooney | manila just set a 5 minute timeout to workaround slow nodes | |
| 14:28:59 | sean-k-mooney | vs the default which i think is 60 seconds | |
| 14:30:25 | gibi | sean-k-mooney: we don't have per test timeout I saw these test run and pass after 300 seconds on the gate | |
| 14:30:57 | sean-k-mooney | thats what the manila patch added | |
| 14:31:17 | gibi | hm, I saw these in the nova test suite | |
| 14:31:37 | sean-k-mooney | https://review.opendev.org/c/openstack/manila/+/291397/ | |
| 14:33:27 | gibi | sean-k-mooney: yeah, I saw this patch yesterday, what I say is that in nova we don't have a per test case timeout, and also I see that in nova these tests run for a long time (both in case of failing or passing) | |
| 14:36:00 | gibi | so I don't see how the extra timeout for these tests would help | |
| 14:37:02 | gibi | logstash don't want to help me know to show some examples | |
| 14:37:23 | gibi | I only found one: TestNovaAPIMigrationsWalkMySQL.test_walk_versions [192.038984s] ... FAILED | |
| 14:37:39 | gibi | TestNovaAPIMigrationsWalkPostgreSQL.test_walk_versions [43.027941s] | |
| 14:41:52 | sean-k-mooney | ah ok so its likely not a timeout issue | |
| 14:42:32 | sean-k-mooney | i did see a timeout in one of the logs | |