| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-12-10 | |||
| 10:38:15 | lyarwood | only joking | |
| 10:38:23 | lyarwood | but seriously get some sleep :D | |
| 10:38:38 | melwitt | I know, that was my OMG I CANT BELIEVE YOU SAID THAT look | |
| 10:38:44 | melwitt | :) | |
| 10:39:38 | melwitt | ok, gnight for reals | |
| 10:39:47 | lyarwood | hehe twas a good one :) | |
| 10:39:54 | lyarwood | \o | |
| 10:48:41 | elod | lyarwood: will the two patch work separately? Don't they need to be squashed to fix every issue? (or will the .zuul.yaml change only run limited tests that could pass? O.o) | |
| 10:49:57 | lyarwood | elod: yup I'm an idiot | |
| 10:51:47 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/victoria: lower-constraints: Bump packaging to 20.4 https://review.opendev.org/c/openstack/nova/+/765749 | |
| 10:51:50 | elod | lyarwood: I'm not sure, just asking | |
| 10:52:37 | lyarwood | elod: no no you are right, lower-constraints would fail in the standalone change | |
| 10:52:52 | lyarwood | elod: so I've squashed it into the fix for that job | |
| 10:53:06 | lyarwood | elod: and once we have a fix for the multi-cell job we should enable it again in that change | |
| 10:53:47 | lyarwood | elod: the only other way of fixing this would be to squash both fixes into one change | |
| 10:54:00 | lyarwood | elod: that I'm not a fan of tbh | |
| 10:55:42 | elod | lyarwood: ok, good, thx! | |
| 11:40:06 | openstackgerrit | Lucian Petrut proposed openstack/nova master: Fix unshelved image ref https://review.opendev.org/c/openstack/nova/+/765924 | |
| 11:58:03 | openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: DNM/WIP Remove device from attach_volume request response https://review.opendev.org/c/openstack/nova-specs/+/765097 | |
| 12:00:25 | gibi | lyarwood: I will start looking into the qos failure on the master as melwitt said we propably hit that on stable after we fix the context targeting issue | |
| 12:01:05 | gibi | hm, first, I will confirm that the stable failure after the context targeting fix is the same as the master failure | |
| 12:12:49 | gibi | OK, I confirm that after https://review.opendev.org/c/openstack/nova/+/766364 on stable we hit the same issue as visible on msater | |
| 12:12:52 | gibi | master | |
| 12:13:33 | gibi | so on master we need to fix https://bugs.launchpad.net/nova/+bug/1907522 and then backport the fix to stable on top of https://review.opendev.org/c/openstack/nova/+/766364 which is only needed on stable | |
| 12:13:33 | openstack | Launchpad bug 1907522 in OpenStack Compute (nova) "test_*_with_qos_min_bw_allocation fails in the nova-multi-cell job with: nova.exception.MigrationPreCheckError: Migration pre-check error: Failed to create port bindings for host <host>" [Undecided,New] | |
| 12:13:41 | gibi | this will be fun | |
| 13:00:06 | gibi | lyarwood, melwitt : I'm affraid that we missed the qos support for cross cell resize. I think qos and cross cell work was done in parallel. | |
| 13:00:58 | gibi | as the migration uses different conductor tasks and ComputeManager methods we simply missing the qos code from the cross cell path | |
| 13:07:27 | openstackgerrit | Illes Elod proposed openstack/nova stable/train: DNM: tox constraints test https://review.opendev.org/c/openstack/nova/+/766463 | |
| 13:38:06 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Disable qos resize tempest test for nova-multi-cell job https://review.opendev.org/c/openstack/nova/+/766471 | |
| 13:38:35 | gibi | lyarwood, melwitt: ^^ this is my way forward on the master and this needs to be backported to stable | |
| 13:44:14 | bauzas | I agree | |
| 13:44:19 | bauzas | we need to revert the tests | |
| 13:44:29 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Disable qos resize tempest test for nova-multi-cell job https://review.opendev.org/c/openstack/nova/+/766471 | |
| 13:45:02 | gibi | bauzas: the test works fine with same cell resize so I would not revert the whole test | |
| 13:47:25 | bauzas | yeah, by reverting, I meant disabling the faulty ones | |
| 13:47:30 | bauzas | not the whole patch | |
| 13:47:37 | bauzas | anyway +2d with comments | |
| 13:49:01 | gibi | let me quicly fix those nits | |
| 13:52:56 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Disable qos resize tempest test for nova-multi-cell job https://review.opendev.org/c/openstack/nova/+/766471 | |
| 13:53:01 | gibi | bauzas: ^^ | |
| 13:54:03 | bauzas | ack | |
| 14:05:55 | elod | lyarwood: I think the bandit patch needs to be merged starting from pike, and then to newer branches, branch by branch... due to grenade job. (which would be unnecessary if bandit 1.6.3 was yanked, but that might not happen soon...) | |
| 14:08:27 | elod | lyarwood: also, I'm experimenting with the 'general' lower-constraint fix ( https://review.opendev.org/q/topic:"constrain-tox-install" ) but it might be better to try to fix the lower-constraints.txt instead, if it does not bring up infinite package version bumps there... | |
| 14:10:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: Disable qos resize tempest test for nova-multi-cell job https://review.opendev.org/c/openstack/nova/+/766474 | |
| 14:23:59 | gibi | elod: lyarwood: now I two possible way forward for stable. a) disable nova-multi-cell job b) disable the qos tests in the nova-multi-cell job | |
| 14:24:50 | lyarwood | hey sorry just back from lunch | |
| 14:25:03 | lyarwood | elod: ack I can work my way forwards from pike if it's going to be easier | |
| 14:25:38 | lyarwood | gibi: https://review.opendev.org/c/openstack/nova/+/765749 is alreadying in the gate for victoria | |
| 14:25:46 | lyarwood | gibi: making the entire job non-voting | |
| 14:26:53 | gibi | lyarwood: OK to merge that. Then to re-enable the job we need https://review.opendev.org/c/openstack/nova/+/766474 | |
| 14:27:41 | gibi | then the question what to do https://review.opendev.org/c/openstack/nova/+/766364/ | |
| 14:29:41 | lyarwood | gibi: https://review.opendev.org/c/openstack/nova/+/765749 isn't actually in the gate yet so we could yank it out now | |
| 14:29:45 | elod | lyarwood: I've just checked that there's no lower-constraint job @ pike, so bandit patch could be merged easier (fingers crossed) :] | |
| 14:30:27 | gibi | lyarwood: keep it as it is closer to merge | |
| 14:30:28 | lyarwood | elod: cool, I'll try to make a start shortly | |
| 14:30:39 | lyarwood | gibi: kk | |
| 14:30:46 | gibi | lyarwood: then I will try to reenable the gate with https://review.opendev.org/c/openstack/nova/+/766474 | |
| 14:31:38 | gibi | regarding https://review.opendev.org/c/openstack/nova/+/766364/ I have to figure out if we need this to avoid leaking something instead of clearly rolling back the migration | |
| 14:40:09 | gmann | brinzhang: ah that is good question, I do not think we need to do any improvement on deprecated APIs. We have not done any update on deprecated API in past too. policy changes were on policy side only so we thought of keeping consistency. But I will discuss it in today meeting to get more opinion. | |
| 14:41:50 | elod | lyarwood: btw, if you are busy, I can do the bandit backports till pike | |
| 14:42:17 | elod | just let me know :) | |
| 14:44:10 | lyarwood | elod: if you have time then please go ahead and I can review | |
| 14:44:42 | gmann | brinzhang: but server sec group is not deprecated API -https://docs.openstack.org/api-ref/compute/?expanded=list-security-groups-by-server-detail#servers-security-groups-servers-os-security-groups | |
| 14:45:10 | elod | lyarwood: ok, doing it now | |
| 14:45:52 | gmann | brinzhang: ah just saw that we did mention about not to change deprecated APIs for tenant-id - https://specs.openstack.org/openstack/nova-specs/specs/wallaby/approved/remove-tenant-id.html | |
| 14:46:24 | gmann | brinzhang: so we are good on that and let's target only non-deprecated APIs only | |
| 14:49:12 | gmann | brinzhang: also our all deprecated APIs are capped with corresponding microversion and they are 404 for higher than deprecated microversion so technically we cannot do any change in those API which include the microversion bump. | |
| 14:49:55 | openstackgerrit | Illes Elod proposed openstack/nova stable/stein: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/766487 | |
| 15:00:52 | openstackgerrit | Illes Elod proposed openstack/nova stable/stein: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/766487 | |
| 15:03:47 | openstackgerrit | Illes Elod proposed openstack/nova stable/rocky: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/766492 | |
| 15:11:14 | lyarwood | elod: Bernard just replied on the ML about the cross project issue with the bandit cap | |
| 15:11:32 | lyarwood | elod: and thinking about it more an overall cap in requirements might be the only way to unblock things here | |
| 15:11:49 | lyarwood | elod: but landing from stable/pike as you suggested for the project specific changes | |
| 15:11:56 | lyarwood | elod: so grenade works etc | |
| 15:12:53 | lyarwood | `ERROR Failed to update project None in 3s` oh cool a new zuul error | |
| 15:13:29 | openstackgerrit | Illes Elod proposed openstack/nova stable/pike: [stable-only] Cap bandit to 1.6.2 https://review.opendev.org/c/openstack/nova/+/766497 | |
| 15:14:30 | lyarwood | https://github.com/PyCQA/bandit/issues/663 that said it looks like they might yank 1.6.3 after all | |
| 15:23:53 | gibi | lyarwood: this is not the first time I saw that zuul ERROR | |
| 15:24:09 | gibi | lyarwood: at the previous time it was around a gerrit restart so I assmue that caused it | |
| 15:24:33 | lyarwood | ah kk | |
| 15:24:54 | bauzas | ergh, call me stupid | |
| 15:25:00 | bauzas | requested_networks can be None | |
| 15:26:39 | gibi | lyarwood: pinged infra about it | |
| 15:28:00 | lyarwood | gibi: ah thanks, I was just going to recheck tbh | |
| 15:28:10 | gibi | lyarwood: sure, | |
| 15:29:27 | gibi | lyarwood: infra increased some git cloning timeouts in the zuul executors | |
| 15:29:32 | gibi | recently | |
| 15:30:22 | gibi | they expect that it helps | |
| 15:35:09 | elod | lyarwood: if they would yank bandit 1.6.3 that would be the best :) | |
| 15:35:53 | elod | lyarwood: until that, let's see if we can merge the bandit cap patches, as it might take some time | |
| 15:36:06 | lyarwood | elod: ack lets keep trying | |
| 15:36:27 | lyarwood | elod: I'm just worried that we will need to land multiple caps across multiple projects to allow the gate to work again | |
| 15:37:08 | elod | lyarwood: yes, that's possible :S | |
| 17:00:12 | gibi | lyarwood, melwitt: I rechecked https://review.opendev.org/c/openstack/nova/+/766471 (master unblock patch) as it bounced due to zuul " ERROR Failed to update project None in" | |
| 17:01:07 | gibi | I have to leave in ~15 minutes so I hope melwitt can take over rechecking it if needed | |
| 17:12:14 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM https://review.opendev.org/c/openstack/nova/+/766509 | |
| 17:13:52 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/victoria: DNM https://review.opendev.org/c/openstack/nova/+/766511 | |
| 17:15:42 | lyarwood | gibi: ack thanks I'll also be around for a while to recheck when required | |
| 17:16:04 | gibi | lyarwood: thanks I will check back too before I go to sleep but now dinner time | |