| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-07-11 | |||
| 09:04:56 | sean-k-mooney | ok i had proposed that more or less in my last top level comment on the review | |
| 09:05:09 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/845660/8#message-718400c522ed5bf53306f28a382300335d592f30= | |
| 09:06:35 | gibi | OK, noted my preference there too now | |
| 09:07:19 | sean-k-mooney | thanks | |
| 09:47:25 | bauzas | folks, I have a question | |
| 09:47:34 | bauzas | https://6f28c6786b3c06159e6f-ac3ef42d4a9c79eb41cb204944df5803.ssl.cf2.rackcdn.com/849133/1/check/nova-tox-functional-py38/9cf47c1/testr_results.html | |
| 09:47:53 | bauzas | test_rebuild_with_keypair fails because the regression test uses the latest microversion | |
| 09:48:25 | bauzas | https://github.com/openstack/nova/blob/master/nova/tests/functional/regressions/test_bug_1843708.py#L26 | |
| 09:48:34 | bauzas | so, what would get your preference ? | |
| 09:48:58 | bauzas | changing the microversion used to cap to 2.90, or providing a pubkey ? | |
| 09:49:08 | bauzas | IMHO, the latter | |
| 09:49:12 | gibi | I would provide a pub key | |
| 09:49:20 | bauzas | yeah | |
| 09:49:36 | bauzas | I was thinking this | |
| 09:49:44 | bauzas | thanks | |
| 09:52:55 | sean-k-mooney | ya proably the same i would proably generate a public key or just hardcode a pair in a fixture somewhere that we can reuse | |
| 09:53:18 | sean-k-mooney | the regressiosn should be freestandign for the most part so i woudl do it in the test personally | |
| 11:21:22 | opendevreview | Gorka Eguileor proposed openstack/nova master: Support os-brick specific lock_path https://review.opendev.org/c/openstack/nova/+/849328 | |
| 12:09:40 | opendevreview | Stephen Finucane proposed openstack/placement master: Fix typo in schema https://review.opendev.org/c/openstack/placement/+/849348 | |
| 12:12:33 | stephenfin | gibi: bauzas: sean-k-mooney: I've already sort of made my mind up on this but can you take a look at https://review.opendev.org/c/openstack/placement/+/849348 (context is a change from ratailor at https://review.opendev.org/c/openstack/placement/+/848634) | |
| 12:15:17 | sean-k-mooney[m] | ya we would need a new microverion i belive | |
| 12:15:29 | sean-k-mooney[m] | unless we were failing on a 500 | |
| 12:15:46 | sean-k-mooney[m] | because we checked in the code somewhere else | |
| 12:17:10 | bauzas | stephenfin: I can try to take a look once I'm done with my own change | |
| 12:18:16 | stephenfin | sean-k-mooney[m] If I revert the code change, the new test I added doesn't fail. That would suggest we were happily accepting an empty object at the code level (presumably because we do falsey checks or similar) | |
| 12:18:38 | sean-k-mooney[m] | ack | |
| 12:18:58 | gibi | "This field may be sent when writing allocations back to the server but will be ignored; this preserves symmetry between read and write representations." | |
| 12:19:09 | gibi | so this value is never used by placement | |
| 12:19:22 | gibi | I think we don't need a microversion in this case | |
| 12:19:54 | sean-k-mooney[m] | oh its the provider tree mappings | |
| 12:20:12 | sean-k-mooney[m] | to lookup the provdier tree form the allocation | |
| 12:20:34 | gibi | (context, the mappings are generated by placement during GET allocation_candidates for the client to know which group are fulfiulled from which RP, so writing this back to placement only make sense for keeping the a_c result directly usable for POST allocations | |
| 12:20:38 | gibi | ) | |
| 12:20:55 | sean-k-mooney[m] | ack | |
| 12:21:26 | sean-k-mooney[m] | so we likely can fix it but we should have the release note to call it out | |
| 12:21:42 | sean-k-mooney[m] | which stephenfin has in there patch | |
| 12:26:06 | gibi | yepp, so I'm +2 | |
| 15:22:37 | bauzas | https://paste.opendev.org/show/bR65i5QPAN7ieYOEJHF4/ fails because the pubkey doesn't re.match() | |
| 15:22:49 | bauzas | but... the pubkey is exactly good | |
| 15:24:17 | bauzas | wtf, /me is wrapping his head | |
| 15:27:56 | sean-k-mooney | my guess is maybe there are som spaces or other whitespace | |
| 15:28:10 | sean-k-mooney | i would try removeing the ^ and $ | |
| 15:29:04 | bauzas | I can't remove those | |
| 15:30:17 | bauzas | sean-k-mooney: https://github.com/openstack/nova/blob/master/nova/tests/functional/api_samples_test_base.py#L248-L251 | |
| 15:30:47 | bauzas | before adding the ^ and $ for regexp, I verified the strings | |
| 15:32:38 | bauzas | https://paste.opendev.org/show/bixG1pb6FdkuMcZNYBqK/ shows the strings matching before we add the trailing chars | |
| 15:46:00 | bauzas | oh, I think I spotted it | |
| 15:54:22 | gibi | -/16 | |
| 15:54:48 | ygk_12345 | Hi all. we have a wallaby setup. the compute compute services are not starting saying "too old compute version 30". But we have deleted that version from the nova.services db table. But still it is picking up version 30 which we cant find. How to resolve this > | |
| 15:55:18 | ygk_12345 | bauzas: any idea about this issue ? | |
| 15:56:09 | ygk_12345 | i tried pdb inside nova/cmd/compute.py file and could see that current_version is getting value 30 | |
| 16:12:12 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP: api: Drop generating a keypair and add special chars to naming https://review.opendev.org/c/openstack/nova/+/849133 | |
| 16:12:28 | bauzas | still a WIP due to UTs missing ^ | |
| 16:17:45 | sean-k-mooney | ygk_12345: in addtion to deleting the old compute services you will have to restart the conductors and posibly the other contoller services | |
| 16:17:57 | sean-k-mooney | ygk_12345: they cache the min compute version on start up of the service | |
| 16:18:07 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP: api: Drop generating a keypair and add special chars to naming https://review.opendev.org/c/openstack/nova/+/849133 | |
| 16:18:14 | ygk_12345 | sean-k-mooney: is this in addition to the db entries deletion ? | |
| 16:18:42 | sean-k-mooney | yes so you will need to delete the old compute service records assuming those host nolonger exist | |
| 16:18:55 | sean-k-mooney | and if you have already tried to start the conductor ectra then you will have to restart it | |
| 16:19:03 | sean-k-mooney | ygk_12345: you should only delete the old entries | |
| 16:19:08 | sean-k-mooney | if those host are gone | |
| 16:19:15 | sean-k-mooney | and will not be coming back | |
| 16:19:38 | sean-k-mooney | ygk_12345: there is a workaround for this if you are fast forward upgrading | |
| 16:20:02 | sean-k-mooney | but if you use that you need to be aware that we dont support configuration where the min compute service version is not met | |
| 16:20:39 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#workarounds.disable_compute_service_check_for_ffu | |
| 16:20:42 | ygk_12345 | sean-k-mooney: when restarting the nova-compute service, do they cache any db entries as well ? | |
| 16:21:31 | sean-k-mooney | restarting the nova-comptue service would clear any caching it had since it only caches info in memory | |
| 16:21:56 | sean-k-mooney | we dont cache stuff on disk so there is nothing that would need to be cleaned up manually | |
| 16:23:06 | ygk_12345 | sean-k-mooney: but from where is it picking up the 30 version , even though its not there in the db ? maybe from the memory ? | |
| 16:24:04 | sean-k-mooney | either the binary you are starting is not at the min version or its coming from the db via an rpc to the conductor to get the min service version in the cell | |
| 16:25:00 | ygk_12345 | sean-k-mooney: but the nova.services table doesn't have any trace of version 30 | |
| 16:25:23 | ygk_12345 | sean-k-mooney: also this workaround , is it in control plane nova.conf or in compute nova.conf ? | |
| 16:25:26 | sean-k-mooney | ygk_12345: have you recently done an upgrade to wallaby or are you in the process of an upgrade? | |
| 16:25:38 | sean-k-mooney | ygk_12345: the workaround need to be set on all hosts | |
| 16:25:49 | ygk_12345 | sean-k-mooney: we have upgraded only the control plane | |
| 16:26:00 | sean-k-mooney | from what release | |
| 16:26:07 | ygk_12345 | sean-k-mooney: and we are upgrading computes step by step | |
| 16:26:16 | ygk_12345 | sean-k-mooney: OSA 23.2.0 | |
| 16:26:43 | sean-k-mooney | what openstack release does that map too | |
| 16:26:51 | ygk_12345 | wallaby | |
| 16:27:01 | sean-k-mooney | so 23 is wallayby | |
| 16:27:07 | sean-k-mooney | and you are coming form victoria? | |
| 16:27:07 | ygk_12345 | yes | |
| 16:27:29 | ygk_12345 | yes ussuri->victoria->Wallaby | |
| 16:27:51 | sean-k-mooney | ok so you upgraded the contolers to victoria then upgreaed all the computes | |
| 16:27:58 | sean-k-mooney | then upgraded contoler to wallayby | |
| 16:28:02 | sean-k-mooney | then upgraded the computes | |
| 16:28:39 | sean-k-mooney | you cant upgrade the controlers directly form ussuri to wallayby in one go | |
| 16:28:52 | ygk_12345 | no no. upgraded first the control plane from stein->train->ussuri>vic>wallaby. Then upgraded the computes | |
| 16:29:08 | sean-k-mooney | ok so you are doing a fast forward upgrade | |
| 16:29:17 | sean-k-mooney | or skiplevel depneind on the branding | |
| 16:29:26 | sean-k-mooney | that is not supported by nova directly | |
| 16:29:37 | sean-k-mooney | so you will need to disable our validation with the workaround | |
| 16:29:47 | sean-k-mooney | then upgade allt eh compute to a supported version | |
| 16:29:52 | sean-k-mooney | then renable the check | |
| 16:30:16 | ygk_12345 | so the version 30 is being picked up by cache somewhere ? | |
| 16:30:18 | sean-k-mooney | nova and most service only suport a n to n+1 version delta | |
| 16:30:44 | sean-k-mooney | unlikely | |
| 16:31:02 | sean-k-mooney | its more likely that you are trying to start a stein compute with a wallaby contoler | |
| 16:31:23 | ygk_12345 | yes | |