| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-03-01 | |||
| 17:37:28 | clarkb | its purely a I'm making a connection we need to negotiate details and can't agree on those details problem | |
| 17:37:29 | bauzas | https://www.rfc-editor.org/rfc/rfc8332#section-5.2 | |
| 17:38:06 | bauzas | "Nevertheless, implementations SHOULD start to disable "ssh-rsa" in their default configurations as soon as the implementers believe that new RSA signature algorithms have been widely adopted." | |
| 17:39:36 | dansmith | clarkb: so why do I need the PubkeyAcceptedTypes change? | |
| 17:40:07 | clarkb | dansmith: because ssh-rsa means sha1 | |
| 17:40:20 | clarkb | dansmith: the protocl uses something liks rsa-sha-256 to denote the sha2 varints | |
| 17:40:39 | dansmith | clarkb: so is the option really "PubkeyExchangeNegotiationTypes" ? | |
| 17:40:43 | clarkb | and the server doesn't know to negotiate rsa-sha-256 because the default when you don't know how to negotiate is ssh-rsa | |
| 17:41:24 | dansmith | like, the key is rsa, and ssh-rsa is not the key type *on disk* but the key type "in our negotiation" ? | |
| 17:41:34 | clarkb | dansmith: sort of? I think what goes on is both sides need to agree on the pubkey type they will accept. The server if old like dropbear says ssh-rsa but not rsa-sha-256. Your client is new and only says rsa-sha-256 and there is no overlp and it fails | |
| 17:42:23 | dansmith | I guess the thing that is confusing is that I consider the "type of the key" to be a property or characteristic of the content of my file on disk | |
| 17:42:44 | dansmith | but it sounds like that's not what "Pubkey .. Type" means both to ssh over the wire and in its config | |
| 17:42:57 | clarkb | correct | |
| 17:43:11 | dansmith | all I want is you to hug me and tell me I'm not crazy for finding that confusing :D | |
| 17:43:32 | clarkb | you are not crzy. When we first ran into this with gerrit and fedora making the change early there was a couple days of major head scratching | |
| 17:54:23 | sean-k-mooney | fedora used to have a tool to cofniuure thigns https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2 | |
| 17:55:03 | sean-k-mooney | and yes you can use rsa-sha2-256 or rsa-sha2-512 | |
| 17:55:40 | bauzas | clarkb: you have way more background than me on this one, do you agree with me on the fact that the current pubkey hint will be later dropped, if I read carefully https://www.rfc-editor.org/rfc/rfc8332#section-5.2 ? | |
| 17:55:55 | sean-k-mooney | so you would add PubkeyAcceptedKeyTypes +rsa-sha2-256 to the sshconfig | |
| 17:56:52 | sean-k-mooney | after i had it working and then it broke with gerrit later i jsut gave in and stopped using rsa keys | |
| 17:58:29 | bauzas | sean-k-mooney: my thought was that PubkeyAcceptedKeyTypes +ssh-rsa was only a temporary workaround until all OpenSSH clients were supposed new enough to drop this compat natively | |
| 17:58:59 | sean-k-mooney | yes but you can disable his on the server side too | |
| 17:59:25 | sean-k-mooney | for gerrit it did not supprot swaping to sha2 until very recently | |
| 18:00:17 | clarkb | bauzas: what that is saying is that ssh-rsa is going away which means rsa + sha1 is going away. There are no plans to remove rsa + sha2 as far as I know | |
| 18:00:44 | bauzas | anyway, for the grenade need, generating a ecdsa key is OK | |
| 18:00:51 | sean-k-mooney | bauzas: i fyou look at https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2 | |
| 18:01:00 | bauzas | clarkb: yeah that's what I found | |
| 18:01:27 | sean-k-mooney | fedor aplanned to drop rsa for key excachange in a future defualt policy that was for f33 | |
| 18:01:35 | sean-k-mooney | i think that came intor affect aroudn f35 | |
| 18:01:52 | bauzas | that's not the RSA keys which are deprecated, that's the SHA1 signature algorithm which is (hence the +o PubkeyAcceptedKeyTypes +ssh-rsa be only a temporary workaround) | |
| 18:02:01 | clarkb | sean-k-mooney: thats only rsa + sha1 | |
| 18:02:17 | sean-k-mooney | key exchange: ECDHE, DHE | |
| 18:02:23 | sean-k-mooney | under FUture | |
| 18:02:50 | clarkb | wut | |
| 18:03:17 | clarkb | oh dhe is rsa iirc | |
| 18:03:28 | sean-k-mooney | anyway it sould liek a this is related to rsa/sha1 and we shoudl be able to fix that by using a ecdsa key which will work for fips too | |
| 18:03:43 | sean-k-mooney | instead of hacking in the old key type which wont | |
| 18:05:33 | clarkb | and if anyone can grok the openssh code better than I a PR to default to rsa + sha2 as teh fallback might generate interesting conversations | |
| 18:05:34 | sean-k-mooney | so we should just replace https://github.com/openstack/grenade/blob/master/projects/70_cinder/resources.sh#L121 | |
| 18:05:46 | sean-k-mooney | with a call to ssh-keygen | |
| 18:05:50 | clarkb | the rfc suggests this happen eventually but as far as I can tell it hasn't happened yet which leads to annoying failures n a lot of cases | |
| 18:06:51 | bauzas | sean-k-mooney: patch is up to change grenade https://review.opendev.org/c/openstack/grenade/+/875940/ | |
| 18:07:09 | sean-k-mooney | cool | |
| 18:07:18 | sean-k-mooney | so ye were just discussing why this is needed | |
| 18:07:24 | bauzas | yeah | |
| 18:07:25 | sean-k-mooney | rahter then trying to find a solution | |
| 18:07:27 | sean-k-mooney | ok | |
| 18:07:29 | dansmith | yeah because I definitely didn't | |
| 18:07:30 | bauzas | well | |
| 18:07:47 | bauzas | we are trying to untangle the oddness of ssh negociation | |
| 18:07:48 | dansmith | and I have a bunch of hacks in my own ssh_config that probably need revisiting now that more of my systems are upgraded | |
| 18:07:56 | bauzas | me too | |
| 18:08:10 | bauzas | this discussion is half-workwise, halp-personalwise | |
| 18:08:14 | sean-k-mooney | ok i got burned by this years ago and have helped other fix it in the past so i mostly just accpet it at this point | |
| 18:08:36 | bauzas | I just shamelessly tuned the signature negociation on the fly with my ssh_config | |
| 18:09:04 | bauzas | as I didn't wanted to generate a new pair of ECDSA or something else keys | |
| 18:09:15 | sean-k-mooney | ya i still have PubkeyAcceptedKeyTypes +ssh-rsa for one site in my config | |
| 18:09:20 | sean-k-mooney | but i think thats offline | |
| 18:09:41 | bauzas | but now if I understand correctly, I could rather continue to use my RSA keys but ask for a different signature | |
| 18:09:58 | sean-k-mooney | i also still have | |
| 18:10:01 | bauzas | using rsa-sha2 | |
| 18:10:01 | sean-k-mooney | host review.opendev.org | |
| 18:10:03 | sean-k-mooney | HostKeyAlgorithms ssh-rsa | |
| 18:10:05 | sean-k-mooney | KexAlgorithms +diffie-hellman-group1-sha1 | |
| 18:10:07 | sean-k-mooney | Ciphers +aes128-cbc | |
| 18:10:12 | sean-k-mooney | for some reason witch i relly dont need | |
| 18:10:46 | clarkb | bauzas: correct. The problem is that some servers don't know how to negotiate that with you like old dropbear and old gerrit | |
| 18:10:49 | sean-k-mooney | thats what i treid when PubkeyAcceptedKeyTypes +ssh-rsa stopped working for gerrit | |
| 18:11:17 | bauzas | clarkb: so the per-host config is still required, gotcha | |
| 18:11:19 | clarkb | the gerrit we have deployed at review.opendev.org should work fine though. ianw and I worked with gerrit and mina sshd upstream and got that all fixed and eventually got it backported to gerrit 3.5 (it was always on 3.6) | |
| 18:11:42 | clarkb | bauzas: if the servers don't know how to negotiate rsa + sha2 | |
| 18:11:48 | clarkb | review.opendev.org should not need this anymore | |
| 18:11:49 | bauzas | yeah got it | |
| 18:12:02 | bauzas | clarkb: how to enable rsa+sha2 globally in the config ? | |
| 18:12:24 | clarkb | bauzas: if your openssh client is new (like on fedora or jammy etc) then thats the only version they will use by default | |
| 18:12:25 | bauzas | because I assume my current OS doesn't have its defaults changed | |
| 18:12:35 | bauzas | oh | |
| 18:12:52 | clarkb | thats why it failed to talk to cirros. jammy will only use rsa + sha2 by default, but cirros dropbear will only do rsa + sha1 | |
| 18:13:02 | clarkb | this mismatch leads to a failure to negotiate between them and no ssh connection | |
| 18:13:04 | sean-k-mooney | i think you would add PubkeyAcceptedKeyTypes rsa-sha2-256 | |
| 18:13:16 | sean-k-mooney | under "HOST *" | |
| 18:13:18 | clarkb | sean-k-mooney: that shouldn't be necessary its automatic | |
| 18:13:27 | clarkb | since all the new lcients only do sha2 | |
| 18:13:28 | bauzas | hmmm | |
| 18:13:43 | sean-k-mooney | ya it should not but if you wanted to force it that would be how | |
| 18:14:08 | bauzas | yeah got it | |
| 18:14:23 | bauzas | this chat is definitely a helper | |
| 18:14:26 | bauzas | clarkb: thanks a lot ! | |
| 18:15:37 | sean-k-mooney | i really would like to just upload a pulic key to keystone and use that to authenticate with osc | |
| 18:23:28 | sean-k-mooney | bauzas: for what its woth its in te seciryt enhancmetns secation fo the 22.04 release notes https://discourse.ubuntu.com/t/jammy-jellyfish-release-notes/24668 | |
| 18:25:45 | sean-k-mooney | it was disabled by defult in openssh https://www.openssh.com/txt/release-8.8 | |
| 18:26:35 | sean-k-mooney | after being deprecated in 8.3 https://lwn.net/Articles/821544/ | |
| 18:42:07 | clarkb | right but they didn't change the fallback key | |
| 18:42:33 | clarkb | so there are layers of problems here. The first is that you can no longer negotiate ssh-rsa with servers that need it. Second is that when that negotiate fails both sides expect ssh-rsa | |
| 18:42:40 | clarkb | which of course fails making the fallback useless | |
| 18:43:23 | clarkb | they should've changed the fallback to rsa-sha-256 so that clients would attempt that after a failed negotiation. This wold still fail on cirros but would've worked with gerrit | |
| 18:52:24 | opendevreview | Merged openstack/nova stable/victoria: Fix the wrong exception used to retry detach API calls https://review.opendev.org/c/openstack/nova/+/866086 | |
| 22:39:08 | opendevreview | Dan Smith proposed openstack/nova master: Add grenade-skip-level-always to nova https://review.opendev.org/c/openstack/nova/+/875773 | |
| 23:59:05 | opendevreview | melanie witt proposed openstack/nova master: testing: Reset affinity support global variables https://review.opendev.org/c/openstack/nova/+/875991 | |
| #openstack-nova - 2023-03-02 | |||
| 08:47:41 | opendevreview | Jorge San Emeterio proposed openstack/nova master: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/873127 | |