| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-07-21 | |||
| 09:18:19 | bauzas | but I'll do it | |
| 09:25:28 | MichielPiscaer[m] | bauzas: I think that when ops travel to the PTG, they are probably newer then pre-train. | |
| 09:25:56 | sean-k-mooney | MichielPiscaer[m]: yes that has been my expiricne as well | |
| 09:26:12 | sean-k-mooney | the ones that only attend the summit/fourm are slower moving | |
| 09:26:50 | sean-k-mooney | the ones that know about and attend the PTG tend to run more recent versions and be more familar with how the comunity works and be more invovled in general | |
| 09:28:01 | MichielPiscaer[m] | indeed | |
| 10:16:22 | EugenMayer | after i upgraded to the latest xena serias with kolla, my nova docker container fails to authenticate to libvirt due to the new sasl password | |
| 10:16:32 | EugenMayer | 2022-07-21 09:19:23.586+0000: 6539: error : virNetSASLSessionServerStep:594 : authentication failed: Failed to start SASL negotiation: -20 (SASL(-13): user not found: unable to canonify user and get auxprops) | |
| 10:17:16 | EugenMayer | i ensure i merged the inventory/password file properly, but i seem to now be able to fix it via kolla. Is there a way i could fix that on nova's side manually= | |
| 10:20:55 | EugenMayer | my question would be, where would the sasl password be configured | |
| 10:29:10 | sean-k-mooney | EugenMayer: sorry had wifi issue | |
| 10:29:48 | sean-k-mooney | EugenMayer: in the libvirt section we have an optional connection uri option | |
| 10:29:50 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.connection_uri | |
| 10:30:21 | sean-k-mooney | EugenMayer: you may be able ot workaround your auth issues by adding parmaters to that | |
| 10:30:44 | sean-k-mooney | EugenMayer: https://libvirt.org/uri.html | |
| 10:30:48 | sean-k-mooney | is the libvift docs | |
| 10:30:55 | EugenMayer | what i did is i ran 'saslpasswd2 -c -p -a libvirt nova' with the sasl pw on the libvirt container | |
| 10:31:11 | EugenMayer | and verified that the auth.conf for nova has the same user/password | |
| 10:33:05 | sean-k-mooney | EugenMayer: auth.conf? | |
| 10:33:28 | sean-k-mooney | i assume that is a libvirt config file | |
| 10:34:22 | EugenMayer | that's a kolla config, but i guess it is mounted into nova, let me inspect the docker containerh | |
| 10:34:59 | sean-k-mooney | EugenMayer: i think the problem you are having is that you are tryign to use a libvirt feature that nova has no offical supprot for | |
| 10:35:05 | sean-k-mooney | EugenMayer: it might be possibel to make it work | |
| 10:35:21 | sean-k-mooney | but its not documenated as supported so its not a bug if it does not | |
| 10:35:26 | sean-k-mooney | its a new feature | |
| 10:35:45 | EugenMayer | "/etc/kolla/nova-compute/:/var/lib/kolla/config_files/:ro", | |
| 10:36:16 | EugenMayer | this probably means that those configs are used with a entrypoint and then generate the actual nova config, i dont know | |
| 10:36:51 | sean-k-mooney | EugenMayer: i assume you are tyring to use https://github.com/openstack/kolla-ansible/commit/d2d4b53d47df3b1a250c21404a8ec140873d4ce5 feature | |
| 10:37:18 | EugenMayer | sean-k-mooney i'am not really doing anything myself, i rather upgraded to the newest kolla version for xena, an they introduced this https://docs.openstack.org/releasenotes/kolla-ansible/xena.html#upgrade-notes | |
| 10:37:36 | EugenMayer | they are now, AFAIU talking with a sasl auth between nova and libvirt | |
| 10:37:48 | EugenMayer | and this is the default in this regard | |
| 10:38:04 | sean-k-mooney | i see | |
| 10:38:13 | sean-k-mooney | the nova comunity was nto invovled in that work | |
| 10:38:18 | EugenMayer | sean-k-mooney what you linked is perfectly right, but you also see, they default to enabling it | |
| 10:38:19 | sean-k-mooney | so its nice that it works | |
| 10:38:36 | EugenMayer | i see | |
| 10:39:13 | sean-k-mooney | https://docs.openstack.org/kolla-ansible/latest/reference/compute/libvirt-guide.html#sasl-authentication | |
| 10:40:04 | sean-k-mooney | so it looks like you need new passwoard in you passwords.yal | |
| 10:40:15 | sean-k-mooney | i assume you geneerated those and mreged them with your exsitng ones | |
| 10:41:10 | EugenMayer | did that already, the upgrade was not working without that | |
| 10:41:30 | sean-k-mooney | ya i would expect the templates to fail to generate the config | |
| 10:41:31 | EugenMayer | that is why i said on the compute, the auth.conf is created, has the right creds | |
| 10:41:37 | sean-k-mooney | ack | |
| 10:41:56 | sean-k-mooney | for now i would proably disabel the sasl auth and complete the upgrade | |
| 10:41:58 | EugenMayer | and i assured that the backend, libvirt, has set the sasls password for the particular user | |
| 10:42:06 | sean-k-mooney | then try and do a reconfigure later | |
| 10:42:13 | EugenMayer | yes, seems that this might be the option i should go for | |
| 10:43:16 | sean-k-mooney | i have not deployed kolla since wallaby by the way | |
| 10:43:27 | sean-k-mooney | so im not up to date on how they currenlty do things | |
| 10:44:28 | sean-k-mooney | lookign at the error | |
| 10:44:45 | sean-k-mooney | i wonder is this related to userids and group ids | |
| 10:44:55 | sean-k-mooney | libvirt is sayign the user does not exist | |
| 10:45:30 | sean-k-mooney | user | |
| 10:45:33 | sean-k-mooney | | not found: unable to canonify user and get auxprops | |
| 10:46:58 | sean-k-mooney | EugenMayer: kolla defiens the nova user id and group id https://github.com/openstack/kolla/blob/master/kolla/common/users.py#L148-L151= | |
| 10:47:08 | sean-k-mooney | but maybe that user is not create in the libvirt contaienr | |
| 10:47:28 | EugenMayer | the docs are wrong there anyway | |
| 10:47:56 | EugenMayer | The username is configured via ``libvirt_sasl_authname``, and defaults to | |
| 10:47:56 | EugenMayer | ``kolla``. The password is configured via ``libvirt_sasl_password``, and is | |
| 10:47:56 | EugenMayer | generated with other passwords using and stored in ``passwords.yml``. | |
| 10:48:14 | EugenMayer | which is wrong, the default user is nova | |
| 10:48:17 | EugenMayer | # Username for libvirt SASL. | |
| 10:48:17 | EugenMayer | libvirt_sasl_authname: "nova" | |
| 10:48:23 | sean-k-mooney | ack | |
| 10:48:24 | EugenMayer | and also in my auth.conf it is nova | |
| 10:48:26 | sean-k-mooney | https://github.com/openstack/kolla/blob/master/docker/nova/nova-libvirt/Dockerfile.j2#L10= | |
| 10:48:38 | sean-k-mooney | and nova should exist in the libvirt container | |
| 10:49:07 | EugenMayer | i did create it using saslpasswd2 -c -p -a libvirt nova | |
| 10:49:18 | EugenMayer | myself, just to ensure it has been provisioned | |
| 10:49:38 | sean-k-mooney | in the container | |
| 10:49:42 | sean-k-mooney | ok | |
| 10:50:04 | EugenMayer | you mean, there should be a linux user 'nova'? | |
| 10:50:12 | sean-k-mooney | yes | |
| 10:50:18 | sean-k-mooney | i think that is what the error is about | |
| 10:50:21 | EugenMayer | (nova-libvirt)[root@compute1 /]# cat /etc/passwd | grep nova | |
| 10:50:21 | EugenMayer | nova:x:42436:42436::/var/lib/nova:/usr/sbin/nologin | |
| 10:50:30 | sean-k-mooney | yep | |
| 10:50:33 | sean-k-mooney | that is what we expect | |
| 10:50:42 | EugenMayer | yes | |
| 10:51:28 | sean-k-mooney | what host os are you using | |
| 10:51:50 | sean-k-mooney | i assum this is not related to md5 beign diabled because of fips or similar | |
| 10:52:48 | EugenMayer | host os would not make any sense here | |
| 10:53:04 | sean-k-mooney | ack | |
| 10:53:08 | EugenMayer | sicne nova_compute and nova_libvirt run in dedicated docker containers, so the os is picked by kolla | |
| 10:53:20 | sean-k-mooney | not nessisarly | |
| 10:53:23 | sean-k-mooney | they share the kernel | |
| 10:53:41 | EugenMayer | the container runs debian bullseye | |
| 10:53:41 | sean-k-mooney | and if md5 was disabeled by the kernel security policy it would not work | |
| 10:53:58 | EugenMayer | i see, well the host is what is expected by kolla, some version of ubuntu | |
| 10:54:15 | sean-k-mooney | ya should not be an issue | |
| 10:54:20 | EugenMayer | 11.4 | |
| 10:54:56 | sean-k-mooney | i just asked becasue i know in fips enforcing mode md5 is not allowed by the kernel and its default to DIGEST_MD5 when libvirt_tls is not enabled | |
| 10:55:22 | sean-k-mooney | that woudl likely give you a diffent error anyway | |
| 10:56:02 | sean-k-mooney | https://github.com/openstack/kolla-ansible/blob/stable/xena/ansible/roles/nova-cell/handlers/main.yml#L118-L129= feels like a bit of a hack but form what i can see you appear to have set it correctly manually too | |
| 10:57:56 | sean-k-mooney | ah i see | |
| 10:57:59 | sean-k-mooney | https://github.com/openstack/kolla-ansible/commit/d2d4b53d47df3b1a250c21404a8ec140873d4ce5#diff-4b6ee2f357ff265811c14e74dd6144d494c4f2baf9e00a9871207114b1172a4bR58 | |
| 10:58:05 | sean-k-mooney | that is how they are making this work | |
| 10:58:40 | EugenMayer | yes | |
| 10:58:51 | sean-k-mooney | EugenMayer: so they are modifying the default session config for the nova user | |
| 10:59:07 | sean-k-mooney | that is distro specific normally | |