| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-11 | |||
| 15:16:56 | efried | having displayed any understanding whatsoever, lyarwood, you're now permanently on call for support. | |
| 15:17:06 | sean-k-mooney | so its only kept in memory and never on disk | |
| 15:17:17 | sean-k-mooney | so if we store it in barbican and then do that that would be better | |
| 15:17:34 | efried | oh, cool, I saw that but hadn't played with it yet, will have to try that out and see if it does the trick. | |
| 15:17:42 | sean-k-mooney | well ephemeral=yes private=yes | |
| 15:17:51 | efried | The secret I created while playing was private but not ephemeral. | |
| 15:18:14 | jroll | can confirm ephemeral=yes leaves it off of disk | |
| 15:18:35 | sean-k-mooney | so form a nova point of view can we use barbican to store it | |
| 15:18:41 | efried | cool, so the nova user is the root of trust because has access to the barbican entry. | |
| 15:18:49 | efried | yeah, what sean-k-mooney said. | |
| 15:18:56 | jroll | and restarting libvirt nukes an ephemeral secret | |
| 15:18:59 | jroll | \o/ | |
| 15:19:22 | sean-k-mooney | jroll: that is proably ok/good | |
| 15:19:25 | efried | now we just gotta hope you don't have to restart libvirt to make it pick up the secret :P | |
| 15:19:29 | lyarwood | thanks sean-k-mooney | |
| 15:19:46 | jroll | efried: fwiw, I didn't need to restart, but I'm on older libvirt playing with tls secrets, not tpm | |
| 15:20:10 | sean-k-mooney | jroll: i dont know if libvirt makes a difference | |
| 15:20:21 | efried | jroll: didn't need to restart to do what? | |
| 15:20:56 | efried | in my case everything appeared to be moving along fine until I tried booting the guest, then the nova log got an exception complaining it couldn't find my secret. | |
| 15:21:27 | efried | anyway, ephemeral would be entirely useless in that case, so it MUST work (famous last words) | |
| 15:21:31 | jroll | efried: ah, I was just doing 'virsh secret-list' | |
| 15:21:33 | jroll | heh | |
| 15:21:46 | jroll | sean-k-mooney: if there's a bug in newer libvirt it might :) | |
| 15:22:04 | sean-k-mooney | oh i ment between secrete usage/types | |
| 15:22:18 | sean-k-mooney | i think it just has a secret that is use for many things | |
| 15:22:21 | efried | jroll: btw, there was a question as to whether we needed to bother providing the non-encrypted version of vTPM support at all. | |
| 15:22:23 | jroll | older libvirt won't let me create a vtpm secret type ¯\_(ツ)_/¯ | |
| 15:22:44 | efried | right, that needs to be >=5.6.0, which afaict you have to compile yourself. | |
| 15:22:46 | jroll | efried: I'd be pro-not-supporting that, I don't like footguns | |
| 15:22:51 | sean-k-mooney | e.g. i dont think a secreate for tls or tpm is a different thing internally in libvirt | |
| 15:23:09 | efried | penick indicated the same. Just that the requirements are going to be way more stringent for it if so. | |
| 15:23:52 | sean-k-mooney | for what its worth rhel/fedora will be shiping it relitvly quickly | |
| 15:23:59 | sean-k-mooney | im not sure when exactly | |
| 15:24:08 | sean-k-mooney | but we have an advance virt stream we use for openstack | |
| 15:24:17 | efried | I guess "with ussuri" is soon enough <shrug> | |
| 15:24:30 | sean-k-mooney | that ships a newer version then base rhel | |
| 15:24:37 | efried | but also the barbican requirement | |
| 15:25:00 | efried | anyway, glad you agree jroll, it makes the code & test surface easier. | |
| 15:25:37 | efried | lyarwood: since you mentioned the qemu side -- do you have any idea what's the minimum qemu version to support encrypted vTPM? I couldn't find that in the docs anywhere. | |
| 15:26:56 | efried | canori01: it would help to know why the allocation update is failing (out of resources? provider conflict? other?), and what action is being performed when that happens. | |
| 15:28:38 | canori01 | efried: I am replacing old hardware. So I powered off the old hypervisor and then did a nova-evacuate. All the guests evacuated except for two which gave me the provider conflict | |
| 15:29:03 | lyarwood | efried: I don't sorry | |
| 15:29:31 | lyarwood | efried: 2.12 is listed in the Stein spec FWIW | |
| 15:30:02 | efried | lyarwood: yeah, that was for non-encrypted, and idk if there's a higher req for encrypted. | |
| 15:30:23 | efried | canori01: If it was a provider conflict, you should just be able to "try again". | |
| 15:30:33 | lyarwood | efried: no idea sorry, kashyap is out today otherwise I'd ask him if he could work that out for us. | |
| 15:30:49 | efried | ah, good plan. I'll stalk him, thanks. | |
| 15:30:52 | sean-k-mooney | efried: tpm emulation was added in 2.11 but im not sure if that included encryption | |
| 15:31:10 | efried | it did not | |
| 15:31:39 | efried | But I don't even know whether qemu required any update for that aspect. It might not have. | |
| 15:31:49 | sean-k-mooney | also this is an intersting site https://repology.org/project/libvirt/versions | |
| 15:33:22 | sean-k-mooney | look like apline has the depencyies you need but very little else at teh moment | |
| 15:34:28 | sean-k-mooney | Fedora Rawhide would "work" too although i suspect nova wont be happy with it in general | |
| 15:35:06 | efried | sean-k-mooney: oh, that reminds me, I ran into another weird quirk with the home-built libvirt... | |
| 15:35:49 | efried | sean-k-mooney: When I restarted the libvirt service, it blew up because it was looking in /usr/lib/x86_64-linux-gnu/ which contained the old libvirt 4.xx | |
| 15:36:06 | efried | I had to go into the service and set LD_LIBRARY_PATH=/usr/lib explicitly to make it work. | |
| 15:36:11 | efried | which seems... weird. | |
| 15:36:17 | efried | had to do the same with n-cpu. | |
| 15:36:52 | sean-k-mooney | ya that does seam strange | |
| 15:37:02 | sean-k-mooney | well clearly i have more work to do | |
| 15:37:56 | dansmith | efried: why is that surprising? | |
| 15:38:19 | dansmith | efried: if you have an old version installed in the standard system library path, and start a new daemon binary that requires the new libs, | |
| 15:38:24 | dansmith | you have to tell it where they are | |
| 15:38:33 | dansmith | (if I'm understanding what you're saying) | |
| 15:38:36 | efried | dansmith: Is /usr/lib/x86_64-linux-gnu/ the "standard system library path"? | |
| 15:38:47 | efried | That was the surprising part to me. I would have thought /usr/lib was. | |
| 15:39:08 | dansmith | efried: for multi-arch stuff on fedora-like distros it is I think yeah | |
| 15:39:12 | sean-k-mooney | dansmith: well the devstack plugin uninstalles the packages and libvirt was compiles with --system | |
| 15:39:34 | efried | sean-k-mooney: Doesn't your plugin just use `make install`? | |
| 15:39:35 | dansmith | sean-k-mooney: clearly not if the old version was still there | |
| 15:39:54 | efried | I would have thought that should have covered symlinking or otherwise replacing versions in any "standard" paths. | |
| 15:40:05 | dansmith | efried: not generally | |
| 15:40:19 | sean-k-mooney | efried: yes but the autogen sript was passed --system so when it invoked configure it should have set it up to instll in the default system lib path | |
| 15:40:27 | dansmith | efried: make install usually puts them in /usr/local/lib unless you tell it otherwise | |
| 15:40:34 | dansmith | or, --system would yeah | |
| 15:40:36 | sean-k-mooney | dansmith: well i dont try an nuke the old verions | |
| 15:41:12 | dansmith | efried: cat /etc/ld.so.conf or /etc/ld.so.conf.d/* | |
| 15:41:19 | dansmith | efried: will show you where the loader looks for libs | |
| 15:41:22 | sean-k-mooney | dansmith: this shoudl do the right thing yes? https://opendev.org/x/devstack-plugin-libvirt-qemu/src/branch/master/devstack/libs/libvirt#L94-L96 | |
| 15:41:32 | dansmith | efried: even on my ubuntu system, /usr/lib/$arch is in there | |
| 15:42:08 | dansmith | sean-k-mooney: should do what? land it in /usr/lib? probably, but he's saying that happened AFAICT | |
| 15:42:14 | efried | ubuntu is what I'm using. And yeah, same. In fact, /usr/lib isn't there at all. | |
| 15:43:04 | sean-k-mooney | /usr/lib is not there at all? | |
| 15:43:24 | efried | stack@breakfast:~/nova$ cat /etc/ld.so.conf.d/* | |
| 15:43:24 | efried | /usr/lib/x86_64-linux-gnu/libfakeroot | |
| 15:43:25 | efried | # libc default configuration | |
| 15:43:25 | efried | /usr/local/lib | |
| 15:43:25 | efried | # Multiarch support | |
| 15:43:25 | efried | /usr/local/lib/x86_64-linux-gnu | |
| 15:43:26 | efried | /lib/x86_64-linux-gnu | |
| 15:43:26 | efried | /usr/lib/x86_64-linux-gnu | |
| 15:45:34 | efried | canori01: I want to say we were working on this, or something very similar, recently, around ignoring *provider* conflicts and just retrying, but it might have been for a different lifecycle operation, I can't remember. Have you looked for an existing launchpad bug? | |
| 15:45:41 | sean-k-mooney | efried: this is what i have http://paste.openstack.org/show/785944/ | |
| 15:47:02 | canori01 | efried: I was lookin at https://bugs.launchpad.net/nova/+bug/1798688 but I'm not 100% sure if it's related | |
| 15:47:02 | openstack | Launchpad bug 1798688 in OpenStack Compute (nova) "AllocationUpdateFailed_Remote: Failed to update allocations for consumer. Error: another process changed the consumer after the report client read the consumer state during the claim" [High,Fix released] - Assigned to Matt Riedemann (mriedem) | |
| 15:47:14 | efried | sean-k-mooney: yeah, that's what I have in /usr/lib as well, point is that the libvirt-bin and n-cpu services were looking in the x86... dir. | |
| 15:47:26 | sean-k-mooney | oh ok | |
| 15:47:57 | efried | sean-k-mooney: iow I assume we should either be installing there, or be blowing those away with symlinks. | |
| 15:48:50 | sean-k-mooney | it might be best to just drop the --system | |
| 15:49:03 | sean-k-mooney | it clearly not doing it write | |