| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-11 | |||
| 15:22:21 | efried | jroll: btw, there was a question as to whether we needed to bother providing the non-encrypted version of vTPM support at all. | |
| 15:22:23 | jroll | older libvirt won't let me create a vtpm secret type ¯\_(ツ)_/¯ | |
| 15:22:44 | efried | right, that needs to be >=5.6.0, which afaict you have to compile yourself. | |
| 15:22:46 | jroll | efried: I'd be pro-not-supporting that, I don't like footguns | |
| 15:22:51 | sean-k-mooney | e.g. i dont think a secreate for tls or tpm is a different thing internally in libvirt | |
| 15:23:09 | efried | penick indicated the same. Just that the requirements are going to be way more stringent for it if so. | |
| 15:23:52 | sean-k-mooney | for what its worth rhel/fedora will be shiping it relitvly quickly | |
| 15:23:59 | sean-k-mooney | im not sure when exactly | |
| 15:24:08 | sean-k-mooney | but we have an advance virt stream we use for openstack | |
| 15:24:17 | efried | I guess "with ussuri" is soon enough <shrug> | |
| 15:24:30 | sean-k-mooney | that ships a newer version then base rhel | |
| 15:24:37 | efried | but also the barbican requirement | |
| 15:25:00 | efried | anyway, glad you agree jroll, it makes the code & test surface easier. | |
| 15:25:37 | efried | lyarwood: since you mentioned the qemu side -- do you have any idea what's the minimum qemu version to support encrypted vTPM? I couldn't find that in the docs anywhere. | |
| 15:26:56 | efried | canori01: it would help to know why the allocation update is failing (out of resources? provider conflict? other?), and what action is being performed when that happens. | |
| 15:28:38 | canori01 | efried: I am replacing old hardware. So I powered off the old hypervisor and then did a nova-evacuate. All the guests evacuated except for two which gave me the provider conflict | |
| 15:29:03 | lyarwood | efried: I don't sorry | |
| 15:29:31 | lyarwood | efried: 2.12 is listed in the Stein spec FWIW | |
| 15:30:02 | efried | lyarwood: yeah, that was for non-encrypted, and idk if there's a higher req for encrypted. | |
| 15:30:23 | efried | canori01: If it was a provider conflict, you should just be able to "try again". | |
| 15:30:33 | lyarwood | efried: no idea sorry, kashyap is out today otherwise I'd ask him if he could work that out for us. | |
| 15:30:49 | efried | ah, good plan. I'll stalk him, thanks. | |
| 15:30:52 | sean-k-mooney | efried: tpm emulation was added in 2.11 but im not sure if that included encryption | |
| 15:31:10 | efried | it did not | |
| 15:31:39 | efried | But I don't even know whether qemu required any update for that aspect. It might not have. | |
| 15:31:49 | sean-k-mooney | also this is an intersting site https://repology.org/project/libvirt/versions | |
| 15:33:22 | sean-k-mooney | look like apline has the depencyies you need but very little else at teh moment | |
| 15:34:28 | sean-k-mooney | Fedora Rawhide would "work" too although i suspect nova wont be happy with it in general | |
| 15:35:06 | efried | sean-k-mooney: oh, that reminds me, I ran into another weird quirk with the home-built libvirt... | |
| 15:35:49 | efried | sean-k-mooney: When I restarted the libvirt service, it blew up because it was looking in /usr/lib/x86_64-linux-gnu/ which contained the old libvirt 4.xx | |
| 15:36:06 | efried | I had to go into the service and set LD_LIBRARY_PATH=/usr/lib explicitly to make it work. | |
| 15:36:11 | efried | which seems... weird. | |
| 15:36:17 | efried | had to do the same with n-cpu. | |
| 15:36:52 | sean-k-mooney | ya that does seam strange | |
| 15:37:02 | sean-k-mooney | well clearly i have more work to do | |
| 15:37:56 | dansmith | efried: why is that surprising? | |
| 15:38:19 | dansmith | efried: if you have an old version installed in the standard system library path, and start a new daemon binary that requires the new libs, | |
| 15:38:24 | dansmith | you have to tell it where they are | |
| 15:38:33 | dansmith | (if I'm understanding what you're saying) | |
| 15:38:36 | efried | dansmith: Is /usr/lib/x86_64-linux-gnu/ the "standard system library path"? | |
| 15:38:47 | efried | That was the surprising part to me. I would have thought /usr/lib was. | |
| 15:39:08 | dansmith | efried: for multi-arch stuff on fedora-like distros it is I think yeah | |
| 15:39:12 | sean-k-mooney | dansmith: well the devstack plugin uninstalles the packages and libvirt was compiles with --system | |
| 15:39:34 | efried | sean-k-mooney: Doesn't your plugin just use `make install`? | |
| 15:39:35 | dansmith | sean-k-mooney: clearly not if the old version was still there | |
| 15:39:54 | efried | I would have thought that should have covered symlinking or otherwise replacing versions in any "standard" paths. | |
| 15:40:05 | dansmith | efried: not generally | |
| 15:40:19 | sean-k-mooney | efried: yes but the autogen sript was passed --system so when it invoked configure it should have set it up to instll in the default system lib path | |
| 15:40:27 | dansmith | efried: make install usually puts them in /usr/local/lib unless you tell it otherwise | |
| 15:40:34 | dansmith | or, --system would yeah | |
| 15:40:36 | sean-k-mooney | dansmith: well i dont try an nuke the old verions | |
| 15:41:12 | dansmith | efried: cat /etc/ld.so.conf or /etc/ld.so.conf.d/* | |
| 15:41:19 | dansmith | efried: will show you where the loader looks for libs | |
| 15:41:22 | sean-k-mooney | dansmith: this shoudl do the right thing yes? https://opendev.org/x/devstack-plugin-libvirt-qemu/src/branch/master/devstack/libs/libvirt#L94-L96 | |
| 15:41:32 | dansmith | efried: even on my ubuntu system, /usr/lib/$arch is in there | |
| 15:42:08 | dansmith | sean-k-mooney: should do what? land it in /usr/lib? probably, but he's saying that happened AFAICT | |
| 15:42:14 | efried | ubuntu is what I'm using. And yeah, same. In fact, /usr/lib isn't there at all. | |
| 15:43:04 | sean-k-mooney | /usr/lib is not there at all? | |
| 15:43:24 | efried | stack@breakfast:~/nova$ cat /etc/ld.so.conf.d/* | |
| 15:43:24 | efried | /usr/lib/x86_64-linux-gnu/libfakeroot | |
| 15:43:25 | efried | # libc default configuration | |
| 15:43:25 | efried | /usr/local/lib | |
| 15:43:25 | efried | # Multiarch support | |
| 15:43:25 | efried | /usr/local/lib/x86_64-linux-gnu | |
| 15:43:26 | efried | /lib/x86_64-linux-gnu | |
| 15:43:26 | efried | /usr/lib/x86_64-linux-gnu | |
| 15:45:34 | efried | canori01: I want to say we were working on this, or something very similar, recently, around ignoring *provider* conflicts and just retrying, but it might have been for a different lifecycle operation, I can't remember. Have you looked for an existing launchpad bug? | |
| 15:45:41 | sean-k-mooney | efried: this is what i have http://paste.openstack.org/show/785944/ | |
| 15:47:02 | canori01 | efried: I was lookin at https://bugs.launchpad.net/nova/+bug/1798688 but I'm not 100% sure if it's related | |
| 15:47:02 | openstack | Launchpad bug 1798688 in OpenStack Compute (nova) "AllocationUpdateFailed_Remote: Failed to update allocations for consumer. Error: another process changed the consumer after the report client read the consumer state during the claim" [High,Fix released] - Assigned to Matt Riedemann (mriedem) | |
| 15:47:14 | efried | sean-k-mooney: yeah, that's what I have in /usr/lib as well, point is that the libvirt-bin and n-cpu services were looking in the x86... dir. | |
| 15:47:26 | sean-k-mooney | oh ok | |
| 15:47:57 | efried | sean-k-mooney: iow I assume we should either be installing there, or be blowing those away with symlinks. | |
| 15:48:50 | sean-k-mooney | it might be best to just drop the --system | |
| 15:49:03 | sean-k-mooney | it clearly not doing it write | |
| 15:49:22 | sean-k-mooney | then it will install in /usr/local/lib which is in the lib path | |
| 15:49:43 | efried | I want to say other things blew up when I tried that | |
| 15:50:30 | efried | anyway, hopefully this is all moot irl. Unless I want to try to run a CI job around this thing. And I'm not sure if there's going to be value in that. | |
| 15:50:51 | sean-k-mooney | well ill try and spend some more time on this and see if i can make it a bit more robost. | |
| 15:52:21 | efried | sean-k-mooney: I appreciate that. Though it's probably not worth trying to make it perfect. I think the sweet spot will be having it work in devstack CI (again, if I end up wanting to do something there), which is of course a pretty narrow subset of usages. | |
| 15:53:13 | canori01 | efried: Is there a way, I coudl just force the instance to take a new allocation and spawn somewhere? I have plenty of spare capacity | |
| 15:53:13 | efried | ...excluding, among other things, "works in my dirty devstack, containing who-knows-what packages, that I've restacked a zillion times" | |
| 15:53:23 | sean-k-mooney | efried: ya i would like to get a ci job in that plugin repo at a minium but i have need to use this plugin alot in the past when enableing things for openstack so i want it to be atleast usable for that | |
| 15:53:36 | efried | canori01: At this point you should still be able to evacuate it, no? | |
| 15:54:06 | efried | oh, you mean YOU have use cases too?? Well, then, by all means :P | |
| 15:54:20 | sean-k-mooney | well not currently | |
| 15:54:53 | sean-k-mooney | i orginally wrote the first version of that plugin back in 2014 | |
| 15:55:06 | sean-k-mooney | i have used it for at least 4 feature at this point | |
| 15:55:07 | canori01 | efried: I've tried resetting the state of the instance and evacuating again, but it consistently gives me this http://paste.openstack.org/show/785945/ | |
| 15:55:21 | sean-k-mooney | so i like keping it working for when i need it | |
| 15:56:12 | efried | canori01: Oh, hm, that's a reproducible failure? That is not expected. I think we need a bug. | |
| 15:56:55 | canori01 | ok, thanks | |
| 15:57:24 | sean-k-mooney | canori01: you dont have two copies of the nova compute agent running on the compute node by any chacce? | |
| 15:58:08 | sean-k-mooney | oh never mind | |
| 15:58:13 | sean-k-mooney | this is in the scduler | |
| 16:07:47 | dansmith | cdent: don't we have some guideline around not making fundamental api behaviors change based on policy or config? (well, I know config) | |
| 16:08:40 | dansmith | cdent: meaning something like a user is granted some policy element, which enables a deep feature that they don't even know about nor can detect, which makes all calls to a specific API suddenly return 202 and be async, where normally they are 200 and sync | |
| 16:08:43 | cdent | dansmith: there's a "what should cause a microversion bump" guideline somewhere that I can dig up. Is that what you're thinkin gof? | |
| 16:09:01 | dansmith | cdent: and two users with differing policies for this element would see the different sync/async behavior | |
| 16:09:23 | cdent | I don't know of anythign with that level of detail, but that sounds like a definite no no | |