| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-15 | |||
| 19:56:26 | mriedem | so...we need to configure the guest with a timer element https://libvirt.org/formatdomain.html#elementsTime | |
| 19:56:46 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Handle volume API failure in _post_live_migration https://review.openstack.org/609517 | |
| 19:56:53 | mriedem | i guess it already does, but always sets it to false | |
| 19:58:41 | dansmith | imacdonn: you're going to love me | |
| 19:58:46 | imacdonn | uh oh | |
| 19:58:49 | dansmith | imacdonn: but I -1d for a single character change | |
| 19:59:01 | dansmith | imacdonn: however, fix it fast and I shall stand by for +2ing | |
| 19:59:25 | imacdonn | dang .. ok | |
| 19:59:55 | mriedem | jackding: i guess it's not clear to me how the libvirt driver will determine that the underlying hypervisor supports hpet | |
| 20:00:10 | mriedem | is it just assumed that all x86 hosts support hpet? | |
| 20:00:46 | openstackgerrit | iain MacDonnell proposed openstack/nova master: Handle online_data_migrations exceptions https://review.openstack.org/608091 | |
| 20:00:47 | artom | Back when I was compiling my own kernels this was an option you could disable, so I don't think we should assume that | |
| 20:01:23 | imacdonn | dansmith: how fast is fast? :) | |
| 20:01:26 | dansmith | imacdonn: that fast | |
| 20:01:29 | imacdonn | cool | |
| 20:01:54 | dansmith | artom: correct | |
| 20:01:57 | jackding | in our implementation we assume x86 support hpet | |
| 20:02:06 | mriedem | blech | |
| 20:02:15 | mriedem | this is when i need kashyap | |
| 20:02:55 | mriedem | i realize windriver can be super opinionated on the stuff they deploy, but not so much upstream | |
| 20:04:08 | dansmith | mriedem: if all x86 could be assumed to have it, you wouldn't need this trait, just require arch=x86 | |
| 20:05:53 | mriedem | good point | |
| 20:08:36 | artom | So, hardware-wise, it's apparently as easy as [artom@jayne ~]$ cat /sys/devices/system/clocksource/clocksource0/available_clocksource | |
| 20:08:36 | artom | tsc hpet acpi_pm | |
| 20:08:46 | artom | But. Do we need anything from libvirt? | |
| 20:09:34 | mriedem | i was hoping something would be in the host capabilities API | |
| 20:09:35 | mriedem | http://logs.openstack.org/39/587439/4/check/tempest-full/fdb1cf9/controller/logs/screen-n-cpu.txt.gz#_Sep_11_14_44_27_658449 | |
| 20:09:37 | mriedem | but i don't see that | |
| 20:10:28 | mriedem | or maybe hpet just doesn't show up there | |
| 20:10:33 | mriedem | ? | |
| 20:10:36 | mriedem | because it's nested virt? | |
| 20:10:55 | dansmith | mriedem: I don't see it on my physical box either | |
| 20:11:07 | mriedem | physical box is kvm or qemu? | |
| 20:11:29 | mriedem | not sure if that matters | |
| 20:11:32 | artom | The SEV guys had to dig into a commit to see how libvirt reports SEV | |
| 20:11:34 | artom | (https://libvirt.org/git/?p=libvirt.git;a=commit;h=6688393c6b222b5d7cba238f21d55134611ede9c) | |
| 20:12:01 | dansmith | mriedem: physical box meaning baremetal | |
| 20:12:17 | artom | Their domain capabilities docs aren't exchaustive :/ https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectGetDomainCapabilities | |
| 20:14:27 | cfriesen | mriedem: I think HPET is supported on x86 since like 2005. | |
| 20:14:45 | artom | So, on a physical machine with an HPET, virsh capabilities doesn't include any mention of it | |
| 20:15:18 | cfriesen | on qemu at least, my understanding is that HPET is always emulated using other clocksources | |
| 20:15:56 | dansmith | cfriesen: is that's the case, then what is the point of the trait? | |
| 20:16:23 | dansmith | if there aren't any x86-capable hypervisors that don't support HPET we shouldn't need it at all right? | |
| 20:17:05 | dansmith | presumably if you don't have HPET enabled on your kernel and you're using libvirt for lxc, then just keying off of "is x86" isn't enough | |
| 20:17:12 | mriedem | artom: yeah this was close to the same thing https://www.reddit.com/r/VFIO/comments/5g6rxb/a_fix_for_windows_vm_performancestuttering_when/ | |
| 20:17:21 | cfriesen | dansmith: I'm told that some operators might not want to enable HPET for all guests for security reasons. | |
| 20:17:49 | cfriesen | dansmith: so I was kind of assuming that we'd have a hypervisor specific config option that when turned off would prevent the trait from being advertised | |
| 20:17:53 | dansmith | cfriesen: so what's the key? | |
| 20:17:54 | dansmith | dear god | |
| 20:18:11 | dansmith | I think we're maybe uncovering that this is a super tiny line item on one person's wish list :) | |
| 20:18:25 | mriedem | so we're back to my original question in the spec, which is some external thing can set hpet on the compute node resource provider | |
| 20:19:12 | dansmith | mriedem: I don't remember where we landed with efried on being able to set custom traits on compute-managed providers | |
| 20:19:23 | mriedem | this is a standard trait | |
| 20:19:24 | dansmith | I thought the current thinking was that those would get wiped | |
| 20:19:27 | mriedem | or being proposed as one | |
| 20:19:32 | dansmith | not what I meant | |
| 20:19:32 | mriedem | we'll merge them | |
| 20:19:35 | dansmith | I don't mean custom I mean random | |
| 20:19:47 | cfriesen | dansmith: the note at the PTG was literally "some way to request an HPET in the guest for x86 guests" :) | |
| 20:19:47 | mriedem | things that the virt driver reports + custom set external things get merged | |
| 20:19:48 | dansmith | meaning set by the operator, unknown to the compute service | |
| 20:19:48 | efried | It's the issue of "trait ownership" | |
| 20:20:01 | dansmith | efried: right | |
| 20:20:16 | efried | It sucks, but basically we need to decide whether the HPET trait is owned by compute or not. | |
| 20:20:20 | mriedem | also discussed at some length related to https://review.openstack.org/#/c/538498/ | |
| 20:20:23 | efried | If it is, then the user *can't* set it. | |
| 20:20:32 | efried | if they do, and compute disagrees, compute will wipe it. | |
| 20:20:47 | efried | and vice versa. | |
| 20:21:05 | dansmith | right, so did we end up being able to declare some as compute-manage and some not, on a given provider? | |
| 20:21:12 | mriedem | it sounds like the only way the libvirt driver can know is if it execs "cat /sys/devices/system/clocksource/clocksource0/available_clocksource" on startup | |
| 20:21:12 | efried | unless we figure some override in the (as yet vaporware) provider customization file... | |
| 20:21:38 | mriedem | and greps for hpet | |
| 20:21:39 | efried | dansmith: Otherwise, it's basically just hardcoded, and we would have to document it somewhere. | |
| 20:21:44 | dansmith | mriedem: even still cfriesen is arguing that it's an operator decision, not a property of the host | |
| 20:21:54 | cfriesen | mriedem: that's whether the host kernel has the hpet clocksource available. I don't think that impacts whether it can provide an emulated HPET to the guest. | |
| 20:21:58 | efried | except for custom traits, which we can namespace like we talked about in denver. | |
| 20:22:16 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: Use tempfile for powervm config drive https://review.openstack.org/610174 | |
| 20:22:22 | mriedem | namespace custom traits? | |
| 20:22:27 | mriedem | i thought CUSTOM_* was it? | |
| 20:22:43 | mriedem | dansmith: yeah b/c some ops want to disable it... | |
| 20:23:10 | dansmith | yeah, so I definitely thought this was going to be an actual capability, not just a "is this turned on" sort of thing when we last discussed it | |
| 20:23:26 | dansmith | not even is it turned on, but is it in policy for this host | |
| 20:24:03 | efried | mriedem: namespaced to signify the owner. Like if the trait.startswith('CUSTOM_NOVA_') then the op isn't allowed to touch it. Otherwise, fair game. | |
| 20:24:17 | efried | ...until CUSTOM_CYBORG_ is a thing, etc. | |
| 20:24:51 | efried | but at least then we have to have a short hardcoded list of possible owners and their namespaces, rather than a long hardcoded list of individual traits (or trait regexes) that are system-owned. | |
| 20:24:59 | mriedem | are there any cases where we'll have CUSTOM_NOVA_* traits that aren't just standard traits in os-traits? | |
| 20:25:23 | mriedem | CUSTOM_NOVA_* sounds pretty not custom to me | |
| 20:25:25 | efried | mriedem: An example would be a trait signifying the PCI address of a device. | |
| 20:26:13 | mriedem | is that a trait? | |
| 20:26:21 | efried | it's not a *capability* | |
| 20:26:24 | efried | But it is a *trait*. | |
| 20:26:26 | efried | Dammit. | |
| 20:26:31 | mriedem | it's a....thing | |
| 20:27:18 | mriedem | what's the point of that being a trait? so we can schedule to a specific PCI address? | |
| 20:27:18 | cfriesen | is it schedulable? | |
| 20:27:25 | efried | yes | |
| 20:27:30 | efried | pets, baby. | |
| 20:27:36 | mriedem | huh | |
| 20:27:48 | efried | more realistically, trait for vendor/product ID. | |
| 20:27:48 | cfriesen | but can't the IOMMU redirect to whatever virtual address is desired? | |