| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-15 | |||
| 18:49:35 | openstackgerrit | Merged openstack/nova master: Add some more docs for upgrade checkers https://review.openstack.org/604486 | |
| 18:57:10 | mriedem | Kevin_Zheng: i replied to your -1 in https://review.openstack.org/#/c/608777/ - are you still -1 on that or can it be dropped? | |
| 19:04:21 | mriedem | this seems...excessive: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22detaching%20network%20adapter%20failed.%3A%20libvirtError%3A%20internal%20error%3A%20End%20of%20file%20from%20qemu%20monitor%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d | |
| 19:07:32 | mriedem | oh i see, | |
| 19:07:54 | mriedem | we're tearing down, we get a network-vif-deleted event right before the guest is deleted in the hypervisor | |
| 19:08:30 | mriedem | doesn't "End of file from qemu monitor" usually indicate the guest is gone? | |
| 19:13:19 | openstackgerrit | Jack Ding proposed openstack/nova-specs master: High Precision Event Timer (HPET) on x86 guests https://review.openstack.org/607989 | |
| 19:29:12 | artom | mriedem, ah, it's because my poor man's mock checks for volume_id, but is called with an attachment_id. | |
| 19:29:20 | artom | So it doesn't actually raise | |
| 19:30:56 | mriedem | yeah... | |
| 19:31:08 | mriedem | so track call count and raise count? | |
| 19:38:31 | mriedem | jackding: just a couple of small updates in that spec | |
| 19:38:33 | mriedem | for hpet | |
| 19:38:59 | mriedem | btw, how does the libvirt driver know it can report the hpet trait? | |
| 19:39:03 | mriedem | something on the host capabilities? | |
| 19:40:10 | dansmith | I think so yeah | |
| 19:47:06 | imacdonn | dansmith mriedem efried: please re-review this - would like to get it wrapped up, so I can look at applying the fix to cinder too.... https://review.openstack.org/608091 | |
| 19:47:33 | efried | imacdonn: My hands are off that one. It's down to dansmith and mriedem. | |
| 19:48:01 | imacdonn | efried: OK. The logic did change a bit, so thought you might want to look again | |
| 19:48:07 | dansmith | I've had it open in the queue | |
| 19:48:19 | dansmith | I keep coming up with excuses :) | |
| 19:48:22 | imacdonn | :) | |
| 19:48:43 | efried | imacdonn: I would, if I weren't completely underwater, or if there weren't already two cores on it who are way more familiar with the domain. | |
| 19:49:04 | imacdonn | efried: understood - thanks! | |
| 19:51:20 | jackding | mriedem: I'm not familiar with traits, but I thought that's something virt driver could query the host from. | |
| 19:53:55 | mriedem | jackding: efried: well efried said the virt driver would report the hpet trait, | |
| 19:54:05 | mriedem | but looking at the nova code change, i don't see it doing anything like that | |
| 19:54:16 | efried | mriedem: The code change hasn't caught up yet. | |
| 19:54:42 | efried | What I said earlier was based on the fact that the virt driver code is doing the error checking. | |
| 19:56:26 | mriedem | so...we need to configure the guest with a timer element https://libvirt.org/formatdomain.html#elementsTime | |
| 19:56:46 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Handle volume API failure in _post_live_migration https://review.openstack.org/609517 | |
| 19:56:53 | mriedem | i guess it already does, but always sets it to false | |
| 19:58:41 | dansmith | imacdonn: you're going to love me | |
| 19:58:46 | imacdonn | uh oh | |
| 19:58:49 | dansmith | imacdonn: but I -1d for a single character change | |
| 19:59:01 | dansmith | imacdonn: however, fix it fast and I shall stand by for +2ing | |
| 19:59:25 | imacdonn | dang .. ok | |
| 19:59:55 | mriedem | jackding: i guess it's not clear to me how the libvirt driver will determine that the underlying hypervisor supports hpet | |
| 20:00:10 | mriedem | is it just assumed that all x86 hosts support hpet? | |
| 20:00:46 | openstackgerrit | iain MacDonnell proposed openstack/nova master: Handle online_data_migrations exceptions https://review.openstack.org/608091 | |
| 20:00:47 | artom | Back when I was compiling my own kernels this was an option you could disable, so I don't think we should assume that | |
| 20:01:23 | imacdonn | dansmith: how fast is fast? :) | |
| 20:01:26 | dansmith | imacdonn: that fast | |
| 20:01:29 | imacdonn | cool | |
| 20:01:54 | dansmith | artom: correct | |
| 20:01:57 | jackding | in our implementation we assume x86 support hpet | |
| 20:02:06 | mriedem | blech | |
| 20:02:15 | mriedem | this is when i need kashyap | |
| 20:02:55 | mriedem | i realize windriver can be super opinionated on the stuff they deploy, but not so much upstream | |
| 20:04:08 | dansmith | mriedem: if all x86 could be assumed to have it, you wouldn't need this trait, just require arch=x86 | |
| 20:05:53 | mriedem | good point | |
| 20:08:36 | artom | tsc hpet acpi_pm | |
| 20:08:36 | artom | So, hardware-wise, it's apparently as easy as [artom@jayne ~]$ cat /sys/devices/system/clocksource/clocksource0/available_clocksource | |
| 20:08:46 | artom | But. Do we need anything from libvirt? | |
| 20:09:34 | mriedem | i was hoping something would be in the host capabilities API | |
| 20:09:35 | mriedem | http://logs.openstack.org/39/587439/4/check/tempest-full/fdb1cf9/controller/logs/screen-n-cpu.txt.gz#_Sep_11_14_44_27_658449 | |
| 20:09:37 | mriedem | but i don't see that | |
| 20:10:28 | mriedem | or maybe hpet just doesn't show up there | |
| 20:10:33 | mriedem | ? | |
| 20:10:36 | mriedem | because it's nested virt? | |
| 20:10:55 | dansmith | mriedem: I don't see it on my physical box either | |
| 20:11:07 | mriedem | physical box is kvm or qemu? | |
| 20:11:29 | mriedem | not sure if that matters | |
| 20:11:32 | artom | The SEV guys had to dig into a commit to see how libvirt reports SEV | |
| 20:11:34 | artom | (https://libvirt.org/git/?p=libvirt.git;a=commit;h=6688393c6b222b5d7cba238f21d55134611ede9c) | |
| 20:12:01 | dansmith | mriedem: physical box meaning baremetal | |
| 20:12:17 | artom | Their domain capabilities docs aren't exchaustive :/ https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectGetDomainCapabilities | |
| 20:14:27 | cfriesen | mriedem: I think HPET is supported on x86 since like 2005. | |
| 20:14:45 | artom | So, on a physical machine with an HPET, virsh capabilities doesn't include any mention of it | |
| 20:15:18 | cfriesen | on qemu at least, my understanding is that HPET is always emulated using other clocksources | |
| 20:15:56 | dansmith | cfriesen: is that's the case, then what is the point of the trait? | |
| 20:16:23 | dansmith | if there aren't any x86-capable hypervisors that don't support HPET we shouldn't need it at all right? | |
| 20:17:05 | dansmith | presumably if you don't have HPET enabled on your kernel and you're using libvirt for lxc, then just keying off of "is x86" isn't enough | |
| 20:17:12 | mriedem | artom: yeah this was close to the same thing https://www.reddit.com/r/VFIO/comments/5g6rxb/a_fix_for_windows_vm_performancestuttering_when/ | |
| 20:17:21 | cfriesen | dansmith: I'm told that some operators might not want to enable HPET for all guests for security reasons. | |
| 20:17:49 | cfriesen | dansmith: so I was kind of assuming that we'd have a hypervisor specific config option that when turned off would prevent the trait from being advertised | |
| 20:17:53 | dansmith | cfriesen: so what's the key? | |
| 20:17:54 | dansmith | dear god | |
| 20:18:11 | dansmith | I think we're maybe uncovering that this is a super tiny line item on one person's wish list :) | |
| 20:18:25 | mriedem | so we're back to my original question in the spec, which is some external thing can set hpet on the compute node resource provider | |
| 20:19:12 | dansmith | mriedem: I don't remember where we landed with efried on being able to set custom traits on compute-managed providers | |
| 20:19:23 | mriedem | this is a standard trait | |
| 20:19:24 | dansmith | I thought the current thinking was that those would get wiped | |
| 20:19:27 | mriedem | or being proposed as one | |
| 20:19:32 | mriedem | we'll merge them | |
| 20:19:32 | dansmith | not what I meant | |
| 20:19:35 | dansmith | I don't mean custom I mean random | |
| 20:19:47 | mriedem | things that the virt driver reports + custom set external things get merged | |
| 20:19:47 | cfriesen | dansmith: the note at the PTG was literally "some way to request an HPET in the guest for x86 guests" :) | |
| 20:19:48 | efried | It's the issue of "trait ownership" | |
| 20:19:48 | dansmith | meaning set by the operator, unknown to the compute service | |
| 20:20:01 | dansmith | efried: right | |
| 20:20:16 | efried | It sucks, but basically we need to decide whether the HPET trait is owned by compute or not. | |
| 20:20:20 | mriedem | also discussed at some length related to https://review.openstack.org/#/c/538498/ | |
| 20:20:23 | efried | If it is, then the user *can't* set it. | |
| 20:20:32 | efried | if they do, and compute disagrees, compute will wipe it. | |
| 20:20:47 | efried | and vice versa. | |
| 20:21:05 | dansmith | right, so did we end up being able to declare some as compute-manage and some not, on a given provider? | |
| 20:21:12 | efried | unless we figure some override in the (as yet vaporware) provider customization file... | |
| 20:21:12 | mriedem | it sounds like the only way the libvirt driver can know is if it execs "cat /sys/devices/system/clocksource/clocksource0/available_clocksource" on startup | |