| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-22 | |||
| 17:02:20 | efried | okay, it was worth a shot :P | |
| 17:02:39 | kashyap | efried: Yeah, completely understand | |
| 17:02:47 | kashyap | Can't dig into every mom-n-pop failure :D | |
| 17:03:05 | melwitt | kashyap: I'm just reminded, we'd appreciate your review on this proposed revert https://review.opendev.org/703596 | |
| 17:03:05 | kashyap | This channel is logged, /me should mind his language | |
| 17:03:11 | efried | unfortunately the mom-n-pop failures have been what's killing us, and consuming a bunch of my time. | |
| 17:03:26 | kashyap | melwitt: Hi, /me clicks | |
| 17:03:29 | efried | mostly me floundering around trying to find someone who can help with them. | |
| 17:03:34 | kashyap | efried: :-( | |
| 17:05:07 | kashyap | melwitt: To start with, that commit doesn't even tell why on god's green earth it is trying to revert | |
| 17:05:21 | melwitt | kashyap: take that up with sean-k-mooney lol | |
| 17:05:34 | kashyap | I despise "naked commit messages | |
| 17:05:41 | kashyap | s/commit/commit"/ | |
| 17:05:43 | kashyap | :D | |
| 17:05:44 | melwitt | me too | |
| 17:06:06 | sean-k-mooney | kashyap: it is what you get form the ui by default | |
| 17:06:16 | kashyap | Okay, Sean comments on PS1 | |
| 17:06:20 | sean-k-mooney | but i left some comment on the backport then fielded the reviert | |
| 17:06:36 | kashyap | sean-k-mooney: It just doesn't make sense at all to do CPU comparison check on AArch64 | |
| 17:06:45 | sean-k-mooney | that is incorrect | |
| 17:06:52 | kashyap | melwitt: I documented "why" on the main change | |
| 17:06:53 | sean-k-mooney | we have the info in /proc/cpuinfo | |
| 17:07:07 | sean-k-mooney | so we can check the cpu flags and cpu model | |
| 17:07:10 | kashyap | sean-k-mooney: I relied on the expertise on the libvirt dev who wrote that code. Want to argue with him? | |
| 17:07:19 | sean-k-mooney | sure | |
| 17:07:40 | kashyap | And hell, even AArch64 folks *themselves* told that KVM guests are to be run via 'host-passthrough' | |
| 17:07:49 | sean-k-mooney | yes | |
| 17:08:01 | kashyap | s/told/tell/ | |
| 17:08:02 | sean-k-mooney | but host-passthough means that the guest will see the host cpu flags and model | |
| 17:08:09 | kashyap | Yes, so what? | |
| 17:08:14 | sean-k-mooney | im not arguing that we should not use host-passthough | |
| 17:08:20 | kashyap | That's the whole point in this case | |
| 17:08:34 | sean-k-mooney | im am saying when you do its more important to ensur the flags and modle do not change | |
| 17:08:52 | kashyap | melwitt: sean-k-mooney: Let me quote my message from the main change, since I don't know what people have read or haven't | |
| 17:08:55 | kashyap | [quote] | |
| 17:08:56 | kashyap | So it doesn't make sense to do CPU compatability check on AArch64. And the AArch64 folks themselves recommend that the way to run KVM guests on AArch64 is via 'host-passthrough'. | |
| 17:09:00 | kashyap | It is true that libvirt does not know how to detect host CPU model on AArch64, but even if it _wants_ to know, it cannot, because even the `/proc/cpuinfo` on AArch64 doesn't show anything interesting. There are lots of vendors making different AArch64 CPUs, and they are not easily comparable. They all differ in various ways. (This is also confirmed by Jiri Denemark of libvirt.) | |
| 17:09:06 | kashyap | [/quote] | |
| 17:09:38 | kashyap | sean-k-mooney: Zooming out: what prompted you to revert? | |
| 17:10:08 | sean-k-mooney | kashyap: that may have been true when they look or on a rhel kerenl but /proc/cpuinfo has cpu Flags and the cpu modle name | |
| 17:10:27 | sean-k-mooney | however they have different capitalistation then x86 | |
| 17:10:43 | sean-k-mooney | kashyap: i saw the backport and i consider it to be a regerssion | |
| 17:11:36 | bauzas | stephenfin: so, once https://review.opendev.org/#/c/703568/ do we still have gate problems with the functional tests ? | |
| 17:12:16 | sean-k-mooney | kashyap: there have been some patches to lscpu to allow it to parse the different way the flags and model are reported on aarch64. i suspect that libvirt just need to have the same change applied to its parsing | |
| 17:12:31 | sean-k-mooney | kashyap: well actully the lscpu patch merged back in 2018 | |
| 17:19:28 | kashyap | My brain is starved of food, and my chest is aching after this bizarre bike fall. So I can't parse you fully yet | |
| 17:19:47 | kashyap | melwitt: sean-k-mooney What I know we shouldn't merge that revert, IMHO | |
| 17:20:19 | melwitt | thanks kashyap | |
| 17:20:43 | kashyap | melwitt: sean-k-mooney: Until libvirt gets the parsing correct and libvirt/QEMU CPU modelling maintainers give guidance, we should stick with current Nova behaviour. | |
| 17:21:27 | sean-k-mooney | i dissagree with that but if im being over ruled fine | |
| 17:22:30 | sean-k-mooney | with the current patch its likely that after a migration if the guess is rebooted the cpu flags and model might change | |
| 17:23:27 | sean-k-mooney | its also not clear to me if there is a risk of the guest crashign if it trys to use a instuction not supported on the dest host | |
| 17:24:26 | sean-k-mooney | given the info regarding cpu model and cpu flags is availabel in /proc/cpuinfo a correct fix in my view would have been to validate those | |
| 17:25:01 | sean-k-mooney | i walso woudl have expected a workaround config optin for this but im not planing on spending time to code that up | |
| 17:27:40 | kashyap | sean-k-mooney: All this requires extensive testing, and extremely careful audit - we haven't done any of that (yet). So we go with "solid heuristics". | |
| 17:28:22 | sean-k-mooney | solid heuristics of check nothing and hope it works? | |
| 17:28:57 | kashyap | The heuristic here being, "avoid needless checks" | |
| 17:29:44 | sean-k-mooney | we would not consider it safe to not check cpu compatabily on x86. its just seam wrong to have a lower standard for other archs | |
| 17:29:56 | sean-k-mooney | anyway i better go back to backporting | |
| 17:34:27 | sean-k-mooney | stephenfin: can you review https://review.opendev.org/#/c/701601 again when you have a chance | |
| 17:42:20 | stephenfin | sean-k-mooney: done | |
| 17:42:48 | sean-k-mooney | stephenfin: thanks | |
| 17:43:30 | sean-k-mooney | stephenfin: do you know if jan is about. he is ment to be moving this week right to start his new job next week | |
| 17:43:49 | stephenfin | bauzas: We shouldn't have, no. The issues are to do with Tempest tests, iirc (something to do with pip not uninstalling pyyaml since it was installed by the package manager) | |
| 17:44:06 | sean-k-mooney | that is being fix | |
| 17:44:07 | bauzas | all good, will recheck then | |
| 17:44:18 | stephenfin | not so fast, I don't think the fix has merged | |
| 17:44:18 | sean-k-mooney | or has been fixed | |
| 17:44:40 | stephenfin | bauzas: https://review.opendev.org/#/c/703735/ | |
| 17:44:57 | sean-k-mooney | you can add a depends on and then recheck | |
| 17:45:02 | bauzas | oh ok | |
| 17:45:47 | sean-k-mooney | that was breaking os-vif as well hence why i want to get that os-vif change merged so we nolonger have legacy jobs for it to break | |
| 17:48:53 | stephenfin | ah, so that's the dependency | |
| 17:48:55 | stephenfin | gtk | |
| 18:14:02 | openstackgerrit | Merged openstack/nova stable/stein: libvirt: remove conditional on VIR_DOMAIN_EVENT_SUSPENDED_POSTCOPY https://review.opendev.org/700773 | |
| 18:14:08 | openstackgerrit | Merged openstack/nova stable/stein: libvirt: check job status for VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED event https://review.opendev.org/700774 | |
| 18:31:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Cleanup of '_test_resize' helper test https://review.opendev.org/664245 | |
| 18:31:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Recalculate 'RequestSpec.numa_topology' on resize https://review.opendev.org/662522 | |
| 18:35:48 | artom | stephenfin, oh, you've picked that up again? ^^ | |
| 18:36:14 | artom | IIRC you handed it off to me, and then I looked it, shrugged my shoulders in despair, and never touched it again :/ | |
| 18:36:18 | stephenfin | artom: I'm getting hassled downstream for it and figured you didn't have bandwidth, so yeah :) | |
| 18:36:23 | stephenfin | all good | |
| 18:36:57 | artom | Yeah, sorry. There was attempt, the Gerrit comments are evidence of that. | |
| 18:38:11 | sean-k-mooney | stephenfin: i can hassel you upstrem for it too if you like | |
| 18:38:43 | sean-k-mooney | i wanted that to be in 16GA | |
| 18:38:52 | sean-k-mooney | well in train | |
| 18:38:57 | sean-k-mooney | same thing | |
| 18:39:25 | stephenfin | sean-k-mooney: https://media0.giphy.com/media/R7IYpzLLMBomk/giphy.gif?cid=790b7611c0d748cc03dc8e8f5456f63a200cd89cd6b5f14f&rid=giphy.gif | |
| 18:40:22 | sean-k-mooney | i think you still being working and only at 18:40 is enough pressue to make you fix anything | |
| 18:41:01 | sean-k-mooney | /only/online/ | |
| 18:44:52 | stephenfin | heh, yeah, I'll be done shortly | |
| 18:45:38 | artom | stephenfin, oh, I remember where I blocked - if we update the request spec with the new numa_topology, but the resize fails in one of a myriad of ways, and we need to revert it | |
| 18:46:04 | artom | IIRC we talked with dansmith about not persisting the numa_topology at all, and just making sure each request to the scheduler has the correct one | |
| 18:46:23 | stephenfin | artom: Just looked into that. FWICT, we don't revert the RequestSpec.flavor, let alone the RequestSpec.numa_topolology, in the case of a failure | |
| 18:46:36 | stephenfin | So I need another follow-up to do that | |
| 18:46:50 | artom | stephenfin, ah yeah, the flavor was in the same boat | |
| 18:48:35 | openstack | Launchpad bug 1831771 in OpenStack Compute (nova) "UnexpectedDeletingTaskStateError exception can leave traces of VIFs on host" [Medium,In progress] - Assigned to Matthew Booth (mbooth-9) | |
| 18:48:35 | sean-k-mooney | oh ye are talking about a different bug. i was refering to https://bugs.launchpad.net/nova/+bug/1831771 | |
| 18:49:57 | sean-k-mooney | sorry i saw you being ping about that downstream so assume that was the one you were refering too | |
| 18:52:58 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Add emulated TPM support to Nova https://review.opendev.org/631363 | |