| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-13 | |||
| 16:08:34 | melwitt | awesome! | |
| 16:09:35 | kashyap | melwitt: I know the mechanics of how QEMU handles things; but the innards of AArch64 and the usage is Greek for me too. (I find Greek aesthetically pleasing, though. I have a couple of books with Greek on left, the English translation on right. :D) | |
| 16:09:48 | melwitt | :) | |
| 16:19:03 | melwitt | kashyap: is this libvirt kvm aarch64 (which is mentioned on feature support matrix)? or is it libvirt qemu aarch64 (not on feature support matrix yet)? https://docs.openstack.org/nova/train/user/support-matrix.html | |
| 16:19:33 | kashyap | melwitt: Yeah, there was two things: | |
| 16:19:56 | kashyap | s/was/are/ | |
| 16:21:42 | kashyap | (1) TCG (the emulatd bits), or what is also referred to as "QEMU"-only; and (2) KVM (with hardware accelerated) in context of AArch64. I guess we want 'care' about both | |
| 16:22:04 | kashyap | Luckily, Peter from QEMU informs that for _both TCG/KVM, we can just use one model: 'max' | |
| 16:22:12 | kashyap | (Comments on why upcoming...) | |
| 16:22:32 | melwitt | kashyap: ok, so the proposed patches aren't for one in particular only | |
| 16:23:13 | kashyap | melwitt: Although for CI, perhaps the dev just cares about QEMU (TCG). Just like how Nova x86 CI runs on TCG guests (because no nested, to get KVM). | |
| 16:23:25 | kashyap | Not sure if I'm helping or confusing :D | |
| 16:23:40 | melwitt | uh ... helping a little :) but it's not you, it's me | |
| 16:23:59 | melwitt | I'm just trying to write something on the lp bug for the patch above the one we've been looking at | |
| 16:24:57 | sean-k-mooney | kashyap: what is the max model usef for? | |
| 16:24:57 | melwitt | the bug is unable to attach volume to instance with config drive on arm64 and according to the feature support matrix the support is "unknown" so I was thinking to set the bug Low based on that https://docs.openstack.org/nova/latest/user/support-matrix.html#operation_attach_volume_driver_libvirt_kvm_aarch64 | |
| 16:25:03 | kashyap | melwitt: I see. I'll collect thoughts and write it in the change, and we can take it from there. | |
| 16:25:49 | kashyap | sean-k-mooney: The 'max' model will apparently give you the moving-target of "all the stuff we [QEMU] can currently emulate". | |
| 16:26:46 | kashyap | melwitt: Yeah, 'low' for now is fine. | |
| 16:26:48 | sean-k-mooney | kashyap: so totally non portable | |
| 16:27:01 | sean-k-mooney | kashyap: where is max used | |
| 16:27:26 | sean-k-mooney | e.g.is it a cpu model or a scisi contoler or what | |
| 16:27:31 | kashyap | sean-k-mooney: Well, the recommendation from the AArch64 experts is to use 'max' for _both_ TCG and KVM: | |
| 16:27:36 | kashyap | ... "unless you really specifically want an always-the-same-thing even in newer QEMU versions fixed target". | |
| 16:27:44 | sean-k-mooney | im trying to figure out the context | |
| 16:27:52 | kashyap | Please read the scrollback with Mel. | |
| 16:28:01 | kashyap | It's in context of https://review.opendev.org/#/c/709494/ | |
| 16:28:04 | sean-k-mooney | ya i was trying to and could not fiutre it out | |
| 16:28:29 | sean-k-mooney | so this has changed form the old advise of use the VIRT cpu model | |
| 16:28:42 | melwitt | kashyap: yeah so I think what should likely happen is that once they have the CI set up and running we will update the feature support matrix with all of the things that are working in the CI? | |
| 16:31:09 | kashyap | sean-k-mooney: No, no you're mixing up CPU model and machine type for AArch64 | |
| 16:31:20 | kashyap | sean-k-mooney: 'virt' is still the recommended machine type for AArch64. | |
| 16:31:32 | kashyap | melwitt: Yeap. | |
| 16:31:33 | sean-k-mooney | ah yes i am | |
| 16:31:45 | melwitt | cool | |
| 16:31:54 | sean-k-mooney | this still will cause issue for live migration if we use max | |
| 16:32:08 | sean-k-mooney | so i dont know if that is a good default | |
| 16:32:13 | kashyap | melwitt: I _think_ first want to care about TCG, because for KVM, you'd need AArch64 hardware in the CI | |
| 16:32:25 | kashyap | sean-k-mooney: I've talked to the AArch64 maintainer, I'm posting the recommendations in the patch. | |
| 16:32:31 | sean-k-mooney | kashyap: we have AArch64 hardware in ci | |
| 16:32:33 | melwitt | kashyap: ack | |
| 16:32:46 | sean-k-mooney | kashyap: thats the whole point lenario donated some | |
| 16:32:49 | kashyap | sean-k-mooney: Good, then. We've got the recommendations for that, too. | |
| 16:32:56 | melwitt | kashyap: so do you think TCG would be a separate column in the matrix? | |
| 16:32:58 | kashyap | Linaro, I take it. Yeah | |
| 16:33:05 | kashyap | melwitt: Yeah, I'd say so. | |
| 16:33:08 | sean-k-mooney | ah yes | |
| 16:33:14 | melwitt | ok. thanks for explaining all this | |
| 16:33:16 | kashyap | (We need to clearly distinguish both cases.) | |
| 16:33:25 | kashyap | No worries, I need to refresh this every few months mysel f:D | |
| 16:33:34 | melwitt | :) | |
| 16:33:53 | sean-k-mooney | kashyap: we proably should be using the cpu_mode config option to define this | |
| 16:34:30 | sean-k-mooney | e.g. tie it in to host-passthough and host-model some how | |
| 16:35:37 | stephenfin | sean-k-mooney: how strongly do you feel about https://review.opendev.org/#/c/468203 ? | |
| 16:35:45 | stephenfin | specifically jaypipes arguments there | |
| 16:35:47 | kashyap | sean-k-mooney: One thing at a time :-) | |
| 16:36:00 | sean-k-mooney | im also not sure how i feel about htis being a bug, it fells more like a specless blueprint but im not going to really object too stronly to it beign a bug | |
| 16:37:00 | stephenfin | I ask because I'm trying to decide how to say "these cores should be dedicated" in a mixed instance | |
| 16:37:16 | sean-k-mooney | kashyap: well for now they could jsut use cpu_mode=custom and cpu_model=max or cpu_model=min right | |
| 16:37:35 | stephenfin | Currently I'm going with 'hw:cpu_dedicated_mask', which is a CPU list | |
| 16:37:38 | sean-k-mooney | stephenfin: am ill take a look now | |
| 16:37:48 | sean-k-mooney | stephenfin: ya i would be fine with that | |
| 16:37:55 | sean-k-mooney | stephenfin: whats the other option | |
| 16:38:05 | stephenfin | there are a few | |
| 16:38:34 | stephenfin | in this scenario that I'm following, you'll be able to use hw:cpu_dedicated_mask *or* hw:cpu_realtime_mask | |
| 16:38:59 | sean-k-mooney | stephenfin: no you would use both | |
| 16:39:01 | stephenfin | I see no reason to say these cores are shared, these are dedicated but non-realtime, and these are dedicated and realtime | |
| 16:39:04 | sean-k-mooney | well optinally | |
| 16:39:16 | sean-k-mooney | e.g. not an exclucive or | |
| 16:39:41 | sean-k-mooney | i dont see a reason to block it | |
| 16:39:41 | stephenfin | why? What real-world user is going to use all three types of core in an instance? | |
| 16:39:49 | stephenfin | Because it's less complicated | |
| 16:39:55 | sean-k-mooney | its more complicated | |
| 16:40:20 | sean-k-mooney | the validation logic to prevent all 3 is extra logic we dont need if we allow it | |
| 16:40:45 | stephenfin | A|B is easier grok than A.issubset(B) | |
| 16:40:56 | stephenfin | and the it makes my XML generation easier | |
| 16:41:02 | stephenfin | s/the // | |
| 16:41:04 | sean-k-mooney | stephenfin: you are over loading hw:cpu_realtime_mask | |
| 16:41:18 | sean-k-mooney | its behavior would change based on the hw:cpu_policy | |
| 16:41:29 | sean-k-mooney | so that gets harder to reason about | |
| 16:41:36 | stephenfin | nope, it stays the same: these are cores that are real-time | |
| 16:41:39 | sean-k-mooney | if we allow both it does not | |
| 16:41:41 | stephenfin | what changes is what happens to the other cores | |
| 16:41:53 | stephenfin | and that's purely based on hw:cpu_policy | |
| 16:42:10 | sean-k-mooney | no it chacnge form tehse are realtime to these are realtime and dedicated and the rest flaot | |
| 16:42:17 | kashyap | sean-k-mooney: Not entirely; there's also a quirk of making sure to specify the interrupt controller (as the default is less featureful) -- `-machine gic-version=max` | |
| 16:42:29 | kashyap | melwitt: For later, added my notes in the change. | |
| 16:42:32 | stephenfin | they're always realtime and dedicated | |
| 16:42:39 | stephenfin | you can't have realtime floating cores | |
| 16:42:45 | melwitt | kashyap: thanks | |
| 16:43:05 | sean-k-mooney | well libvirt allows you to but that is a seperate thign | |
| 16:43:14 | stephenfin | ...in nova | |
| 16:43:14 | kashyap | Yep | |
| 16:43:20 | sean-k-mooney | stephenfin: my perference would be to allow both to be set | |
| 16:43:38 | stephenfin | I could allow it, but I don't want to force it | |
| 16:43:46 | sean-k-mooney | and always require hw:dedicated_cpu_mask for mixed | |
| 16:43:58 | stephenfin | and because I don't want to force it, I'd rather say there's only one way to do this | |
| 16:44:14 | sean-k-mooney | which one do you not want to force | |
| 16:44:32 | sean-k-mooney | the realtime mask or the dedicated mask | |
| 16:44:33 | stephenfin | having to set both hw:cpu_dedicated_mask and hw:cpu_realtime_mask | |