| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-02-10 | |||
| 10:08:58 | kashyap | lyarwood: Just one; shall I add one more? /me looks at his working older DevStack setup | |
| 10:09:30 | lyarwood | yeah I've always had two, I assume that's why it's unhappy | |
| 10:10:27 | kashyap | lyarwood: Gah. But ... in the name Zeus, I got it working w/ one NIC on an F31 deployment | |
| 10:10:33 | kashyap | Lemme try; sorry for the noise | |
| 10:25:27 | openstackgerrit | Merged openstack/placement stable/ussuri: Fix Placement Doc https://review.opendev.org/c/openstack/placement/+/762607 | |
| 10:25:34 | openstackgerrit | Merged openstack/placement stable/train: Fix Placement Doc https://review.opendev.org/c/openstack/placement/+/762608 | |
| 11:10:49 | openstackgerrit | Stephen Finucane proposed openstack/placement master: policy: Add releasenote for RBAC work https://review.opendev.org/c/openstack/placement/+/774654 | |
| 11:10:49 | openstackgerrit | Stephen Finucane proposed openstack/placement master: Implement secure RBAC for reshaper https://review.opendev.org/c/openstack/placement/+/774827 | |
| 11:11:01 | stephenfin | gibi, lyarwood: ^ | |
| 11:11:16 | stephenfin | Also spotted a policy that hadn't been updated. Don't know if that was intentional from lbragstad but I suspect not | |
| 11:11:24 | gibi | stephenfin: ack | |
| 11:15:50 | gibi | stephenfin: all looks good to me | |
| 11:22:17 | sean-k-mooney | kashyap: you should aways use at least 2 nics for a devstack setup | |
| 11:22:34 | sean-k-mooney | 1 for ssh/managment and a second for ovs br-ex | |
| 11:23:05 | sean-k-mooney | you can do it with one but devstack will try and move the ip and add it to the ovs bridge | |
| 11:23:13 | sean-k-mooney | which can cause ssh to drop | |
| 11:23:36 | sean-k-mooney | so unless you run devstack in tmux or screen that can break things | |
| 11:24:01 | sean-k-mooney | you used to be able to just not add any interface but they changed something recently | |
| 11:24:20 | sean-k-mooney | havent spend time figuring out what | |
| 11:27:20 | kashyap | sean-k-mooney: I do run in a 'tmux', IIRC | |
| 11:27:59 | kashyap | sean-k-mooney: In the past, I recall testing just fine w/ one network that is _not_ the 'default' libvirt network. I created a separate NAT br to use for OpenStack n/w | |
| 11:28:25 | sean-k-mooney | then it should complete stacking then if you just have one nic | |
| 11:29:33 | sean-k-mooney | devstack pretty much just works if you use it today on ubuntu and centos. i got it to stack on fedroa too but that is less stable generally due to package conflicts | |
| 11:29:43 | sean-k-mooney | e.g. between pip and dnf | |
| 11:36:07 | kashyap | Fedora is the beast I know; I'm gonna skip CentOS. First, helping a different colleague | |
| 11:37:15 | kashyap | sean-k-mooney: Unrelated: BTW, did you know that the "tsx=off" on kernel command-line will _not_ have any effect in certain cases? | |
| 11:38:26 | sean-k-mooney | like when its off already or the kernel does not have it :P | |
| 11:38:54 | kashyap | sean-k-mooney: Off on the kernel command-line; the processor itself has it | |
| 11:38:55 | sean-k-mooney | centos, centos stream and ubunut all work well | |
| 11:39:08 | kashyap | sean-k-mooney: Reading the upstream documentation in the kernel commit, it points out that "tsx=off" will only have effect on newer CPUs that are not vulnerable to MDS (which is indicated by a microcode) -- https://github.com/torvalds/linux/commit/95c5824f75f3 | |
| 11:39:41 | sean-k-mooney | kashyap: sure so it turns it off on procees that dont have it disabled by microcode | |
| 11:40:01 | sean-k-mooney | and for older cpus they are assuming you have teh microcode updated | |
| 11:40:13 | sean-k-mooney | which sound resounable | |
| 11:40:28 | sean-k-mooney | if you want it turned off and there is a newer microcode that does that then you should update | |
| 11:41:14 | kashyap | Yeah; what a bloody mess this is | |
| 11:42:00 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use neutronclient's port binding APIs https://review.opendev.org/c/openstack/nova/+/706295 | |
| 11:42:48 | sean-k-mooney | kashyap: i just do this http://paste.openstack.org/show/802510/ | |
| 11:44:55 | kashyap | sean-k-mooney: Also, aside: even "tsx=off" does not remove it from /proc/cpuinfo -- and we should not rely on it anyway. Because it only shows what the hardware has. | |
| 11:45:27 | sean-k-mooney | well that is the only thing we can check | |
| 11:45:36 | sean-k-mooney | what would we use instead | |
| 11:45:49 | sean-k-mooney | dmidecode? | |
| 11:46:04 | sean-k-mooney | that the only thing i can think off but i dont think that will help | |
| 11:46:24 | sean-k-mooney | if /proc/cpuinfo is wrong libvirt will be | |
| 11:46:59 | sean-k-mooney | /proc/cpuinfo change when you have hypterthreading disable/enable in the bios not sure about at the kernel level | |
| 11:47:09 | sean-k-mooney | is that the delta | |
| 11:47:15 | sean-k-mooney | bios vs kernel disable | |
| 11:48:55 | kashyap | There is no BIOS switch for it here. | |
| 11:49:24 | kashyap | sean-k-mooney: But /proc/cpuinfo only shows what the hardware itself has -- the kernel parameter 'tsx=off' will _not_ physicallly remove it from /proc/cpuinfo; it just tells the OS not to use it | |
| 11:49:53 | kashyap | So, anyway, I'm jotting down some notes on this to see find out a reliable approach to check | |
| 11:50:25 | sean-k-mooney | kashyap: well that is not always the case | |
| 11:50:28 | kashyap | sean-k-mooney: Also, there _are_ other ways: We should instead check 'virsh domcapabilities' on the compute node to see the effect of 'tsx=off' parameter. | |
| 11:50:48 | sean-k-mooney | kashyap: if you disabel hyperthreading in the bios ht is removed form /proc/cpuinfo | |
| 11:51:04 | kashyap | sean-k-mooney: That's not the same -- TSX has no BIOS option. | |
| 11:51:37 | sean-k-mooney | right thats why i was asking is this a delta between disabeling a feature at the bios vs kernel | |
| 11:51:58 | sean-k-mooney | e.g. /proc/cpuinfo is showing what the bios reports | |
| 11:52:16 | kashyap | Yep | |
| 11:53:01 | sean-k-mooney | oh i have tsx on my laptop | |
| 11:53:07 | sean-k-mooney | or at least hle | |
| 11:53:13 | sean-k-mooney | wht the other one | |
| 11:53:58 | kashyap | 'rtm' | |
| 11:54:02 | sean-k-mooney | ya i have both hle and rtm | |
| 11:58:20 | sean-k-mooney | skylake so it should be enabled by defult which it does | |
| 11:58:43 | sean-k-mooney | kashyap: i tought you said the upstream kernel disabled tsx? | |
| 11:59:09 | kashyap | sean-k-mooney: It did (like I said, but its effect depends on the processor); what kernel are you running? | |
| 11:59:13 | sean-k-mooney | im running 5.10.8 | |
| 11:59:17 | sean-k-mooney | from fedora 32 | |
| 11:59:25 | kashyap | sean-k-mooney: Yours is older | |
| 11:59:30 | kashyap | sean-k-mooney: You need v5.11 or above | |
| 11:59:41 | kashyap | I even mentioned it in today's agenda meeting notes :D | |
| 12:00:04 | sean-k-mooney | so you ment it was disabled in an unreleased kernel | |
| 12:00:12 | sean-k-mooney | 5.11 is not released yet | |
| 12:00:35 | kashyap | sean-k-mooney: Wait, it was disabled in Oct 2019 | |
| 12:00:43 | kashyap | There have been plenty kernels released since then: https://github.com/torvalds/linux/commit/95c5824f75f3 | |
| 12:00:59 | sean-k-mooney | well fedora has it enabled | |
| 12:01:43 | sean-k-mooney | and the kernel config has CONFIG_X86_INTEL_TSX_MODE_OFF=y | |
| 12:02:07 | sean-k-mooney | i guess this is what you ment by you cant trust /proc/cpuinfo | |
| 12:02:17 | sean-k-mooney | let me check in a vm | |
| 12:02:19 | kashyap | sean-k-mooney: How are you saying it is "enabled"? If you're looking in /proc/cpuinfo, it is not | |
| 12:02:31 | kashyap | s/it is not/don't trust that/ :) | |
| 12:02:46 | kashyap | sean-k-mooney: Yes, as you see, your config has it _OFF=y | |
| 12:03:30 | sean-k-mooney | kashyap: that is the only interface we really have in userspace without poking registers | |
| 12:03:39 | sean-k-mooney | if we cant trust that we cant trust libvirt | |
| 12:04:51 | kashyap | sean-k-mooney: Wait. libvirt developers are telling me to use 'virsh domcapabilities' to detect what effect the 'tsx=off' parameter has | |
| 12:05:11 | kashyap | sean-k-mooney: The only other option is to poke for the MSR -- which I have to look, even if it is possible | |
| 12:06:28 | kashyap | sean-k-mooney: BTW, this not showing up "/proc/cpuinfo" is nothing new -- another flag is "mds-no" -- it also won't show up in /proc/cpuinfo | |
| 12:06:34 | sean-k-mooney | cpuid show RTM: restricted transactional memory = true | |
| 12:06:36 | kashyap | I wrote the upstream QEMU docs for it; /me looks | |
| 12:07:50 | kashyap | FWIW: https://lists.nongnu.org/archive/html/qemu-devel/2020-02/msg07164.html (qemu-cpu-models.rst: Document -noTSX, mds-no, taa-no, and tsx-ctrl) | |
| 12:08:20 | kashyap | sean-k-mooney: So you had it off in the kernel; but the hardware supports it. But `cpuid` shows RTM true? | |
| 12:08:59 | sean-k-mooney | yes cpu id shows true | |
| 12:09:15 | sean-k-mooney | and its off in the kernel | |
| 12:11:12 | sean-k-mooney | http://paste.openstack.org/show/802511/ | |
| 12:11:31 | sean-k-mooney | Skylake-Client-IBRS has it enabled | |
| 12:15:03 | kashyap | Yes, libvirt/QEMU introduced: "Skylake-Client-IBRS-noTSX" | |
| 12:20:02 | sean-k-mooney | ya so right now i do have rtm and hle in a vm on my laptop | |
| 12:21:28 | sean-k-mooney | with libvirt 6.1.0 | |
| 13:03:51 | sean-k-mooney | stephenfin: requirements_cehcks is unhappy but everything else passed https://review.opendev.org/c/openstack/os-vif/+/774694 | |
| 13:12:53 | openstackgerrit | Merged openstack/nova master: db: Compact Queens database migrations https://review.opendev.org/c/openstack/nova/+/759088 | |
| 14:41:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use neutronclient's port binding APIs https://review.opendev.org/c/openstack/nova/+/706295 | |
| 14:45:29 | openstackgerrit | Stephen Finucane proposed openstack/python-novaclient master: Add support for microversion v2.88 https://review.opendev.org/c/openstack/python-novaclient/+/770573 | |