| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-02-10 | |||
| 11:28:25 | sean-k-mooney | then it should complete stacking then if you just have one nic | |
| 11:29:33 | sean-k-mooney | devstack pretty much just works if you use it today on ubuntu and centos. i got it to stack on fedroa too but that is less stable generally due to package conflicts | |
| 11:29:43 | sean-k-mooney | e.g. between pip and dnf | |
| 11:36:07 | kashyap | Fedora is the beast I know; I'm gonna skip CentOS. First, helping a different colleague | |
| 11:37:15 | kashyap | sean-k-mooney: Unrelated: BTW, did you know that the "tsx=off" on kernel command-line will _not_ have any effect in certain cases? | |
| 11:38:26 | sean-k-mooney | like when its off already or the kernel does not have it :P | |
| 11:38:54 | kashyap | sean-k-mooney: Off on the kernel command-line; the processor itself has it | |
| 11:38:55 | sean-k-mooney | centos, centos stream and ubunut all work well | |
| 11:39:08 | kashyap | sean-k-mooney: Reading the upstream documentation in the kernel commit, it points out that "tsx=off" will only have effect on newer CPUs that are not vulnerable to MDS (which is indicated by a microcode) -- https://github.com/torvalds/linux/commit/95c5824f75f3 | |
| 11:39:41 | sean-k-mooney | kashyap: sure so it turns it off on procees that dont have it disabled by microcode | |
| 11:40:01 | sean-k-mooney | and for older cpus they are assuming you have teh microcode updated | |
| 11:40:13 | sean-k-mooney | which sound resounable | |
| 11:40:28 | sean-k-mooney | if you want it turned off and there is a newer microcode that does that then you should update | |
| 11:41:14 | kashyap | Yeah; what a bloody mess this is | |
| 11:42:00 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use neutronclient's port binding APIs https://review.opendev.org/c/openstack/nova/+/706295 | |
| 11:42:48 | sean-k-mooney | kashyap: i just do this http://paste.openstack.org/show/802510/ | |
| 11:44:55 | kashyap | sean-k-mooney: Also, aside: even "tsx=off" does not remove it from /proc/cpuinfo -- and we should not rely on it anyway. Because it only shows what the hardware has. | |
| 11:45:27 | sean-k-mooney | well that is the only thing we can check | |
| 11:45:36 | sean-k-mooney | what would we use instead | |
| 11:45:49 | sean-k-mooney | dmidecode? | |
| 11:46:04 | sean-k-mooney | that the only thing i can think off but i dont think that will help | |
| 11:46:24 | sean-k-mooney | if /proc/cpuinfo is wrong libvirt will be | |
| 11:46:59 | sean-k-mooney | /proc/cpuinfo change when you have hypterthreading disable/enable in the bios not sure about at the kernel level | |
| 11:47:09 | sean-k-mooney | is that the delta | |
| 11:47:15 | sean-k-mooney | bios vs kernel disable | |
| 11:48:55 | kashyap | There is no BIOS switch for it here. | |
| 11:49:24 | kashyap | sean-k-mooney: But /proc/cpuinfo only shows what the hardware itself has -- the kernel parameter 'tsx=off' will _not_ physicallly remove it from /proc/cpuinfo; it just tells the OS not to use it | |
| 11:49:53 | kashyap | So, anyway, I'm jotting down some notes on this to see find out a reliable approach to check | |
| 11:50:25 | sean-k-mooney | kashyap: well that is not always the case | |
| 11:50:28 | kashyap | sean-k-mooney: Also, there _are_ other ways: We should instead check 'virsh domcapabilities' on the compute node to see the effect of 'tsx=off' parameter. | |
| 11:50:48 | sean-k-mooney | kashyap: if you disabel hyperthreading in the bios ht is removed form /proc/cpuinfo | |
| 11:51:04 | kashyap | sean-k-mooney: That's not the same -- TSX has no BIOS option. | |
| 11:51:37 | sean-k-mooney | right thats why i was asking is this a delta between disabeling a feature at the bios vs kernel | |
| 11:51:58 | sean-k-mooney | e.g. /proc/cpuinfo is showing what the bios reports | |
| 11:52:16 | kashyap | Yep | |
| 11:53:01 | sean-k-mooney | oh i have tsx on my laptop | |
| 11:53:07 | sean-k-mooney | or at least hle | |
| 11:53:13 | sean-k-mooney | wht the other one | |
| 11:53:58 | kashyap | 'rtm' | |
| 11:54:02 | sean-k-mooney | ya i have both hle and rtm | |
| 11:58:20 | sean-k-mooney | skylake so it should be enabled by defult which it does | |
| 11:58:43 | sean-k-mooney | kashyap: i tought you said the upstream kernel disabled tsx? | |
| 11:59:09 | kashyap | sean-k-mooney: It did (like I said, but its effect depends on the processor); what kernel are you running? | |
| 11:59:13 | sean-k-mooney | im running 5.10.8 | |
| 11:59:17 | sean-k-mooney | from fedora 32 | |
| 11:59:25 | kashyap | sean-k-mooney: Yours is older | |
| 11:59:30 | kashyap | sean-k-mooney: You need v5.11 or above | |
| 11:59:41 | kashyap | I even mentioned it in today's agenda meeting notes :D | |
| 12:00:04 | sean-k-mooney | so you ment it was disabled in an unreleased kernel | |
| 12:00:12 | sean-k-mooney | 5.11 is not released yet | |
| 12:00:35 | kashyap | sean-k-mooney: Wait, it was disabled in Oct 2019 | |
| 12:00:43 | kashyap | There have been plenty kernels released since then: https://github.com/torvalds/linux/commit/95c5824f75f3 | |
| 12:00:59 | sean-k-mooney | well fedora has it enabled | |
| 12:01:43 | sean-k-mooney | and the kernel config has CONFIG_X86_INTEL_TSX_MODE_OFF=y | |
| 12:02:07 | sean-k-mooney | i guess this is what you ment by you cant trust /proc/cpuinfo | |
| 12:02:17 | sean-k-mooney | let me check in a vm | |
| 12:02:19 | kashyap | sean-k-mooney: How are you saying it is "enabled"? If you're looking in /proc/cpuinfo, it is not | |
| 12:02:31 | kashyap | s/it is not/don't trust that/ :) | |
| 12:02:46 | kashyap | sean-k-mooney: Yes, as you see, your config has it _OFF=y | |
| 12:03:30 | sean-k-mooney | kashyap: that is the only interface we really have in userspace without poking registers | |
| 12:03:39 | sean-k-mooney | if we cant trust that we cant trust libvirt | |
| 12:04:51 | kashyap | sean-k-mooney: Wait. libvirt developers are telling me to use 'virsh domcapabilities' to detect what effect the 'tsx=off' parameter has | |
| 12:05:11 | kashyap | sean-k-mooney: The only other option is to poke for the MSR -- which I have to look, even if it is possible | |
| 12:06:28 | kashyap | sean-k-mooney: BTW, this not showing up "/proc/cpuinfo" is nothing new -- another flag is "mds-no" -- it also won't show up in /proc/cpuinfo | |
| 12:06:34 | sean-k-mooney | cpuid show RTM: restricted transactional memory = true | |
| 12:06:36 | kashyap | I wrote the upstream QEMU docs for it; /me looks | |
| 12:07:50 | kashyap | FWIW: https://lists.nongnu.org/archive/html/qemu-devel/2020-02/msg07164.html (qemu-cpu-models.rst: Document -noTSX, mds-no, taa-no, and tsx-ctrl) | |
| 12:08:20 | kashyap | sean-k-mooney: So you had it off in the kernel; but the hardware supports it. But `cpuid` shows RTM true? | |
| 12:08:59 | sean-k-mooney | yes cpu id shows true | |
| 12:09:15 | sean-k-mooney | and its off in the kernel | |
| 12:11:12 | sean-k-mooney | http://paste.openstack.org/show/802511/ | |
| 12:11:31 | sean-k-mooney | Skylake-Client-IBRS has it enabled | |
| 12:15:03 | kashyap | Yes, libvirt/QEMU introduced: "Skylake-Client-IBRS-noTSX" | |
| 12:20:02 | sean-k-mooney | ya so right now i do have rtm and hle in a vm on my laptop | |
| 12:21:28 | sean-k-mooney | with libvirt 6.1.0 | |
| 13:03:51 | sean-k-mooney | stephenfin: requirements_cehcks is unhappy but everything else passed https://review.opendev.org/c/openstack/os-vif/+/774694 | |
| 13:12:53 | openstackgerrit | Merged openstack/nova master: db: Compact Queens database migrations https://review.opendev.org/c/openstack/nova/+/759088 | |
| 14:41:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Use neutronclient's port binding APIs https://review.opendev.org/c/openstack/nova/+/706295 | |
| 14:45:29 | openstackgerrit | Stephen Finucane proposed openstack/python-novaclient master: Add support for microversion v2.88 https://review.opendev.org/c/openstack/python-novaclient/+/770573 | |
| 15:08:17 | sean-k-mooney | gibi: you can see nova security bug correct. | |
| 15:09:26 | sean-k-mooney | is there a group i can check i want to see who is on the list | |
| 15:09:28 | gibi | sean-k-mooney: hm, I saw some before, but then I was explicitly included in them. I think I'm not in the secu group in lauchpad | |
| 15:10:12 | sean-k-mooney | ah ok im being cauios with a bug im filing so im going to mark it as a security one initally but id like your input | |
| 15:10:20 | sean-k-mooney | ill add you to cc directly just in case | |
| 15:10:40 | gibi | sean-k-mooney: yes that will work | |
| 15:22:48 | openstackgerrit | Stephen Finucane proposed openstack/placement master: Move policy deprecation to base rules https://review.opendev.org/c/openstack/placement/+/772784 | |
| 15:23:14 | stephenfin | gmann: I found/fixed the bug on that and am +2 on it now ^ | |
| 15:24:58 | stephenfin | lyarwood: When you're back, would appreciate you weighing in on https://review.opendev.org/c/openstack/nova/+/751368 (stable patch) | |
| 15:26:01 | gmann | stephenfin: ah i see. thanks. | |
| 15:26:09 | stephenfin | it was so close to be perfect :D | |
| 15:26:13 | stephenfin | *being | |
| 15:27:34 | gmann | I thought we have broken something :) | |
| 15:32:44 | openstackgerrit | Stephen Finucane proposed openstack/nova master: cmd: Remove 'nova-manage db ironic_flavor_migration' https://review.opendev.org/c/openstack/nova/+/773776 | |
| 15:33:10 | stephenfin | gibi: You were +2 on that previously but I'd to address a doc failure. Care to revisit? Only change is to 'doc/source/cli/nova-manage.rst' ^ | |
| 15:33:20 | gibi | sure | |
| 15:33:24 | stephenfin | thanks | |
| 15:34:09 | stephenfin | gibi: We're _almost_ done with merging the main DB migration squash series (about time). We can probably look at removing that tweak to tox.ini once it's done, I guess? | |
| 15:35:00 | gibi | stephenfin: yepp, I can propose the rewert on top of the db series | |
| 15:35:10 | stephenfin | cool | |
| 15:44:13 | openstackgerrit | Stephen Finucane proposed openstack/nova master: apidb: Add manage.py script https://review.opendev.org/c/openstack/nova/+/771419 | |