Earlier  
Posted Nick Remark
#openstack-nova - 2019-04-25
14:06:22 efried edleafe: This set of traits is being mapped directly from that document
14:06:25 mriedem bauzas: ^ is a nice little perf improvement when listing AZs in a large cloud
14:06:38 edleafe efried: ok, I understand a bit more.
14:06:43 bauzas mriedem: okay
14:06:46 efried edleafe: sections important_cpu_features_intel_x86 and important_cpu_features_amd_x86
14:06:49 tssurya mriedem: yeah I have it open
14:06:59 bauzas mriedem: FWIW, I'll be traveling starting tonight
14:07:02 tssurya thanks for doing that
14:07:08 edleafe That doc also defines things such as mutually exclusive traits, which os-traits and placement doesn't enforce
14:07:34 sean-k-mooney efried: edleafe i still think we need to consider the securtiy impact of having NO_SSB as a trait before we add it
14:07:35 efried edleafe: true story, but not really relevant; the driver is going to detect what's on the CPU and report that.
14:07:36 mriedem tssurya: thank avolkov
14:07:56 kashyap edleafe: Only 'amd-no-ssb
14:08:08 kashyap edleafe: Only 'amd-no-ssb' is the mutually exclusive trait.
14:08:29 kashyap Again, what efried said ^ (on driver detecting)
14:08:36 edleafe sean-k-mooney: what would the potential impact be? A CPU reporting it that is actually vulnerable to it?
14:08:45 sean-k-mooney yes
14:08:46 tssurya I tweaked a downstream patch when we opened that bug, never got around to doing it upstream
14:09:00 sean-k-mooney and then a user uploading an image that require a cpu with the vulnerablity
14:09:01 efried edleafe: sean-k-mooney is saying a malicious user could specify NO_SSB as a forbidden trait to deliberately land on a vulnerable host.
14:09:02 kashyap sean-k-mooney: Also what is "forbidden" in this context?
14:09:19 efried kashyap: We have the ability to specify a request with "DON'T land on a host if it has this trait"
14:09:23 sean-k-mooney kashyap: we support requireing traitns or selecting host that do not have a trait
14:09:28 kashyap efried: Ah-ha
14:09:44 efried kashyap: The placement request would look like required=!HW_CPU_X86_AMD_NO_SSB
14:09:47 sean-k-mooney so if we report either the presence or absence of ssbd
14:10:01 efried kashyap: From a flavor extra spec like trait:HW_CPU_X86_AMD_NO_SSB=forbidden IIRC
14:10:20 sean-k-mooney i can exploit that to target a host with the volnerablity with my payload
14:10:28 efried Yeah, I don't know if this is a problem though sean-k-mooney
14:10:29 kashyap Yeah, noted.
14:10:41 edleafe sean-k-mooney: so without that trait, attackers could just spin up random VMs and see if they are on a vulnerable host. Having the trait just makes those hosts easier to find. Is that it?
14:10:49 kashyap efried: Yeah, that's what I was wondering. The positives of having it outweigh some theoretical threat
14:10:51 efried The admin is responsible for putting together the flavors. Why would the admin do that?
14:10:56 sean-k-mooney edleafe: yes
14:11:18 sean-k-mooney i would like the openstack security team to weigh in on this personally
14:11:37 efried there's an email address for that, right?
14:11:46 mriedem efried: gibi: trying to recreate this TestRPC infinite recursion is getting frustrating :) https://review.opendev.org/#/c/654468/ - i'm thinking maybe push another change to oslo.config (or nova?) that sets the max recursion depth and see if that blows things up?
14:11:47 sean-k-mooney efried: users can upload image with traits request
14:11:59 sean-k-mooney so its not admin im worried about
14:12:00 kashyap efried: Yes, very good point on admin's responsibility
14:12:28 efried http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-security
14:12:32 gibi mriedem: I also tried to reproduce it locally without success. Do you assume that if you set the recursion limit to small enough then the stack trace will not be truncated?
14:12:44 efried ugh, do you have to be subscribed to post to that?
14:13:00 kashyap efried: Not necessarily
14:13:05 sean-k-mooney efried: there is but we can also talk to them in person next week
14:13:09 kashyap efried: Mailman moderator can approve you
14:13:14 sean-k-mooney we will all be in the same place
14:13:29 kashyap sean-k-mooney: In-person, things get too long-winded, and they get lost.
14:13:32 efried sean-k-mooney: Okay, so maybe we need to have an explicit check for such traits and disallow them from image meta?
14:13:34 gibi mriedem: anyhow sounds like a good idea
14:13:36 mriedem gibi: that's the idea
14:13:50 sean-k-mooney efried: maybe
14:13:54 mriedem alternatively, and this is probably easier, just change my oslo.config change from logging a traceback whe it hits a recursion of 3 to just raise
14:13:58 mriedem i'll do that first
14:14:09 efried sean-k-mooney, kashyap: So is one of you going to compose an email? Please copy me. (I'm not subscribed to that list.)
14:14:23 stephenfin ralonsoh: It's a nit, but any chance you can fix this before we merge and backport? https://review.opendev.org/#/c/655332/4/vif_plug_linux_bridge/linux_net.py
14:14:40 kashyap efried: I'm not subscribed either. But will do, if I write.
14:14:47 efried thanks.
14:15:17 sean-k-mooney ok as i said im sure we can grab 10 mins with them next week at the ptg/fourm too
14:15:45 kashyap Also, "exploiting" here is all hand-wavy, and requires some serious competency, and timing to take advantage of such things.
14:15:46 sean-k-mooney efried: and yes we could have a blacklist of traits that cant be in the image
14:16:07 sean-k-mooney kashyap: sure but not allo fo them do
14:16:09 efried sean-k-mooney: Security people will tell you it should be a whitelist :(
14:16:20 kashyap sean-k-mooney: The thing is, I'll forget it in 5 seconds. My memory is worse than a bumblebee's (less than 8 seconds)
14:16:28 sean-k-mooney well the sepc said that too if i recall
14:16:30 efried or a goldfish
14:16:33 efried did we have this conversation?
14:16:34 sean-k-mooney but we never implemented it
14:16:43 kashyap efried: Heh, I guess so
14:16:47 efried and you forgot
14:16:50 efried typical
14:17:47 efried (My memory is pretty unpredictable. Some things I remember forever, some I need hammered in a thousand times and still can't hold onto.)
14:17:48 kashyap efried: Yeah. On the other hand, we (I) should work towards getting the rest of the "uncontroversial" triats in, as that helps admins _today_
14:18:19 aspiers efried, kashyap: what's the current thinking on how to handle the SEV trait? keep as HW_CPU_AMD_SEV, or move to HW_CPU_X86_AMD_SEV?
14:18:21 kashyap efried: Yeah, I hear ya...something similar here.
14:18:26 efried kashyap, sean-k-mooney: I'm going to say it's fine to add the traits to os-traits regardless; the vulnerability occurs when we expose them via compute.
14:18:46 ralonsoh stephenfin, well it makes sense. Because both parameters should be different, the current implementation is valid. But yes, I'll change it
14:18:48 aspiers efried, kashyap: I have ongoing reviews which kinda rely on a decision on this ...
14:18:49 efried aspiers: "deprecate" HW_CPU_AMD_SEV, "add" HW_CPU_X86_AMD_SEV
14:18:49 sean-k-mooney efried: im wondering if we should namespace them
14:19:07 sean-k-mooney or otherwise mark them up in os-traits
14:19:11 aspiers efried: do we need a traits deprecation mechanism? other than just documenting?
14:19:16 efried sean-k-mooney: Namespace them as HW_CPU_X86_POTENTIAL_SECURITY_VULNERABILITY_NO_SSB??
14:19:36 efried aspiers: It's been discussed before, but we decided not to do it. So just documenting/commenting.
14:19:54 aspiers efried: decided not to do it ever, or just not any time soon?
14:19:56 kashyap aspiers: I saw your reorg comment, still have to look at it. (I'm still a bit unwell, so, slow today as well)
14:20:09 aspiers kashyap: sorry to hear that :-( no pressure
14:20:10 efried when we first split os-traits out of placement (in nova) there was a proposal (from jaypipes iirc) to do it with a more complex architecture that would allow for deprecation and removal.
14:20:21 efried but we decided on the super-simple approach instead.
14:20:40 sean-k-mooney efried: not quite but maybe have _NO_IMG_ or something
14:20:43 efried aspiers: No plans in the forseeable future.
14:20:47 aspiers efried: just wondering if https://review.opendev.org/#/c/655673/ should have explained that too
14:21:00 sean-k-mooney that or add them to a seperate dict in os-traits
14:21:05 efried sean-k-mooney: Yeah, no. We don't want to try to predict what those would be, and that's not the responsibility of os-traits IMHO.
14:21:07 sean-k-mooney or list
14:21:16 kashyap aspiers: Ah, nice. It's actually merged; efried++
14:21:43 efried aspiers: Sure, could do, looks like mriedem felt the same. It's docs, easy to add more.
14:22:13 sean-k-mooney well we should ask ourselve is this a security issue when adding new traits in general. anway i need to go finsih packing
14:22:22 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/os-vif master: Prevent "qbr" Linux Bridge from replying to ARP messages https://review.opendev.org/655332
14:23:00 kashyap efried: Thanks; good point -- so nothing is blocking from adding the trait. Good

Earlier   Later