| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-11 | |||
| 17:17:47 | dansmith | and there have been exploits where even dropping the map could be circumvented by tricking the thing into granting you access to a region again without it being zeroed | |
| 17:18:03 | efried | I mean, it doesn't say "Really really zero the output device" | |
| 17:18:09 | efried | but still | |
| 17:18:15 | efried | you would think it would be set up to prevent contamination | |
| 17:18:19 | dansmith | efried: and for silicon, it almost never does because it costs money (i.e. wears the media) | |
| 17:18:20 | efried | otherwise what good is it? | |
| 17:18:51 | sean-k-mooney | efried: anyway were are side tracking | |
| 17:18:58 | efried | agreed | |
| 17:19:01 | dansmith | efried: you said "regular deleted data" above, which is what I took exception to | |
| 17:19:30 | efried | ack | |
| 17:19:38 | sean-k-mooney | right its specal hadware that needs special handeling | |
| 17:19:39 | dansmith | but I will say, all the spectre stuff is about sussing out data via sidechannel which is not reading it direclty | |
| 17:19:44 | dansmith | so let's not pretend that's not a thing :) | |
| 17:19:52 | efried | Yeah, of course if you just trash the inode and don't actually zero the content | |
| 17:20:29 | efried | I'm talking about: if you overwrite with zeros, but just once, don't you need something special (that VMs don't have) to read the ghost data? | |
| 17:20:46 | dansmith | not necessarily | |
| 17:20:53 | efried | okay | |
| 17:20:55 | dansmith | you're thinking about low-level hardware detection of things, | |
| 17:21:04 | dansmith | but that's not the only way to get access to that "ghost" data | |
| 17:21:17 | dansmith | that's the whole point of sidechannel attacks | |
| 17:22:02 | alex_xu | efried: do we have something to ignore a host for placement allocation_candidates call now? | |
| 17:22:04 | efried | We still have some more minor deltas to do right? | |
| 17:22:04 | efried | alex_xu, sean-k-mooney: looks like the CI fixups have merged now https://github.com/intel/Intel-OpenStack-CI-jobs/pull/2 | |
| 17:22:26 | efried | alex_xu: you mean something like in_tree=!$uuid ? | |
| 17:22:32 | alex_xu | yes | |
| 17:23:10 | sean-k-mooney | i havent done a full review but the inital change i have looked at seam correct | |
| 17:23:35 | efried | alex_xu: I don't think so. cdent, we haven't done !in_tree yet, right? | |
| 17:23:48 | efried | alex_xu: why? | |
| 17:24:06 | efried | oh | |
| 17:24:08 | alex_xu | efried: for his https://review.opendev.org/#/c/681383/5/nova/scheduler/manager.py@155 | |
| 17:24:36 | alex_xu | if we have that, then we can ignore the src host for the same host resize | |
| 17:24:40 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595 | |
| 17:24:40 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | |
| 17:25:14 | alex_xu | efried: emm...or we do a manually filter after that | |
| 17:25:18 | sean-k-mooney | all the numa jobs just got killed | |
| 17:25:52 | artom | mriedem, dansmith ^^ | |
| 17:26:13 | efried | alex_xu: Isn't this the famous "doubled allocations" bug? | |
| 17:26:30 | alex_xu | efried: no, it isn't | |
| 17:27:07 | alex_xu | efried: when a host enable same host resize, and that host is the only host has available PCPU now. | |
| 17:27:22 | sean-k-mooney | alex_xu: cant ever ignore the source host for same host reszie or rebuild | |
| 17:27:40 | efried | yeah, I would think for same-host resize we would want to restrict to *only* the current host | |
| 17:27:53 | alex_xu | efried: we will try to allocate PCPU in placement, and yes, placement tell you the source host has PCPU, but later the scheduler will check ignore host, the source host is in the ignore host. so the scheduling failed. | |
| 17:27:53 | efried | ...having calculated the positive difference from our current allocations | |
| 17:28:06 | dansmith | efried: you don't know you're doing a same-host resize | |
| 17:28:19 | efried | oh, bummer | |
| 17:28:23 | alex_xu | sean-k-mooney: sorry, I mean the enable same host resize | |
| 17:28:27 | alex_xu | oops | |
| 17:28:35 | alex_xu | disalbe the same host resize | |
| 17:28:40 | sean-k-mooney | same host resize with pcpus is way more involved | |
| 17:28:47 | efried | so what we should really do is two GET /a_c calls | |
| 17:28:48 | sean-k-mooney | well not pcpu numa in general | |
| 17:29:24 | efried | and then combine the results. | |
| 17:29:24 | efried | one with in_tree=$cur_host with the diff'd set of resources | |
| 17:29:24 | efried | one with in_tree=!$cur_host with the full set of resources | |
| 17:29:56 | mriedem | umm, you're not talking about adding that into the series for train now are you? | |
| 17:30:05 | sean-k-mooney | that still has the double allocation proablem | |
| 17:30:12 | alex_xu | dansmith: I know the ignored_host from request spec, so I can check that, if the allocation_candidates return the host in the ignore_hosts, then fallback to VCPU | |
| 17:30:17 | efried | sean-k-mooney: how would it? | |
| 17:30:23 | mriedem | are you essentially trying to say, avoid the same host for a resize if the instance has PCPU resources? | |
| 17:30:35 | dansmith | alex_xu: you want to fall back to vcpu allocation if the only host that gets returned is the one you're on? | |
| 17:30:39 | dansmith | alex_xu: that's....crazy | |
| 17:30:54 | sean-k-mooney | the in_tree=$cur_host call would require double the resouces | |
| 17:31:01 | sean-k-mooney | the current resouce and the resize to resouces | |
| 17:31:07 | dansmith | alex_xu: or are you saying fall back to vcpu allocation but with the current host excluded? | |
| 17:31:26 | sean-k-mooney | anyway the numa toplogy filter requires double the resouces so its not a new thing | |
| 17:31:32 | efried | sean-k-mooney: no, that's why the in_tree=$cur_host is made with the *diff* of the current and new resources (only the positive ones) | |
| 17:31:45 | efried | (this is all theoretical btw) | |
| 17:31:52 | alex_xu | dansmith: fallback to vcpu if the only host get returned is the one you are on. After fallback vcpu, I won't get VCPu availalbe on the source host again | |
| 17:32:01 | sean-k-mooney | efried: i mention this on irc last week | |
| 17:32:06 | sean-k-mooney | in placment channel | |
| 17:32:23 | sean-k-mooney | efried: i have a customer request to fix inplemce rebuild with numa toplogy | |
| 17:32:37 | efried | sean-k-mooney: Yeah, I thought it sounded familiar. | |
| 17:32:39 | sean-k-mooney | so i or someone on my team needs to actully make this work in U | |
| 17:32:47 | cdent | i confirm that in_tree=! hasn't happened yet (sorry was away from computer for a few minutes doing network management) | |
| 17:32:53 | sean-k-mooney | and backport somthi esle to queens | |
| 17:33:01 | mriedem | sean-k-mooney: there is another nickel | |
| 17:33:18 | mriedem | sean-k-mooney: remember when you said you'd make sriov pci device live migration claims part of the actual claims code in U too/ | |
| 17:33:19 | mriedem | ? | |
| 17:33:32 | sean-k-mooney | mriedem: yes but this time we have the customer wanting use to ship them the backported code in october | |
| 17:33:34 | dansmith | U is going to be a big cycle.. HUGE. YUGE. | |
| 17:33:42 | dtroyer | efried, sean-k-mooney, alex_xu: status update: we've re-triggered the pmem job on Intel CI with the changes on PS22, should have some results in about 45 min… | |
| 17:33:52 | mriedem | sean-k-mooney: backported code for PCPU? | |
| 17:33:54 | efried | dansmith: no it's not. | |
| 17:33:58 | alex_xu | dtroyer: thanks | |
| 17:33:58 | mriedem | including all of the upgrade shit that entails? | |
| 17:33:59 | dtroyer | https://review.opendev.org/#/c/679640/22 | |
| 17:34:02 | sean-k-mooney | mriedem: no inplace numa vma rebuild | |
| 17:34:10 | efried | thanks dtroyer | |
| 17:34:24 | mriedem | inplace numa vma rebuild? | |
| 17:34:25 | efried | it appears that 'pleaselookagain' is the magic invocation for pmem CI | |
| 17:34:37 | mriedem | i'm lost - i don't know what that has to do with what we're talking about | |
| 17:34:47 | alex_xu | mriedem: https://review.opendev.org/#/c/681383/5/nova/scheduler/manager.py@155 | |
| 17:34:59 | alex_xu | mriedem: or you can see the PS2 comment I give | |
| 17:35:48 | sean-k-mooney | mriedem: https://bugzilla.redhat.com/show_bug.cgi?id=1700412 | |
| 17:35:49 | efried | dansmith: If the PTL has anything to say about it, we're going to accept only blueprints that were already approved or proposed-but-didn't-make-spec-freeze | |
| 17:35:50 | openstack | bugzilla.redhat.com bug 1700412 in openstack-nova "[RFE] Rebuild with a different image fails due to NUMATopologyFilter" [High,New] - Assigned to nova-maint | |
| 17:35:51 | mriedem | "if same-host resize is disabled and the only host available is the host you're on there are resource problems" is not a new issue | |
| 17:36:13 | mriedem | ^ is also a problem for servers in a strict affinity group | |
| 17:36:18 | mriedem | since you can't resize out of that | |
| 17:36:27 | dansmith | efried: good luck with that | |