Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-09-11
15:07:26 mriedem artom: i left a suggestion on wording
15:07:28 stephenfin luyao: Implicit NUMA affinity is what we do for hugepages and CPU pinning. If you use either of those features, your guest will have a one-node NUMA topology and it will be pinned to a host NUMA node
15:07:45 stephenfin alex_xu: Yeah, an implicit NUMA topology
15:08:19 luyao stephenfin: I thought implicit Alex said is the guest numa not bind to host numa
15:08:24 mriedem stephenfin: oh i have a solution - drop the numa part but only support vpmems on ppc64 https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_domain.c#L11604-L11615 :)
15:08:32 mriedem power to the rescue
15:08:47 stephenfin fight the power
15:08:51 luyao stephenfin, alex_xu: so you are discuss different things ?
15:09:15 mriedem does tonyb have openshift working on power yet?
15:09:23 stephenfin luyao: no, the disagreement is on the lack of pinning
15:09:45 artom mriedem, https://review.opendev.org/#/c/640021/50/nova/conductor/tasks/live_migrate.py@200
15:10:40 stephenfin the fact that for hugepages, the single guest NUMA cell is pinned to a host NUMA cell, but for VPMEM, the single guest NUMA cell floats across all host NUMA cells
15:10:48 stephenfin *for hugepages and CPU pinning
15:11:19 alex_xu stephenfin: yes, so you want the pinning, but is it ok without affinity for now?
15:11:25 stephenfin bauzas: Think you could hit these two too? https://review.opendev.org/#/c/681060/ https://review.opendev.org/#/c/681061/
15:11:43 stephenfin alex_xu: I'll take that as a middle ground, yes
15:12:04 bauzas stephenfin: looking
15:12:15 alex_xu stephenfin: so the only I need is fall into this branch https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1740, right
15:12:24 stephenfin yup
15:12:28 alex_xu cool
15:12:57 mriedem artom: replied
15:13:02 mriedem with anger
15:13:07 luyao stephenfin: Does it make sense to do the numa pinning now without affinity.
15:13:39 sean-k-mooney luyao: yes
15:13:47 stephenfin luyao: It's definitely sub-optimal, I agree, but it's better than breaking our long-standing one guest NUMA node == one host NUMA node policy
15:14:01 artom mriedem, ok, there's something I genuinely don't get then
15:14:01 sean-k-mooney its not ideal but its not a bad thing
15:14:09 luyao sean-k-mooney: Okay
15:14:14 stephenfin you'll get inferior performance but from a user perspective, things behavior as expected
15:14:20 artom Not trying to make you angry on purpose, for serious
15:14:32 stephenfin *behave
15:14:59 artom If the conductor's train, and the source is train, and the dest is train, but there are other stein computes in the cell
15:15:15 artom We'll get past the min sevice version check
15:15:39 artom get past = not return
15:15:49 artom Meaning we check the value of the workaround
15:15:54 artom If the workaround is disabled, that's that
15:16:22 artom If the workaround is enabled, in *this specific case of dest and source being new*
15:16:27 artom We'll get NUMA LM
15:17:47 mriedem artom: even if the source is new and you checked that, the dest - which we might not have yet if the user didn't force the dest from the api and bypass the scheduler - might not be new, and we won't know in that code with the warning log message because it comes before we ask the scheduler for a dest
15:18:15 mriedem so if we get lucky and the source host and selected dest host are new enough, yeah it might be fine
15:18:24 mriedem but,
15:18:26 dansmith mriedem: I don't think that's whathe's saying
15:18:29 mriedem it probably won't be because if you have old computes,
15:18:42 mriedem rpc is going to be pinned and we'll backlevel the new numa migrate_data object stuff
15:18:56 dansmith this ^ is what he's saying
15:19:14 dansmith so he just doesn't want to specifically say that the src or dst is too old.. right?
15:19:23 mriedem we never asked him to say that
15:19:35 mriedem we just said "say something about computes not being upgraded yet"
15:19:38 dansmith I don't think he claimed we did
15:19:46 mriedem who's on first?
15:19:55 dansmith I think he just said he's trying to wordsmith a suitably generic message right?
15:20:03 mriedem i provided one
15:20:16 artom Well there's the logging on L196 as well - it all fits together
15:20:32 artom Or I'm just overcomplicating everything?
15:20:50 dansmith I don't see any logging on L 196
15:21:14 artom You asked for it :)
15:21:40 dansmith you mean the logging that would be on L196 if you updated?
15:21:40 mriedem artom: i think dan just meant adjusting the msg object
15:21:53 mriedem if the config is enabled, we log a warning
15:21:58 mriedem else we raise, but with the same message
15:22:21 dansmith mriedem: I was suggesting one more log message inside the service version check
15:23:10 mriedem ok i'm going to move onto reviewing gibi's stuff now
15:23:14 artom Haha
15:23:45 dansmith how about artom just fixes the test issue, adds logs like he thinks he needs, and then we can sort out the log minutia in a later patch if necessary?
15:24:20 bauzas stephenfin: good news, I now start looking at your reshaper change :)
15:24:22 mriedem that's fine
15:24:30 stephenfin huzzah!
15:26:15 gibi_fly mriedem: hit me with any questions
15:27:14 artom dansmith, btw, the compute RPC version is per-cell, right? So, if we're in a cell with mixed stein/train computes, none of the will be able to send 5.3?
15:27:46 artom There's no backporting like what's done for objects - if a node can send 5.3, then all others must be able to receive it?
15:27:48 dansmith artom: not really, but I definitely do not think that level of detail needs to be in there
15:27:49 efried http://52.27.155.124/74/681474/2/check/pmem-tempest-plugin-filtered/134b243/job-output.txt.gz @ 2019-09-11 14:17:14.914120 is still applying PS13
15:27:49 efried sean-k-mooney: https://review.opendev.org/#/c/681474/2 appears not to have worked --
15:28:05 artom dansmith, yeah, maybe I'm overthinking it
15:29:27 efried sean-k-mooney, dtroyer: I can't tell by looking at https://github.com/intel/Intel-OpenStack-CI-jobs/pull/1/commits/5d4a998a890fef3a2b0a46f73d26aa1deeb3cd8b -- is this merged?
15:29:29 sean-k-mooney efried in that case they are likely using the git plug not the gitub plugin
15:30:09 sean-k-mooney efried: the zuul git driver does not support pull request but works with any git repo via a timer
15:30:27 sean-k-mooney the gitub one require the zuul app to be installed in the github org
15:30:35 sean-k-mooney but once that is done pull requeist work
15:30:47 sean-k-mooney i suspect intel it dont allow the zuul app
15:30:49 efried sean-k-mooney, dtroyer: I wanted to just recheck one of the top vpmem patches to get new results
15:31:03 efried but no point unless that pr merged
15:32:04 mriedem gibi_fly:
15:32:05 mriedem https://review.opendev.org/#/c/676980/19
15:32:11 gibi_fly mriedem: looking
15:33:22 bauzas I believe I can gibi_fly
15:33:32 bauzas (sorry)
15:33:35 mriedem get out
15:35:16 gibi_fly mriedem: good points. I will fix it up today.
15:36:01 sean-k-mooney efried its not merged by the way
15:36:02 gibi_fly bauzas: I will change to a bus soon so you were on time with that joke :)
15:36:22 bauzas gibi_fly: do you need help for providing a new revision ?
15:36:23 sean-k-mooney efried: the PR woudl be closed if it was merged
15:36:39 bauzas gibi_fly: since you're a bit on-off
15:37:04 bauzas FWIW, I starbucks'd last Friday and it wasn't a great experience
15:37:06 gibi_fly bauzas: thanks. I will manage
15:37:18 gibi_fly bauzas: I will ping you for review :)
15:37:21 bauzas gibi_fly: cool
15:37:55 gibi_fly bauzas: I don't know which net is better, plane or starbucks
15:38:05 efried sean-k-mooney: okay, that's what I thought, but dtroyer said he merged it so I wasn't sure if I just didn't know how to read the thing (which would be totally likely)

Earlier   Later