]> git.proxmox.com Git - mirror_ubuntu-kernels.git/commit
mm/tlb, x86/mm: Support invalidating TLB caches for RCU_TABLE_FREE
authorPeter Zijlstra <peterz@infradead.org>
Wed, 22 Aug 2018 15:30:15 +0000 (17:30 +0200)
committerLinus Torvalds <torvalds@linux-foundation.org>
Thu, 23 Aug 2018 18:55:58 +0000 (11:55 -0700)
commitd86564a2f085b79ec046a5cba90188e612352806
tree18172b419a9ec64fe7e2e569726a1e8db498a67c
parenta6f572084fbee8b30f91465f4a085d7a90901c57
mm/tlb, x86/mm: Support invalidating TLB caches for RCU_TABLE_FREE

Jann reported that x86 was missing required TLB invalidates when he
hit the !*batch slow path in tlb_remove_table().

This is indeed the case; RCU_TABLE_FREE does not provide TLB (cache)
invalidates, the PowerPC-hash where this code originated and the
Sparc-hash where this was subsequently used did not need that. ARM
which later used this put an explicit TLB invalidate in their
__p*_free_tlb() functions, and PowerPC-radix followed that example.

But when we hooked up x86 we failed to consider this. Fix this by
(optionally) hooking tlb_remove_table() into the TLB invalidate code.

NOTE: s390 was also needing something like this and might now
      be able to use the generic code again.

[ Modified to be on top of Nick's cleanups, which simplified this patch
  now that tlb_flush_mmu_tlbonly() really only flushes the TLB - Linus ]

Fixes: 9e52fc2b50de ("x86/mm: Enable RCU based page table freeing (CONFIG_HAVE_RCU_TABLE_FREE=y)")
Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Rik van Riel <riel@surriel.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: David Miller <davem@davemloft.net>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: stable@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
arch/Kconfig
arch/x86/Kconfig
mm/memory.c