x86/mm/pat: Disable preemption around __flush_tlb_all()

BugLink: https://bugs.launchpad.net/bugs/1830433
The WARN_ON_ONCE(__read_cr3() != build_cr3()) in switch_mm_irqs_off()
triggers every once in a while during a snapshotted system upgrade.

The warning triggers since commit decab0888e6e ("x86/mm: Remove
preempt_disable/enable() from __native_flush_tlb()"). The callchain is:

  get_page_from_freelist() -> post_alloc_hook() -> __kernel_map_pages()

with CONFIG_DEBUG_PAGEALLOC enabled.

Disable preemption during CR3 reset / __flush_tlb_all() and add a comment
why preemption has to be disabled so it won't be removed accidentaly.

Add another preemptible() check in __flush_tlb_all() to catch callers with
enabled preemption when PGE is enabled, because PGE enabled does not
trigger the warning in __native_flush_tlb(). Suggested by Andy Lutomirski.

Fixes: decab0888e6e ("x86/mm: Remove preempt_disable/enable() from __native_flush_tlb()")
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20181017103432.zgv46nlu3hc7k4rq@linutronix.de
(cherry picked from commit f77084d96355f5fba8e2c1fb3a51a393b1570de7)
Tested-by: Connor Kuehl <connor.kuehl@canonical.com>
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
Acked-by: Stefan Bader <stefan.bader@canonical.com>
Acked-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
Signed-off-by: Khalid Elmously <khalid.elmously@canonical.com>

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 1bbc43077583702cff4dc78557ca9ff71f578455..1a56d1df5c216edfb451b7a0a38038594425e0b3 100644 (file)
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -473,6 +473,12 @@ static inline void __native_flush_tlb_one_user(unsigned long addr)
  */
 static inline void __flush_tlb_all(void)
 {
+       /*
+        * This is to catch users with enabled preemption and the PGE feature
+        * and don't trigger the warning in __native_flush_tlb().
+        */
+       VM_WARN_ON_ONCE(preemptible());
+
        if (boot_cpu_has(X86_FEATURE_PGE)) {
                __flush_tlb_global();
        } else {

diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 8d6c34fe49be9567157b65fa69d8d1059f7779ee..800de88208d7d61a117e53e2bfe2461fa4d2fcf7 100644 (file)
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -2063,9 +2063,13 @@ void __kernel_map_pages(struct page *page, int numpages, int enable)
 
        /*
         * We should perform an IPI and flush all tlbs,
-        * but that can deadlock->flush only current cpu:
+        * but that can deadlock->flush only current cpu.
+        * Preemption needs to be disabled around __flush_tlb_all() due to
+        * CR3 reload in __native_flush_tlb().
         */
+       preempt_disable();
        __flush_tlb_all();
+       preempt_enable();
 
        arch_flush_lazy_mmu_mode();
 }