patches/kernel/0009-KVM-x86-mmu-Grab-memslot-for-correct-address-space-i.patch

   1 From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
   2 From: Sean Christopherson <seanjc@google.com>
   3 Date: Thu, 1 Jun 2023 18:01:37 -0700
   4 Subject: [PATCH] KVM: x86/mmu: Grab memslot for correct address space in NX
   5  recovery worker
   6
   7 commit 817fa998362d6ea9fabd5e97af8e9e2eb5f0e6f2 upstream.
   8
   9 Factor in the address space (non-SMM vs. SMM) of the target shadow page
  10 when recovering potential NX huge pages, otherwise KVM will retrieve the
  11 wrong memslot when zapping shadow pages that were created for SMM.  The
  12 bug most visibly manifests as a WARN on the memslot being non-NULL, but
  13 the worst case scenario is that KVM could unaccount the shadow page
  14 without ensuring KVM won't install a huge page, i.e. if the non-SMM slot
  15 is being dirty logged, but the SMM slot is not.
  16
  17  ------------[ cut here ]------------
  18  WARNING: CPU: 1 PID: 3911 at arch/x86/kvm/mmu/mmu.c:7015
  19  kvm_nx_huge_page_recovery_worker+0x38c/0x3d0 [kvm]
  20  CPU: 1 PID: 3911 Comm: kvm-nx-lpage-re
  21  RIP: 0010:kvm_nx_huge_page_recovery_worker+0x38c/0x3d0 [kvm]
  22  RSP: 0018:ffff99b284f0be68 EFLAGS: 00010246
  23  RAX: 0000000000000000 RBX: ffff99b284edd000 RCX: 0000000000000000
  24  RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
  25  RBP: ffff9271397024e0 R08: 0000000000000000 R09: ffff927139702450
  26  R10: 0000000000000000 R11: 0000000000000001 R12: ffff99b284f0be98
  27  R13: 0000000000000000 R14: ffff9270991fcd80 R15: 0000000000000003
  28  FS:  0000000000000000(0000) GS:ffff927f9f640000(0000) knlGS:0000000000000000
  29  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  30  CR2: 00007f0aacad3ae0 CR3: 000000088fc2c005 CR4: 00000000003726e0
  31  Call Trace:
  32   <TASK>
  33 __pfx_kvm_nx_huge_page_recovery_worker+0x10/0x10 [kvm]
  34   kvm_vm_worker_thread+0x106/0x1c0 [kvm]
  35   kthread+0xd9/0x100
  36   ret_from_fork+0x2c/0x50
  37   </TASK>
  38  ---[ end trace 0000000000000000 ]---
  39
  40 This bug was exposed by commit edbdb43fc96b ("KVM: x86: Preserve TDP MMU
  41 roots until they are explicitly invalidated"), which allowed KVM to retain
  42 SMM TDP MMU roots effectively indefinitely.  Before commit edbdb43fc96b,
  43 KVM would zap all SMM TDP MMU roots and thus all SMM TDP MMU shadow pages
  44 once all vCPUs exited SMM, which made the window where this bug (recovering
  45 an SMM NX huge page) could be encountered quite tiny.  To hit the bug, the
  46 NX recovery thread would have to run while at least one vCPU was in SMM.
  47 Most VMs typically only use SMM during boot, and so the problematic shadow
  48 pages were gone by the time the NX recovery thread ran.
  49
  50 Now that KVM preserves TDP MMU roots until they are explicitly invalidated
  51 (e.g. by a memslot deletion), the window to trigger the bug is effectively
  52 never closed because most VMMs don't delete memslots after boot (except
  53 for a handful of special scenarios).
  54
  55 Fixes: eb298605705a ("KVM: x86/mmu: Do not recover dirty-tracked NX Huge Pages")
  56 Reported-by: Fabio Coatti <fabio.coatti@gmail.com>
  57 Closes: https://lore.kernel.org/all/CADpTngX9LESCdHVu_2mQkNGena_Ng2CphWNwsRGSMxzDsTjU2A@mail.gmail.com
  58 Cc: stable@vger.kernel.org
  59 Link: https://lore.kernel.org/r/20230602010137.784664-1-seanjc@google.com
  60 Signed-off-by: Sean Christopherson <seanjc@google.com>
  61 Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  62 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
  63 ---
  64  arch/x86/kvm/mmu/mmu.c | 5 ++++-
  65  1 file changed, 4 insertions(+), 1 deletion(-)
  66
  67 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
  68 index dcca08a08bd0..3220c1285984 100644
  69 --- a/arch/x86/kvm/mmu/mmu.c
  70 +++ b/arch/x86/kvm/mmu/mmu.c
  71 @@ -6945,7 +6945,10 @@ static void kvm_recover_nx_huge_pages(struct kvm *kvm)
  72                  */
  73                 slot = NULL;
  74                 if (atomic_read(&kvm->nr_memslots_dirty_logging)) {
  75 -                       slot = gfn_to_memslot(kvm, sp->gfn);
  76 +                       struct kvm_memslots *slots;
  77 +
  78 +                       slots = kvm_memslots_for_spte_role(kvm, sp->role);
  79 +                       slot = __gfn_to_memslot(slots, sp->gfn);
  80                         WARN_ON_ONCE(!slot);
  81                 }
  82