x86/xen: allow nesting of same lazy mode

When running as a paravirtualized guest under Xen, Linux is using
"lazy mode" for issuing hypercalls which don't need to take immediate
effect in order to improve performance (examples are e.g. multiple
PTE changes).

There are two different lazy modes defined: MMU and CPU lazy mode.
Today it is not possible to nest multiple lazy mode sections, even if
they are of the same kind. A recent change in memory management added
nesting of MMU lazy mode sections, resulting in a regression when
running as Xen PV guest.

Technically there is no reason why nesting of multiple sections of the
same kind of lazy mode shouldn't be allowed. So add support for that
for fixing the regression.

Fixes: bcc6cc832573 ("mm: add default definition of set_ptes()")
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://lore.kernel.org/r/20230913113828.18421-4-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>

diff --git a/arch/x86/include/asm/xen/hypervisor.h b/arch/x86/include/asm/xen/hypervisor.h
index ed05ce3df5c7b55a5c9496648dbf145862b5f183..7048dfacc04b2413acba1054fb4412c7048b8249 100644 (file)
--- a/arch/x86/include/asm/xen/hypervisor.h
+++ b/arch/x86/include/asm/xen/hypervisor.h
@@ -72,10 +72,18 @@ enum xen_lazy_mode {
 };
 
 DECLARE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode);
+DECLARE_PER_CPU(unsigned int, xen_lazy_nesting);
 
 static inline void enter_lazy(enum xen_lazy_mode mode)
 {
-       BUG_ON(this_cpu_read(xen_lazy_mode) != XEN_LAZY_NONE);
+       enum xen_lazy_mode old_mode = this_cpu_read(xen_lazy_mode);
+
+       if (mode == old_mode) {
+               this_cpu_inc(xen_lazy_nesting);
+               return;
+       }
+
+       BUG_ON(old_mode != XEN_LAZY_NONE);
 
        this_cpu_write(xen_lazy_mode, mode);
 }
@@ -84,7 +92,10 @@ static inline void leave_lazy(enum xen_lazy_mode mode)
 {
        BUG_ON(this_cpu_read(xen_lazy_mode) != mode);
 
-       this_cpu_write(xen_lazy_mode, XEN_LAZY_NONE);
+       if (this_cpu_read(xen_lazy_nesting) == 0)
+               this_cpu_write(xen_lazy_mode, XEN_LAZY_NONE);
+       else
+               this_cpu_dec(xen_lazy_nesting);
 }
 
 enum xen_lazy_mode xen_get_lazy_mode(void);

diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 54b83825c4b618d8da88a7730369054a28346692..bbbfdd495ebd3ac590fd152e0504c6bedae611de 100644 (file)
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -102,6 +102,7 @@ struct tls_descs {
 };
 
 DEFINE_PER_CPU(enum xen_lazy_mode, xen_lazy_mode) = XEN_LAZY_NONE;
+DEFINE_PER_CPU(unsigned int, xen_lazy_nesting);
 
 enum xen_lazy_mode xen_get_lazy_mode(void)
 {