1 From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2 From: Andy Lutomirski <luto@kernel.org>
3 Date: Mon, 4 Dec 2017 15:07:24 +0100
4 Subject: [PATCH] x86/entry/64: Return to userspace from the trampoline stack
6 Content-Type: text/plain; charset=UTF-8
7 Content-Transfer-Encoding: 8bit
11 By itself, this is useless. It gives us the ability to run some final code
12 before exit that cannnot run on the kernel stack. This could include a CR3
13 switch a la PAGE_TABLE_ISOLATION or some kernel stack erasing, for
14 example. (Or even weird things like *changing* which kernel stack gets
15 used as an ASLR-strengthening mechanism.)
17 The SYSRET32 path is not covered yet. It could be in the future or
18 we could just ignore it and force the slow path if needed.
20 Signed-off-by: Andy Lutomirski <luto@kernel.org>
21 Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
22 Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
23 Reviewed-by: Borislav Petkov <bp@suse.de>
24 Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
25 Cc: Borislav Petkov <bp@alien8.de>
26 Cc: Borislav Petkov <bpetkov@suse.de>
27 Cc: Brian Gerst <brgerst@gmail.com>
28 Cc: Dave Hansen <dave.hansen@intel.com>
29 Cc: Dave Hansen <dave.hansen@linux.intel.com>
30 Cc: David Laight <David.Laight@aculab.com>
31 Cc: Denys Vlasenko <dvlasenk@redhat.com>
32 Cc: Eduardo Valentin <eduval@amazon.com>
33 Cc: Greg KH <gregkh@linuxfoundation.org>
34 Cc: H. Peter Anvin <hpa@zytor.com>
35 Cc: Josh Poimboeuf <jpoimboe@redhat.com>
36 Cc: Juergen Gross <jgross@suse.com>
37 Cc: Linus Torvalds <torvalds@linux-foundation.org>
38 Cc: Peter Zijlstra <peterz@infradead.org>
39 Cc: Rik van Riel <riel@redhat.com>
40 Cc: Will Deacon <will.deacon@arm.com>
41 Cc: aliguori@amazon.com
42 Cc: daniel.gruss@iaik.tugraz.at
44 Cc: keescook@google.com
45 Link: https://lkml.kernel.org/r/20171204150606.306546484@linutronix.de
46 Signed-off-by: Ingo Molnar <mingo@kernel.org>
47 (cherry picked from commit 3e3b9293d392c577b62e24e4bc9982320438e749)
48 Signed-off-by: Andy Whitcroft <apw@canonical.com>
49 Signed-off-by: Kleber Sacilotto de Souza <kleber.souza@canonical.com>
50 (cherry picked from commit 40eb58584f732a2fefb5959e79e408bedeaaa43c)
51 Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
53 arch/x86/entry/entry_64.S | 55 +++++++++++++++++++++++++++++++++++++++++++----
54 1 file changed, 51 insertions(+), 4 deletions(-)
56 diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
57 index f70fedc58bac..4abe5b806d2a 100644
58 --- a/arch/x86/entry/entry_64.S
59 +++ b/arch/x86/entry/entry_64.S
60 @@ -325,8 +325,24 @@ syscall_return_via_sysret:
61 popq %rsi /* skip rcx */
66 + * Now all regs are restored except RSP and RDI.
67 + * Save old stack pointer and switch to trampoline stack.
70 + movq PER_CPU_VAR(cpu_tss + TSS_sp0), %rsp
72 + pushq RSP-RDI(%rdi) /* RSP */
73 + pushq (%rdi) /* RDI */
76 + * We are on the trampoline stack. All regs except RDI are live.
77 + * We can do future final exit work right here.
81 - movq RSP-ORIG_RAX(%rsp), %rsp
86 @@ -629,10 +645,41 @@ GLOBAL(swapgs_restore_regs_and_return_to_usermode)
93 - addq $8, %rsp /* skip regs->orig_ax */
104 + * The stack is now user RDI, orig_ax, RIP, CS, EFLAGS, RSP, SS.
105 + * Save old stack pointer and switch to trampoline stack.
108 + movq PER_CPU_VAR(cpu_tss + TSS_sp0), %rsp
110 + /* Copy the IRET frame to the trampoline stack. */
111 + pushq 6*8(%rdi) /* SS */
112 + pushq 5*8(%rdi) /* RSP */
113 + pushq 4*8(%rdi) /* EFLAGS */
114 + pushq 3*8(%rdi) /* CS */
115 + pushq 2*8(%rdi) /* RIP */
117 + /* Push user RDI on the trampoline stack. */
121 + * We are on the trampoline stack. All regs except RDI are live.
122 + * We can do future final exit work right here.