1 \input texinfo @c -*- texinfo -*-
4 @settitle QEMU Internals
7 @center @titlefont{QEMU Internals}
16 QEMU is a FAST! processor emulator using a portable dynamic
19 QEMU has two operating modes:
24 Full system emulation. In this mode, QEMU emulates a full system
25 (usually a PC), including a processor and various peripherials. It can
26 be used to launch an different Operating System without rebooting the
27 PC or to debug system code.
30 User mode emulation (Linux host only). In this mode, QEMU can launch
31 Linux processes compiled for one CPU on another CPU. It can be used to
32 launch the Wine Windows API emulator (@url{http://www.winehq.org}) or
33 to ease cross-compilation and cross-debugging.
37 As QEMU requires no host kernel driver to run, it is very safe and
40 QEMU generic features:
44 @item User space only or full system emulation.
46 @item Using dynamic translation to native code for reasonnable speed.
48 @item Working on x86 and PowerPC hosts. Being tested on ARM, Sparc32, Alpha and S390.
50 @item Self-modifying code support.
52 @item Precise exceptions support.
54 @item The virtual CPU is a library (@code{libqemu}) which can be used
59 QEMU user mode emulation features:
61 @item Generic Linux system call converter, including most ioctls.
63 @item clone() emulation using native CPU clone() to use Linux scheduler for threads.
65 @item Accurate signal handling by remapping host signals to target signals.
69 QEMU full system emulation features:
71 @item QEMU can either use a full software MMU for maximum portability or use the host system call mmap() to simulate the target MMU.
74 @section x86 emulation
76 QEMU x86 target features:
80 @item The virtual x86 CPU supports 16 bit and 32 bit addressing with segmentation.
81 LDT/GDT and IDT are emulated. VM86 mode is also supported to run DOSEMU.
83 @item Support of host page sizes bigger than 4KB in user mode emulation.
85 @item QEMU can emulate itself on x86.
87 @item An extensive Linux x86 CPU test program is included @file{tests/test-i386}.
88 It can be used to test other x86 virtual CPUs.
92 Current QEMU limitations:
96 @item No SSE/MMX support (yet).
98 @item No x86-64 support.
100 @item IPC syscalls are missing.
102 @item The x86 segment limits and access rights are not tested at every
103 memory access (yet). Hopefully, very few OSes seem to rely on that for
106 @item On non x86 host CPUs, @code{double}s are used instead of the non standard
107 10 byte @code{long double}s of x86 for floating point emulation to get
108 maximum performances.
112 @section ARM emulation
116 @item Full ARM 7 user emulation.
118 @item NWFPE FPU support included in user Linux emulation.
120 @item Can run most ARM Linux binaries.
124 @section PowerPC emulation
128 @item Full PowerPC 32 bit emulation, including priviledged instructions,
131 @item Can run most PowerPC Linux binaries.
135 @section SPARC emulation
139 @item SPARC V8 user support, except FPU instructions.
141 @item Can run some SPARC Linux binaries.
145 @chapter QEMU Internals
147 @section QEMU compared to other emulators
149 Like bochs [3], QEMU emulates an x86 CPU. But QEMU is much faster than
150 bochs as it uses dynamic compilation. Bochs is closely tied to x86 PC
151 emulation while QEMU can emulate several processors.
153 Like Valgrind [2], QEMU does user space emulation and dynamic
154 translation. Valgrind is mainly a memory debugger while QEMU has no
155 support for it (QEMU could be used to detect out of bound memory
156 accesses as Valgrind, but it has no support to track uninitialised data
157 as Valgrind does). The Valgrind dynamic translator generates better code
158 than QEMU (in particular it does register allocation) but it is closely
159 tied to an x86 host and target and has no support for precise exceptions
160 and system emulation.
162 EM86 [4] is the closest project to user space QEMU (and QEMU still uses
163 some of its code, in particular the ELF file loader). EM86 was limited
164 to an alpha host and used a proprietary and slow interpreter (the
165 interpreter part of the FX!32 Digital Win32 code translator [5]).
167 TWIN [6] is a Windows API emulator like Wine. It is less accurate than
168 Wine but includes a protected mode x86 interpreter to launch x86 Windows
169 executables. Such an approach as greater potential because most of the
170 Windows API is executed natively but it is far more difficult to develop
171 because all the data structures and function parameters exchanged
172 between the API and the x86 code must be converted.
174 User mode Linux [7] was the only solution before QEMU to launch a
175 Linux kernel as a process while not needing any host kernel
176 patches. However, user mode Linux requires heavy kernel patches while
177 QEMU accepts unpatched Linux kernels. The price to pay is that QEMU is
180 The new Plex86 [8] PC virtualizer is done in the same spirit as the
181 qemu-fast system emulator. It requires a patched Linux kernel to work
182 (you cannot launch the same kernel on your PC), but the patches are
183 really small. As it is a PC virtualizer (no emulation is done except
184 for some priveledged instructions), it has the potential of being
185 faster than QEMU. The downside is that a complicated (and potentially
186 unsafe) host kernel patch is needed.
188 The commercial PC Virtualizers (VMWare [9], VirtualPC [10], TwoOStwo
189 [11]) are faster than QEMU, but they all need specific, proprietary
190 and potentially unsafe host drivers. Moreover, they are unable to
191 provide cycle exact simulation as an emulator can.
193 @section Portable dynamic translation
195 QEMU is a dynamic translator. When it first encounters a piece of code,
196 it converts it to the host instruction set. Usually dynamic translators
197 are very complicated and highly CPU dependent. QEMU uses some tricks
198 which make it relatively easily portable and simple while achieving good
201 The basic idea is to split every x86 instruction into fewer simpler
202 instructions. Each simple instruction is implemented by a piece of C
203 code (see @file{target-i386/op.c}). Then a compile time tool
204 (@file{dyngen}) takes the corresponding object file (@file{op.o})
205 to generate a dynamic code generator which concatenates the simple
206 instructions to build a function (see @file{op.h:dyngen_code()}).
208 In essence, the process is similar to [1], but more work is done at
211 A key idea to get optimal performances is that constant parameters can
212 be passed to the simple operations. For that purpose, dummy ELF
213 relocations are generated with gcc for each constant parameter. Then,
214 the tool (@file{dyngen}) can locate the relocations and generate the
215 appriopriate C code to resolve them when building the dynamic code.
217 That way, QEMU is no more difficult to port than a dynamic linker.
219 To go even faster, GCC static register variables are used to keep the
220 state of the virtual CPU.
222 @section Register allocation
224 Since QEMU uses fixed simple instructions, no efficient register
225 allocation can be done. However, because RISC CPUs have a lot of
226 register, most of the virtual CPU state can be put in registers without
227 doing complicated register allocation.
229 @section Condition code optimisations
231 Good CPU condition codes emulation (@code{EFLAGS} register on x86) is a
232 critical point to get good performances. QEMU uses lazy condition code
233 evaluation: instead of computing the condition codes after each x86
234 instruction, it just stores one operand (called @code{CC_SRC}), the
235 result (called @code{CC_DST}) and the type of operation (called
238 @code{CC_OP} is almost never explicitely set in the generated code
239 because it is known at translation time.
241 In order to increase performances, a backward pass is performed on the
242 generated simple instructions (see
243 @code{target-i386/translate.c:optimize_flags()}). When it can be proved that
244 the condition codes are not needed by the next instructions, no
245 condition codes are computed at all.
247 @section CPU state optimisations
249 The x86 CPU has many internal states which change the way it evaluates
250 instructions. In order to achieve a good speed, the translation phase
251 considers that some state information of the virtual x86 CPU cannot
252 change in it. For example, if the SS, DS and ES segments have a zero
253 base, then the translator does not even generate an addition for the
256 [The FPU stack pointer register is not handled that way yet].
258 @section Translation cache
260 A 2MByte cache holds the most recently used translations. For
261 simplicity, it is completely flushed when it is full. A translation unit
262 contains just a single basic block (a block of x86 instructions
263 terminated by a jump or by a virtual CPU state change which the
264 translator cannot deduce statically).
266 @section Direct block chaining
268 After each translated basic block is executed, QEMU uses the simulated
269 Program Counter (PC) and other cpu state informations (such as the CS
270 segment base value) to find the next basic block.
272 In order to accelerate the most common cases where the new simulated PC
273 is known, QEMU can patch a basic block so that it jumps directly to the
276 The most portable code uses an indirect jump. An indirect jump makes
277 it easier to make the jump target modification atomic. On some host
278 architectures (such as x86 or PowerPC), the @code{JUMP} opcode is
279 directly patched so that the block chaining has no overhead.
281 @section Self-modifying code and translated code invalidation
283 Self-modifying code is a special challenge in x86 emulation because no
284 instruction cache invalidation is signaled by the application when code
287 When translated code is generated for a basic block, the corresponding
288 host page is write protected if it is not already read-only (with the
289 system call @code{mprotect()}). Then, if a write access is done to the
290 page, Linux raises a SEGV signal. QEMU then invalidates all the
291 translated code in the page and enables write accesses to the page.
293 Correct translated code invalidation is done efficiently by maintaining
294 a linked list of every translated block contained in a given page. Other
295 linked lists are also maintained to undo direct block chaining.
297 Although the overhead of doing @code{mprotect()} calls is important,
298 most MSDOS programs can be emulated at reasonnable speed with QEMU and
301 Note that QEMU also invalidates pages of translated code when it detects
302 that memory mappings are modified with @code{mmap()} or @code{munmap()}.
304 When using a software MMU, the code invalidation is more efficient: if
305 a given code page is invalidated too often because of write accesses,
306 then a bitmap representing all the code inside the page is
307 built. Every store into that page checks the bitmap to see if the code
308 really needs to be invalidated. It avoids invalidating the code when
309 only data is modified in the page.
311 @section Exception support
313 longjmp() is used when an exception such as division by zero is
316 The host SIGSEGV and SIGBUS signal handlers are used to get invalid
317 memory accesses. The exact CPU state can be retrieved because all the
318 x86 registers are stored in fixed host registers. The simulated program
319 counter is found by retranslating the corresponding basic block and by
320 looking where the host program counter was at the exception point.
322 The virtual CPU cannot retrieve the exact @code{EFLAGS} register because
323 in some cases it is not computed because of condition code
324 optimisations. It is not a big concern because the emulated code can
325 still be restarted in any cases.
327 @section MMU emulation
329 For system emulation, QEMU uses the mmap() system call to emulate the
330 target CPU MMU. It works as long the emulated OS does not use an area
331 reserved by the host OS (such as the area above 0xc0000000 on x86
334 In order to be able to launch any OS, QEMU also supports a soft
335 MMU. In that mode, the MMU virtual to physical address translation is
336 done at every memory access. QEMU uses an address translation cache to
337 speed up the translation.
339 In order to avoid flushing the translated code each time the MMU
340 mappings change, QEMU uses a physically indexed translation cache. It
341 means that each basic block is indexed with its physical address.
343 When MMU mappings change, only the chaining of the basic blocks is
344 reset (i.e. a basic block can no longer jump directly to another one).
346 @section Hardware interrupts
348 In order to be faster, QEMU does not check at every basic block if an
349 hardware interrupt is pending. Instead, the user must asynchrously
350 call a specific function to tell that an interrupt is pending. This
351 function resets the chaining of the currently executing basic
352 block. It ensures that the execution will return soon in the main loop
353 of the CPU emulator. Then the main loop can test if the interrupt is
354 pending and handle it.
356 @section User emulation specific details
358 @subsection Linux system call translation
360 QEMU includes a generic system call translator for Linux. It means that
361 the parameters of the system calls can be converted to fix the
362 endianness and 32/64 bit issues. The IOCTLs are converted with a generic
363 type description system (see @file{ioctls.h} and @file{thunk.c}).
365 QEMU supports host CPUs which have pages bigger than 4KB. It records all
366 the mappings the process does and try to emulated the @code{mmap()}
367 system calls in cases where the host @code{mmap()} call would fail
368 because of bad page alignment.
370 @subsection Linux signals
372 Normal and real-time signals are queued along with their information
373 (@code{siginfo_t}) as it is done in the Linux kernel. Then an interrupt
374 request is done to the virtual CPU. When it is interrupted, one queued
375 signal is handled by generating a stack frame in the virtual CPU as the
376 Linux kernel does. The @code{sigreturn()} system call is emulated to return
377 from the virtual signal handler.
379 Some signals (such as SIGALRM) directly come from the host. Other
380 signals are synthetized from the virtual CPU exceptions such as SIGFPE
381 when a division by zero is done (see @code{main.c:cpu_loop()}).
383 The blocked signal mask is still handled by the host Linux kernel so
384 that most signal system calls can be redirected directly to the host
385 Linux kernel. Only the @code{sigaction()} and @code{sigreturn()} system
386 calls need to be fully emulated (see @file{signal.c}).
388 @subsection clone() system call and threads
390 The Linux clone() system call is usually used to create a thread. QEMU
391 uses the host clone() system call so that real host threads are created
392 for each emulated thread. One virtual CPU instance is created for each
395 The virtual x86 CPU atomic operations are emulated with a global lock so
396 that their semantic is preserved.
398 Note that currently there are still some locking issues in QEMU. In
399 particular, the translated cache flush is not protected yet against
402 @subsection Self-virtualization
404 QEMU was conceived so that ultimately it can emulate itself. Although
405 it is not very useful, it is an important test to show the power of the
408 Achieving self-virtualization is not easy because there may be address
409 space conflicts. QEMU solves this problem by being an executable ELF
410 shared object as the ld-linux.so ELF interpreter. That way, it can be
411 relocated at load time.
413 @section Bibliography
418 @url{http://citeseer.nj.nec.com/piumarta98optimizing.html}, Optimizing
419 direct threaded code by selective inlining (1998) by Ian Piumarta, Fabio
423 @url{http://developer.kde.org/~sewardj/}, Valgrind, an open-source
424 memory debugger for x86-GNU/Linux, by Julian Seward.
427 @url{http://bochs.sourceforge.net/}, the Bochs IA-32 Emulator Project,
428 by Kevin Lawton et al.
431 @url{http://www.cs.rose-hulman.edu/~donaldlf/em86/index.html}, the EM86
432 x86 emulator on Alpha-Linux.
435 @url{http://www.usenix.org/publications/library/proceedings/usenix-nt97/full_papers/chernoff/chernoff.pdf},
436 DIGITAL FX!32: Running 32-Bit x86 Applications on Alpha NT, by Anton
437 Chernoff and Ray Hookway.
440 @url{http://www.willows.com/}, Windows API library emulation from
444 @url{http://user-mode-linux.sourceforge.net/},
445 The User-mode Linux Kernel.
448 @url{http://www.plex86.org/},
449 The new Plex86 project.
452 @url{http://www.vmware.com/},
453 The VMWare PC virtualizer.
456 @url{http://www.microsoft.com/windowsxp/virtualpc/},
457 The VirtualPC PC virtualizer.
460 @url{http://www.twoostwo.org/},
461 The TwoOStwo PC virtualizer.
465 @chapter Regression Tests
467 In the directory @file{tests/}, various interesting testing programs
468 are available. There are used for regression testing.
470 @section @file{test-i386}
472 This program executes most of the 16 bit and 32 bit x86 instructions and
473 generates a text output. It can be compared with the output obtained with
474 a real CPU or another emulator. The target @code{make test} runs this
475 program and a @code{diff} on the generated output.
477 The Linux system call @code{modify_ldt()} is used to create x86 selectors
478 to test some 16 bit addressing and 32 bit with segmentation cases.
480 The Linux system call @code{vm86()} is used to test vm86 emulation.
482 Various exceptions are raised to test most of the x86 user space
485 @section @file{linux-test}
487 This program tests various Linux system calls. It is used to verify
488 that the system call parameters are correctly converted between target
491 @section @file{hello-i386}
493 Very simple statically linked x86 program, just to test QEMU during a
494 port to a new host CPU.
496 @section @file{hello-arm}
498 Very simple statically linked ARM program, just to test QEMU during a
499 port to a new host CPU.
503 It is a simple benchmark. Care must be taken to interpret the results
504 because it mostly tests the ability of the virtual CPU to optimize the
505 @code{rol} x86 instruction and the condition code computations.