]>
Commit | Line | Data |
---|---|---|
a14f0bf1 PB |
1 | ================================== |
2 | The QEMU build system architecture | |
3 | ================================== | |
717171bd DB |
4 | |
5 | This document aims to help developers understand the architecture of the | |
6 | QEMU build system. As with projects using GNU autotools, the QEMU build | |
7 | system has two stages, first the developer runs the "configure" script | |
8 | to determine the local build environment characteristics, then they run | |
9 | "make" to build the project. There is about where the similarities with | |
10 | GNU autotools end, so try to forget what you know about them. | |
11 | ||
12 | ||
13 | Stage 1: configure | |
14 | ================== | |
15 | ||
16 | The QEMU configure script is written directly in shell, and should be | |
17 | compatible with any POSIX shell, hence it uses #!/bin/sh. An important | |
18 | implication of this is that it is important to avoid using bash-isms on | |
19 | development platforms where bash is the primary host. | |
20 | ||
21 | In contrast to autoconf scripts, QEMU's configure is expected to be | |
22 | silent while it is checking for features. It will only display output | |
23 | when an error occurs, or to show the final feature enablement summary | |
24 | on completion. | |
25 | ||
77d27b92 PB |
26 | Because QEMU uses the Meson build system under the hood, only VPATH |
27 | builds are supported. There are two general ways to invoke configure & | |
28 | perform a build: | |
29 | ||
a14f0bf1 | 30 | - VPATH, build artifacts outside of QEMU source tree entirely:: |
77d27b92 PB |
31 | |
32 | cd ../ | |
33 | mkdir build | |
34 | cd build | |
35 | ../qemu/configure | |
36 | make | |
37 | ||
a14f0bf1 | 38 | - VPATH, build artifacts in a subdir of QEMU source tree:: |
77d27b92 PB |
39 | |
40 | mkdir build | |
41 | cd build | |
42 | ../configure | |
43 | make | |
44 | ||
45 | For now, checks on the compilation environment are found in configure | |
46 | rather than meson.build, though this is expected to change. The command | |
47 | line is parsed in the configure script and, whenever needed, converted | |
48 | into the appropriate options to Meson. | |
717171bd | 49 | |
77d27b92 PB |
50 | New checks should be added to Meson, which usually comprises the |
51 | following tasks: | |
717171bd | 52 | |
77d27b92 | 53 | - Add a Meson build option to meson_options.txt. |
717171bd DB |
54 | |
55 | - Add support to the command line arg parser to handle any new | |
a14f0bf1 | 56 | `--enable-XXX`/`--disable-XXX` flags required by the feature. |
717171bd DB |
57 | |
58 | - Add information to the help output message to report on the new | |
59 | feature flag. | |
60 | ||
77d27b92 PB |
61 | - Add code to perform the actual feature check. |
62 | ||
a14f0bf1 | 63 | - Add code to include the feature status in `config-host.h` |
717171bd DB |
64 | |
65 | - Add code to print out the feature status in the configure summary | |
66 | upon completion. | |
67 | ||
717171bd | 68 | |
1a94933f | 69 | Taking the probe for SDL2_Image as an example, we have the following pieces |
a14f0bf1 | 70 | in configure:: |
717171bd DB |
71 | |
72 | # Initial variable state | |
1a94933f | 73 | sdl_image=auto |
717171bd DB |
74 | |
75 | ..snip.. | |
76 | ||
77 | # Configure flag processing | |
1a94933f | 78 | --disable-sdl-image) sdl_image=disabled |
717171bd | 79 | ;; |
1a94933f | 80 | --enable-sdl-image) sdl_image=enabled |
717171bd DB |
81 | ;; |
82 | ||
83 | ..snip.. | |
84 | ||
85 | # Help output feature message | |
1a94933f | 86 | sdl-image SDL Image support for icons |
717171bd DB |
87 | |
88 | ..snip.. | |
89 | ||
77d27b92 | 90 | # Meson invocation |
1a94933f | 91 | -Dsdl_image=$sdl_image |
717171bd | 92 | |
a14f0bf1 | 93 | In meson_options.txt:: |
717171bd | 94 | |
1a94933f PB |
95 | option('sdl', type : 'feature', value : 'auto', |
96 | description: 'SDL Image support for icons') | |
717171bd | 97 | |
a14f0bf1 | 98 | In meson.build:: |
77d27b92 PB |
99 | |
100 | # Detect dependency | |
1a94933f PB |
101 | sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), |
102 | method: 'pkg-config', | |
103 | static: enable_static) | |
77d27b92 | 104 | |
1a94933f PB |
105 | # Create config-host.h (if applicable) |
106 | config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found()) | |
77d27b92 PB |
107 | |
108 | # Summary | |
1a94933f | 109 | summary_info += {'SDL image support': sdl_image.found()} |
717171bd | 110 | |
717171bd DB |
111 | |
112 | ||
113 | Helper functions | |
114 | ---------------- | |
115 | ||
116 | The configure script provides a variety of helper functions to assist | |
117 | developers in checking for system features: | |
118 | ||
a14f0bf1 | 119 | `do_cc $ARGS...` |
717171bd DB |
120 | Attempt to run the system C compiler passing it $ARGS... |
121 | ||
a14f0bf1 | 122 | `do_cxx $ARGS...` |
717171bd DB |
123 | Attempt to run the system C++ compiler passing it $ARGS... |
124 | ||
a14f0bf1 | 125 | `compile_object $CFLAGS` |
717171bd DB |
126 | Attempt to compile a test program with the system C compiler using |
127 | $CFLAGS. The test program must have been previously written to a file | |
738aa606 PB |
128 | called $TMPC. The replacement in Meson is the compiler object `cc`, |
129 | which has methods such as `cc.compiles()`, | |
130 | `cc.check_header()`, `cc.has_function()`. | |
717171bd | 131 | |
a14f0bf1 | 132 | `compile_prog $CFLAGS $LDFLAGS` |
717171bd DB |
133 | Attempt to compile a test program with the system C compiler using |
134 | $CFLAGS and link it with the system linker using $LDFLAGS. The test | |
135 | program must have been previously written to a file called $TMPC. | |
738aa606 | 136 | The replacement in Meson is `cc.find_library()` and `cc.links()`. |
717171bd | 137 | |
a14f0bf1 | 138 | `has $COMMAND` |
717171bd | 139 | Determine if $COMMAND exists in the current environment, either as a |
738aa606 PB |
140 | shell builtin, or executable binary, returning 0 on success. The |
141 | replacement in Meson is `find_program()`. | |
717171bd | 142 | |
a14f0bf1 | 143 | `check_define $NAME` |
717171bd DB |
144 | Determine if the macro $NAME is defined by the system C compiler |
145 | ||
a14f0bf1 | 146 | `check_include $NAME` |
717171bd | 147 | Determine if the include $NAME file is available to the system C |
738aa606 | 148 | compiler. The replacement in Meson is `cc.has_header()`. |
717171bd | 149 | |
a14f0bf1 | 150 | `write_c_skeleton` |
717171bd DB |
151 | Write a minimal C program main() function to the temporary file |
152 | indicated by $TMPC | |
153 | ||
a14f0bf1 | 154 | `feature_not_found $NAME $REMEDY` |
717171bd DB |
155 | Print a message to stderr that the feature $NAME was not available |
156 | on the system, suggesting the user try $REMEDY to address the | |
157 | problem. | |
158 | ||
a14f0bf1 | 159 | `error_exit $MESSAGE $MORE...` |
717171bd DB |
160 | Print $MESSAGE to stderr, followed by $MORE... and then exit from the |
161 | configure script with non-zero status | |
162 | ||
a14f0bf1 | 163 | `query_pkg_config $ARGS...` |
717171bd DB |
164 | Run pkg-config passing it $ARGS. If QEMU is doing a static build, |
165 | then --static will be automatically added to $ARGS | |
166 | ||
167 | ||
77d27b92 PB |
168 | Stage 2: Meson |
169 | ============== | |
717171bd | 170 | |
77d27b92 PB |
171 | The Meson build system is currently used to describe the build |
172 | process for: | |
717171bd | 173 | |
77d27b92 | 174 | 1) executables, which include: |
a14f0bf1 | 175 | |
77d27b92 | 176 | - Tools - qemu-img, qemu-nbd, qga (guest agent), etc |
a14f0bf1 | 177 | |
77d27b92 | 178 | - System emulators - qemu-system-$ARCH |
a14f0bf1 | 179 | |
77d27b92 | 180 | - Userspace emulators - qemu-$ARCH |
a14f0bf1 | 181 | |
ef6a0d6e | 182 | - Unit tests |
717171bd | 183 | |
77d27b92 | 184 | 2) documentation |
717171bd | 185 | |
77d27b92 | 186 | 3) ROMs, which can be either installed as binary blobs or compiled |
717171bd | 187 | |
77d27b92 | 188 | 4) other data files, such as icons or desktop files |
717171bd | 189 | |
77d27b92 PB |
190 | The source code is highly modularized, split across many files to |
191 | facilitate building of all of these components with as little duplicated | |
192 | compilation as possible. The Meson "sourceset" functionality is used | |
193 | to list the files and their dependency on various configuration | |
194 | symbols. | |
717171bd | 195 | |
77d27b92 | 196 | Various subsystems that are common to both tools and emulators have |
a14f0bf1 PB |
197 | their own sourceset, for example `block_ss` for the block device subsystem, |
198 | `chardev_ss` for the character device subsystem, etc. These sourcesets | |
199 | are then turned into static libraries as follows:: | |
717171bd | 200 | |
77d27b92 PB |
201 | libchardev = static_library('chardev', chardev_ss.sources(), |
202 | name_suffix: 'fa', | |
203 | build_by_default: false) | |
717171bd | 204 | |
77d27b92 | 205 | chardev = declare_dependency(link_whole: libchardev) |
717171bd | 206 | |
ef6a0d6e PB |
207 | As of Meson 0.55.1, the special `.fa` suffix should be used for everything |
208 | that is used with `link_whole`, to ensure that the link flags are placed | |
209 | correctly in the command line. | |
717171bd | 210 | |
77d27b92 PB |
211 | Files linked into emulator targets there can be split into two distinct groups |
212 | of files, those which are independent of the QEMU emulation target and | |
213 | those which are dependent on the QEMU emulation target. | |
717171bd DB |
214 | |
215 | In the target-independent set lives various general purpose helper code, | |
216 | such as error handling infrastructure, standard data structures, | |
217 | platform portability wrapper functions, etc. This code can be compiled | |
218 | once only and the .o files linked into all output binaries. | |
a14f0bf1 PB |
219 | Target-independent code lives in the `common_ss`, `softmmu_ss` and |
220 | `user_ss` sourcesets. `common_ss` is linked into all emulators, `softmmu_ss` | |
221 | only in system emulators, `user_ss` only in user-mode emulators. | |
717171bd DB |
222 | |
223 | In the target-dependent set lives CPU emulation, device emulation and | |
224 | much glue code. This sometimes also has to be compiled multiple times, | |
ef6a0d6e PB |
225 | once for each target being built. Target-dependent files are included |
226 | in the `specific_ss` sourceset. | |
717171bd | 227 | |
a14f0bf1 PB |
228 | All binaries link with a static library `libqemuutil.a`, which is then |
229 | linked to all the binaries. `libqemuutil.a` is built from several | |
77d27b92 | 230 | sourcesets; most of them however host generated code, and the only two |
a14f0bf1 | 231 | of general interest are `util_ss` and `stub_ss`. |
77d27b92 PB |
232 | |
233 | The separation between these two is purely for documentation purposes. | |
a14f0bf1 | 234 | `util_ss` contains generic utility files. Even though this code is only |
77d27b92 PB |
235 | linked in some binaries, sometimes it requires hooks only in some of |
236 | these and depend on other functions that are not fully implemented by | |
a14f0bf1 | 237 | all QEMU binaries. `stub_ss` links dummy stubs that will only be linked |
ebedb37c PB |
238 | into the binary if the real implementation is not present. In a way, |
239 | the stubs can be thought of as a portable implementation of the weak | |
240 | symbols concept. | |
241 | ||
77d27b92 PB |
242 | The following files concur in the definition of which files are linked |
243 | into each emulator: | |
ebedb37c | 244 | |
a14f0bf1 PB |
245 | `default-configs/*.mak` |
246 | The files under default-configs/ control what emulated hardware is built | |
247 | into each QEMU system and userspace emulator targets. They merely contain | |
248 | a list of config variable definitions like the machines that should be | |
249 | included. For example, default-configs/aarch64-softmmu.mak has:: | |
717171bd | 250 | |
a14f0bf1 PB |
251 | include arm-softmmu.mak |
252 | CONFIG_XLNX_ZYNQMP_ARM=y | |
253 | CONFIG_XLNX_VERSAL=y | |
717171bd | 254 | |
a14f0bf1 PB |
255 | `*/Kconfig` |
256 | These files are processed together with `default-configs/*.mak` and | |
257 | describe the dependencies between various features, subsystems and | |
258 | device models. They are described in kconfig.rst. | |
717171bd | 259 | |
77d27b92 PB |
260 | These files rarely need changing unless new devices / hardware need to |
261 | be enabled for a particular system/userspace emulation target | |
717171bd | 262 | |
77d27b92 PB |
263 | |
264 | Support scripts | |
265 | --------------- | |
266 | ||
267 | Meson has a special convention for invoking Python scripts: if their | |
a14f0bf1 | 268 | first line is `#! /usr/bin/env python3` and the file is *not* executable, |
77d27b92 PB |
269 | find_program() arranges to invoke the script under the same Python |
270 | interpreter that was used to invoke Meson. This is the most common | |
271 | and preferred way to invoke support scripts from Meson build files, | |
272 | because it automatically uses the value of configure's --python= option. | |
717171bd | 273 | |
a14f0bf1 | 274 | In case the script is not written in Python, use a `#! /usr/bin/env ...` |
77d27b92 | 275 | line and make the script executable. |
717171bd | 276 | |
77d27b92 PB |
277 | Scripts written in Python, where it is desirable to make the script |
278 | executable (for example for test scripts that developers may want to | |
279 | invoke from the command line, such as tests/qapi-schema/test-qapi.py), | |
a14f0bf1 PB |
280 | should be invoked through the `python` variable in meson.build. For |
281 | example:: | |
717171bd | 282 | |
77d27b92 PB |
283 | test('QAPI schema regression tests', python, |
284 | args: files('test-qapi.py'), | |
285 | env: test_env, suite: ['qapi-schema', 'qapi-frontend']) | |
717171bd | 286 | |
77d27b92 PB |
287 | This is needed to obey the --python= option passed to the configure |
288 | script, which may point to something other than the first python3 | |
289 | binary on the path. | |
717171bd DB |
290 | |
291 | ||
77d27b92 PB |
292 | Stage 3: makefiles |
293 | ================== | |
294 | ||
295 | The use of GNU make is required with the QEMU build system. | |
296 | ||
297 | The output of Meson is a build.ninja file, which is used with the Ninja | |
298 | build system. QEMU uses a different approach, where Makefile rules are | |
299 | synthesized from the build.ninja file. The main Makefile includes these | |
300 | rules and wraps them so that e.g. submodules are built before QEMU. | |
301 | The resulting build system is largely non-recursive in nature, in | |
302 | contrast to common practices seen with automake. | |
303 | ||
a14f0bf1 | 304 | Tests are also ran by the Makefile with the traditional `make check` |
ef6a0d6e PB |
305 | phony target, while benchmarks are run with `make bench`. Meson test |
306 | suites such as `unit` can be ran with `make check-unit` too. It is also | |
307 | possible to run tests defined in meson.build with `meson test`. | |
717171bd | 308 | |
77d27b92 PB |
309 | Important files for the build system |
310 | ==================================== | |
311 | ||
717171bd DB |
312 | Statically defined files |
313 | ------------------------ | |
314 | ||
315 | The following key files are statically defined in the source tree, with | |
316 | the rules needed to build QEMU. Their behaviour is influenced by a | |
317 | number of dynamically created files listed later. | |
318 | ||
a14f0bf1 PB |
319 | `Makefile` |
320 | The main entry point used when invoking make to build all the components | |
321 | of QEMU. The default 'all' target will naturally result in the build of | |
322 | every component. Makefile takes care of recursively building submodules | |
323 | directly via a non-recursive set of rules. | |
324 | ||
325 | `*/meson.build` | |
326 | The meson.build file in the root directory is the main entry point for the | |
327 | Meson build system, and it coordinates the configuration and build of all | |
328 | executables. Build rules for various subdirectories are included in | |
329 | other meson.build files spread throughout the QEMU source tree. | |
330 | ||
a14f0bf1 | 331 | `tests/Makefile.include` |
ef6a0d6e PB |
332 | Rules for external test harnesses. These include the TCG tests, |
333 | `qemu-iotests` and the Avocado-based acceptance tests. | |
a14f0bf1 PB |
334 | |
335 | `tests/docker/Makefile.include` | |
336 | Rules for Docker tests. Like tests/Makefile, this file is included | |
337 | directly by the top level Makefile, anything defined in this file will | |
338 | influence the entire build system. | |
339 | ||
340 | `tests/vm/Makefile.include` | |
341 | Rules for VM-based tests. Like tests/Makefile, this file is included | |
342 | directly by the top level Makefile, anything defined in this file will | |
343 | influence the entire build system. | |
717171bd DB |
344 | |
345 | Dynamically created files | |
346 | ------------------------- | |
347 | ||
348 | The following files are generated dynamically by configure in order to | |
349 | control the behaviour of the statically defined makefiles. This avoids | |
350 | the need for QEMU makefiles to go through any pre-processing as seen | |
351 | with autotools, where Makefile.am generates Makefile.in which generates | |
352 | Makefile. | |
353 | ||
77d27b92 | 354 | Built by configure: |
717171bd | 355 | |
a14f0bf1 PB |
356 | `config-host.mak` |
357 | When configure has determined the characteristics of the build host it | |
358 | will write a long list of variables to config-host.mak file. This | |
359 | provides the various install directories, compiler / linker flags and a | |
360 | variety of `CONFIG_*` variables related to optionally enabled features. | |
361 | This is imported by the top level Makefile and meson.build in order to | |
362 | tailor the build output. | |
717171bd | 363 | |
a14f0bf1 PB |
364 | config-host.mak is also used as a dependency checking mechanism. If make |
365 | sees that the modification timestamp on configure is newer than that on | |
366 | config-host.mak, then configure will be re-run. | |
717171bd | 367 | |
a14f0bf1 PB |
368 | The variables defined here are those which are applicable to all QEMU |
369 | build outputs. Variables which are potentially different for each | |
370 | emulator target are defined by the next file... | |
717171bd | 371 | |
a14f0bf1 PB |
372 | `$TARGET-NAME/config-target.mak` |
373 | TARGET-NAME is the name of a system or userspace emulator, for example, | |
374 | x86_64-softmmu denotes the system emulator for the x86_64 architecture. | |
375 | This file contains the variables which need to vary on a per-target | |
376 | basis. For example, it will indicate whether KVM or Xen are enabled for | |
377 | the target and any other potential custom libraries needed for linking | |
378 | the target. | |
717171bd DB |
379 | |
380 | ||
77d27b92 PB |
381 | Built by Meson: |
382 | ||
a14f0bf1 PB |
383 | `${TARGET-NAME}-config-devices.mak` |
384 | TARGET-NAME is again the name of a system or userspace emulator. The | |
385 | config-devices.mak file is automatically generated by make using the | |
386 | scripts/make_device_config.sh program, feeding it the | |
387 | default-configs/$TARGET-NAME file as input. | |
77d27b92 | 388 | |
a14f0bf1 PB |
389 | `config-host.h`, `$TARGET-NAME/config-target.h`, `$TARGET-NAME/config-devices.h` |
390 | These files are used by source code to determine what features | |
391 | are enabled. They are generated from the contents of the corresponding | |
392 | `*.h` files using the scripts/create_config program. This extracts | |
393 | relevant variables and formats them as C preprocessor macros. | |
77d27b92 | 394 | |
a14f0bf1 PB |
395 | `build.ninja` |
396 | The build rules. | |
77d27b92 PB |
397 | |
398 | ||
399 | Built by Makefile: | |
400 | ||
a14f0bf1 PB |
401 | `Makefile.ninja` |
402 | A Makefile conversion of the build rules in build.ninja. The conversion | |
403 | is straightforward and, were it necessary to debug the rules produced | |
404 | by Meson, it should be enough to look at build.ninja. The conversion | |
405 | is performed by scripts/ninjatool.py. | |
77d27b92 | 406 | |
a14f0bf1 PB |
407 | `Makefile.mtest` |
408 | The Makefile definitions that let "make check" run tests defined in | |
409 | meson.build. The rules are produced from Meson's JSON description of | |
410 | tests (obtained with "meson introspect --tests") through the script | |
411 | scripts/mtest2make.py. | |
de1da442 MAL |
412 | |
413 | ||
414 | Useful make targets | |
a14f0bf1 | 415 | ------------------- |
de1da442 | 416 | |
a14f0bf1 | 417 | `help` |
de1da442 MAL |
418 | Print a help message for the most common build targets. |
419 | ||
a14f0bf1 | 420 | `print-VAR` |
de1da442 MAL |
421 | Print the value of the variable VAR. Useful for debugging the build |
422 | system. |