]>
Commit | Line | Data |
---|---|---|
a14f0bf1 PB |
1 | ================================== |
2 | The QEMU build system architecture | |
3 | ================================== | |
717171bd DB |
4 | |
5 | This document aims to help developers understand the architecture of the | |
6 | QEMU build system. As with projects using GNU autotools, the QEMU build | |
7 | system has two stages, first the developer runs the "configure" script | |
8 | to determine the local build environment characteristics, then they run | |
9 | "make" to build the project. There is about where the similarities with | |
10 | GNU autotools end, so try to forget what you know about them. | |
11 | ||
12 | ||
13 | Stage 1: configure | |
14 | ================== | |
15 | ||
16 | The QEMU configure script is written directly in shell, and should be | |
17 | compatible with any POSIX shell, hence it uses #!/bin/sh. An important | |
18 | implication of this is that it is important to avoid using bash-isms on | |
19 | development platforms where bash is the primary host. | |
20 | ||
21 | In contrast to autoconf scripts, QEMU's configure is expected to be | |
22 | silent while it is checking for features. It will only display output | |
23 | when an error occurs, or to show the final feature enablement summary | |
24 | on completion. | |
25 | ||
77d27b92 PB |
26 | Because QEMU uses the Meson build system under the hood, only VPATH |
27 | builds are supported. There are two general ways to invoke configure & | |
28 | perform a build: | |
29 | ||
a14f0bf1 | 30 | - VPATH, build artifacts outside of QEMU source tree entirely:: |
77d27b92 PB |
31 | |
32 | cd ../ | |
33 | mkdir build | |
34 | cd build | |
35 | ../qemu/configure | |
36 | make | |
37 | ||
a14f0bf1 | 38 | - VPATH, build artifacts in a subdir of QEMU source tree:: |
77d27b92 PB |
39 | |
40 | mkdir build | |
41 | cd build | |
42 | ../configure | |
43 | make | |
44 | ||
45 | For now, checks on the compilation environment are found in configure | |
46 | rather than meson.build, though this is expected to change. The command | |
47 | line is parsed in the configure script and, whenever needed, converted | |
48 | into the appropriate options to Meson. | |
717171bd | 49 | |
77d27b92 PB |
50 | New checks should be added to Meson, which usually comprises the |
51 | following tasks: | |
717171bd | 52 | |
77d27b92 | 53 | - Add a Meson build option to meson_options.txt. |
717171bd DB |
54 | |
55 | - Add support to the command line arg parser to handle any new | |
a14f0bf1 | 56 | `--enable-XXX`/`--disable-XXX` flags required by the feature. |
717171bd DB |
57 | |
58 | - Add information to the help output message to report on the new | |
59 | feature flag. | |
60 | ||
77d27b92 PB |
61 | - Add code to perform the actual feature check. |
62 | ||
a14f0bf1 | 63 | - Add code to include the feature status in `config-host.h` |
717171bd DB |
64 | |
65 | - Add code to print out the feature status in the configure summary | |
66 | upon completion. | |
67 | ||
717171bd | 68 | |
1a94933f | 69 | Taking the probe for SDL2_Image as an example, we have the following pieces |
a14f0bf1 | 70 | in configure:: |
717171bd DB |
71 | |
72 | # Initial variable state | |
1a94933f | 73 | sdl_image=auto |
717171bd DB |
74 | |
75 | ..snip.. | |
76 | ||
77 | # Configure flag processing | |
1a94933f | 78 | --disable-sdl-image) sdl_image=disabled |
717171bd | 79 | ;; |
1a94933f | 80 | --enable-sdl-image) sdl_image=enabled |
717171bd DB |
81 | ;; |
82 | ||
83 | ..snip.. | |
84 | ||
85 | # Help output feature message | |
1a94933f | 86 | sdl-image SDL Image support for icons |
717171bd DB |
87 | |
88 | ..snip.. | |
89 | ||
77d27b92 | 90 | # Meson invocation |
1a94933f | 91 | -Dsdl_image=$sdl_image |
717171bd | 92 | |
a14f0bf1 | 93 | In meson_options.txt:: |
717171bd | 94 | |
1a94933f PB |
95 | option('sdl', type : 'feature', value : 'auto', |
96 | description: 'SDL Image support for icons') | |
717171bd | 97 | |
a14f0bf1 | 98 | In meson.build:: |
77d27b92 PB |
99 | |
100 | # Detect dependency | |
1a94933f PB |
101 | sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), |
102 | method: 'pkg-config', | |
103 | static: enable_static) | |
77d27b92 | 104 | |
1a94933f PB |
105 | # Create config-host.h (if applicable) |
106 | config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found()) | |
77d27b92 PB |
107 | |
108 | # Summary | |
1a94933f | 109 | summary_info += {'SDL image support': sdl_image.found()} |
717171bd | 110 | |
717171bd DB |
111 | |
112 | ||
113 | Helper functions | |
114 | ---------------- | |
115 | ||
116 | The configure script provides a variety of helper functions to assist | |
117 | developers in checking for system features: | |
118 | ||
a14f0bf1 | 119 | `do_cc $ARGS...` |
717171bd DB |
120 | Attempt to run the system C compiler passing it $ARGS... |
121 | ||
a14f0bf1 | 122 | `do_cxx $ARGS...` |
717171bd DB |
123 | Attempt to run the system C++ compiler passing it $ARGS... |
124 | ||
a14f0bf1 | 125 | `compile_object $CFLAGS` |
717171bd DB |
126 | Attempt to compile a test program with the system C compiler using |
127 | $CFLAGS. The test program must have been previously written to a file | |
738aa606 PB |
128 | called $TMPC. The replacement in Meson is the compiler object `cc`, |
129 | which has methods such as `cc.compiles()`, | |
130 | `cc.check_header()`, `cc.has_function()`. | |
717171bd | 131 | |
a14f0bf1 | 132 | `compile_prog $CFLAGS $LDFLAGS` |
717171bd DB |
133 | Attempt to compile a test program with the system C compiler using |
134 | $CFLAGS and link it with the system linker using $LDFLAGS. The test | |
135 | program must have been previously written to a file called $TMPC. | |
738aa606 | 136 | The replacement in Meson is `cc.find_library()` and `cc.links()`. |
717171bd | 137 | |
a14f0bf1 | 138 | `has $COMMAND` |
717171bd | 139 | Determine if $COMMAND exists in the current environment, either as a |
738aa606 PB |
140 | shell builtin, or executable binary, returning 0 on success. The |
141 | replacement in Meson is `find_program()`. | |
717171bd | 142 | |
a14f0bf1 | 143 | `check_define $NAME` |
717171bd DB |
144 | Determine if the macro $NAME is defined by the system C compiler |
145 | ||
a14f0bf1 | 146 | `check_include $NAME` |
717171bd | 147 | Determine if the include $NAME file is available to the system C |
738aa606 | 148 | compiler. The replacement in Meson is `cc.has_header()`. |
717171bd | 149 | |
a14f0bf1 | 150 | `write_c_skeleton` |
717171bd DB |
151 | Write a minimal C program main() function to the temporary file |
152 | indicated by $TMPC | |
153 | ||
a14f0bf1 | 154 | `feature_not_found $NAME $REMEDY` |
717171bd DB |
155 | Print a message to stderr that the feature $NAME was not available |
156 | on the system, suggesting the user try $REMEDY to address the | |
157 | problem. | |
158 | ||
a14f0bf1 | 159 | `error_exit $MESSAGE $MORE...` |
717171bd DB |
160 | Print $MESSAGE to stderr, followed by $MORE... and then exit from the |
161 | configure script with non-zero status | |
162 | ||
a14f0bf1 | 163 | `query_pkg_config $ARGS...` |
717171bd DB |
164 | Run pkg-config passing it $ARGS. If QEMU is doing a static build, |
165 | then --static will be automatically added to $ARGS | |
166 | ||
167 | ||
77d27b92 PB |
168 | Stage 2: Meson |
169 | ============== | |
717171bd | 170 | |
77d27b92 PB |
171 | The Meson build system is currently used to describe the build |
172 | process for: | |
717171bd | 173 | |
77d27b92 | 174 | 1) executables, which include: |
a14f0bf1 | 175 | |
77d27b92 | 176 | - Tools - qemu-img, qemu-nbd, qga (guest agent), etc |
a14f0bf1 | 177 | |
77d27b92 | 178 | - System emulators - qemu-system-$ARCH |
a14f0bf1 | 179 | |
77d27b92 | 180 | - Userspace emulators - qemu-$ARCH |
a14f0bf1 | 181 | |
ef6a0d6e | 182 | - Unit tests |
717171bd | 183 | |
77d27b92 | 184 | 2) documentation |
717171bd | 185 | |
77d27b92 | 186 | 3) ROMs, which can be either installed as binary blobs or compiled |
717171bd | 187 | |
77d27b92 | 188 | 4) other data files, such as icons or desktop files |
717171bd | 189 | |
77d27b92 PB |
190 | The source code is highly modularized, split across many files to |
191 | facilitate building of all of these components with as little duplicated | |
192 | compilation as possible. The Meson "sourceset" functionality is used | |
193 | to list the files and their dependency on various configuration | |
194 | symbols. | |
717171bd | 195 | |
27d551c0 PB |
196 | All executables are built by default, except for some `contrib/` |
197 | binaries that are known to fail to build on some platforms (for example | |
198 | 32-bit or big-endian platforms). Tests are also built by default, | |
199 | though that might change in the future. | |
200 | ||
77d27b92 | 201 | Various subsystems that are common to both tools and emulators have |
a14f0bf1 PB |
202 | their own sourceset, for example `block_ss` for the block device subsystem, |
203 | `chardev_ss` for the character device subsystem, etc. These sourcesets | |
204 | are then turned into static libraries as follows:: | |
717171bd | 205 | |
77d27b92 PB |
206 | libchardev = static_library('chardev', chardev_ss.sources(), |
207 | name_suffix: 'fa', | |
208 | build_by_default: false) | |
717171bd | 209 | |
77d27b92 | 210 | chardev = declare_dependency(link_whole: libchardev) |
717171bd | 211 | |
ef6a0d6e PB |
212 | As of Meson 0.55.1, the special `.fa` suffix should be used for everything |
213 | that is used with `link_whole`, to ensure that the link flags are placed | |
214 | correctly in the command line. | |
717171bd | 215 | |
77d27b92 PB |
216 | Files linked into emulator targets there can be split into two distinct groups |
217 | of files, those which are independent of the QEMU emulation target and | |
218 | those which are dependent on the QEMU emulation target. | |
717171bd DB |
219 | |
220 | In the target-independent set lives various general purpose helper code, | |
221 | such as error handling infrastructure, standard data structures, | |
222 | platform portability wrapper functions, etc. This code can be compiled | |
223 | once only and the .o files linked into all output binaries. | |
a14f0bf1 PB |
224 | Target-independent code lives in the `common_ss`, `softmmu_ss` and |
225 | `user_ss` sourcesets. `common_ss` is linked into all emulators, `softmmu_ss` | |
226 | only in system emulators, `user_ss` only in user-mode emulators. | |
717171bd DB |
227 | |
228 | In the target-dependent set lives CPU emulation, device emulation and | |
229 | much glue code. This sometimes also has to be compiled multiple times, | |
ef6a0d6e PB |
230 | once for each target being built. Target-dependent files are included |
231 | in the `specific_ss` sourceset. | |
717171bd | 232 | |
a14f0bf1 PB |
233 | All binaries link with a static library `libqemuutil.a`, which is then |
234 | linked to all the binaries. `libqemuutil.a` is built from several | |
77d27b92 | 235 | sourcesets; most of them however host generated code, and the only two |
a14f0bf1 | 236 | of general interest are `util_ss` and `stub_ss`. |
77d27b92 PB |
237 | |
238 | The separation between these two is purely for documentation purposes. | |
a14f0bf1 | 239 | `util_ss` contains generic utility files. Even though this code is only |
77d27b92 PB |
240 | linked in some binaries, sometimes it requires hooks only in some of |
241 | these and depend on other functions that are not fully implemented by | |
a14f0bf1 | 242 | all QEMU binaries. `stub_ss` links dummy stubs that will only be linked |
ebedb37c PB |
243 | into the binary if the real implementation is not present. In a way, |
244 | the stubs can be thought of as a portable implementation of the weak | |
245 | symbols concept. | |
246 | ||
77d27b92 PB |
247 | The following files concur in the definition of which files are linked |
248 | into each emulator: | |
ebedb37c | 249 | |
a14f0bf1 PB |
250 | `default-configs/*.mak` |
251 | The files under default-configs/ control what emulated hardware is built | |
252 | into each QEMU system and userspace emulator targets. They merely contain | |
253 | a list of config variable definitions like the machines that should be | |
254 | included. For example, default-configs/aarch64-softmmu.mak has:: | |
717171bd | 255 | |
a14f0bf1 PB |
256 | include arm-softmmu.mak |
257 | CONFIG_XLNX_ZYNQMP_ARM=y | |
258 | CONFIG_XLNX_VERSAL=y | |
717171bd | 259 | |
a14f0bf1 PB |
260 | `*/Kconfig` |
261 | These files are processed together with `default-configs/*.mak` and | |
262 | describe the dependencies between various features, subsystems and | |
263 | device models. They are described in kconfig.rst. | |
717171bd | 264 | |
77d27b92 PB |
265 | These files rarely need changing unless new devices / hardware need to |
266 | be enabled for a particular system/userspace emulation target | |
717171bd | 267 | |
77d27b92 PB |
268 | |
269 | Support scripts | |
270 | --------------- | |
271 | ||
272 | Meson has a special convention for invoking Python scripts: if their | |
a14f0bf1 | 273 | first line is `#! /usr/bin/env python3` and the file is *not* executable, |
77d27b92 PB |
274 | find_program() arranges to invoke the script under the same Python |
275 | interpreter that was used to invoke Meson. This is the most common | |
276 | and preferred way to invoke support scripts from Meson build files, | |
277 | because it automatically uses the value of configure's --python= option. | |
717171bd | 278 | |
a14f0bf1 | 279 | In case the script is not written in Python, use a `#! /usr/bin/env ...` |
77d27b92 | 280 | line and make the script executable. |
717171bd | 281 | |
77d27b92 PB |
282 | Scripts written in Python, where it is desirable to make the script |
283 | executable (for example for test scripts that developers may want to | |
284 | invoke from the command line, such as tests/qapi-schema/test-qapi.py), | |
a14f0bf1 PB |
285 | should be invoked through the `python` variable in meson.build. For |
286 | example:: | |
717171bd | 287 | |
77d27b92 PB |
288 | test('QAPI schema regression tests', python, |
289 | args: files('test-qapi.py'), | |
290 | env: test_env, suite: ['qapi-schema', 'qapi-frontend']) | |
717171bd | 291 | |
77d27b92 PB |
292 | This is needed to obey the --python= option passed to the configure |
293 | script, which may point to something other than the first python3 | |
294 | binary on the path. | |
717171bd DB |
295 | |
296 | ||
77d27b92 PB |
297 | Stage 3: makefiles |
298 | ================== | |
299 | ||
300 | The use of GNU make is required with the QEMU build system. | |
301 | ||
302 | The output of Meson is a build.ninja file, which is used with the Ninja | |
303 | build system. QEMU uses a different approach, where Makefile rules are | |
304 | synthesized from the build.ninja file. The main Makefile includes these | |
305 | rules and wraps them so that e.g. submodules are built before QEMU. | |
306 | The resulting build system is largely non-recursive in nature, in | |
307 | contrast to common practices seen with automake. | |
308 | ||
a14f0bf1 | 309 | Tests are also ran by the Makefile with the traditional `make check` |
ef6a0d6e PB |
310 | phony target, while benchmarks are run with `make bench`. Meson test |
311 | suites such as `unit` can be ran with `make check-unit` too. It is also | |
312 | possible to run tests defined in meson.build with `meson test`. | |
717171bd | 313 | |
77d27b92 PB |
314 | Important files for the build system |
315 | ==================================== | |
316 | ||
717171bd DB |
317 | Statically defined files |
318 | ------------------------ | |
319 | ||
320 | The following key files are statically defined in the source tree, with | |
321 | the rules needed to build QEMU. Their behaviour is influenced by a | |
322 | number of dynamically created files listed later. | |
323 | ||
a14f0bf1 PB |
324 | `Makefile` |
325 | The main entry point used when invoking make to build all the components | |
326 | of QEMU. The default 'all' target will naturally result in the build of | |
327 | every component. Makefile takes care of recursively building submodules | |
328 | directly via a non-recursive set of rules. | |
329 | ||
330 | `*/meson.build` | |
331 | The meson.build file in the root directory is the main entry point for the | |
332 | Meson build system, and it coordinates the configuration and build of all | |
333 | executables. Build rules for various subdirectories are included in | |
334 | other meson.build files spread throughout the QEMU source tree. | |
335 | ||
a14f0bf1 | 336 | `tests/Makefile.include` |
ef6a0d6e PB |
337 | Rules for external test harnesses. These include the TCG tests, |
338 | `qemu-iotests` and the Avocado-based acceptance tests. | |
a14f0bf1 PB |
339 | |
340 | `tests/docker/Makefile.include` | |
341 | Rules for Docker tests. Like tests/Makefile, this file is included | |
342 | directly by the top level Makefile, anything defined in this file will | |
343 | influence the entire build system. | |
344 | ||
345 | `tests/vm/Makefile.include` | |
346 | Rules for VM-based tests. Like tests/Makefile, this file is included | |
347 | directly by the top level Makefile, anything defined in this file will | |
348 | influence the entire build system. | |
717171bd DB |
349 | |
350 | Dynamically created files | |
351 | ------------------------- | |
352 | ||
353 | The following files are generated dynamically by configure in order to | |
354 | control the behaviour of the statically defined makefiles. This avoids | |
355 | the need for QEMU makefiles to go through any pre-processing as seen | |
356 | with autotools, where Makefile.am generates Makefile.in which generates | |
357 | Makefile. | |
358 | ||
77d27b92 | 359 | Built by configure: |
717171bd | 360 | |
a14f0bf1 PB |
361 | `config-host.mak` |
362 | When configure has determined the characteristics of the build host it | |
363 | will write a long list of variables to config-host.mak file. This | |
364 | provides the various install directories, compiler / linker flags and a | |
365 | variety of `CONFIG_*` variables related to optionally enabled features. | |
366 | This is imported by the top level Makefile and meson.build in order to | |
367 | tailor the build output. | |
717171bd | 368 | |
a14f0bf1 PB |
369 | config-host.mak is also used as a dependency checking mechanism. If make |
370 | sees that the modification timestamp on configure is newer than that on | |
371 | config-host.mak, then configure will be re-run. | |
717171bd | 372 | |
a14f0bf1 PB |
373 | The variables defined here are those which are applicable to all QEMU |
374 | build outputs. Variables which are potentially different for each | |
375 | emulator target are defined by the next file... | |
717171bd | 376 | |
a14f0bf1 PB |
377 | `$TARGET-NAME/config-target.mak` |
378 | TARGET-NAME is the name of a system or userspace emulator, for example, | |
379 | x86_64-softmmu denotes the system emulator for the x86_64 architecture. | |
380 | This file contains the variables which need to vary on a per-target | |
381 | basis. For example, it will indicate whether KVM or Xen are enabled for | |
382 | the target and any other potential custom libraries needed for linking | |
383 | the target. | |
717171bd DB |
384 | |
385 | ||
77d27b92 PB |
386 | Built by Meson: |
387 | ||
a14f0bf1 PB |
388 | `${TARGET-NAME}-config-devices.mak` |
389 | TARGET-NAME is again the name of a system or userspace emulator. The | |
390 | config-devices.mak file is automatically generated by make using the | |
391 | scripts/make_device_config.sh program, feeding it the | |
392 | default-configs/$TARGET-NAME file as input. | |
77d27b92 | 393 | |
a14f0bf1 PB |
394 | `config-host.h`, `$TARGET-NAME/config-target.h`, `$TARGET-NAME/config-devices.h` |
395 | These files are used by source code to determine what features | |
396 | are enabled. They are generated from the contents of the corresponding | |
397 | `*.h` files using the scripts/create_config program. This extracts | |
398 | relevant variables and formats them as C preprocessor macros. | |
77d27b92 | 399 | |
a14f0bf1 PB |
400 | `build.ninja` |
401 | The build rules. | |
77d27b92 PB |
402 | |
403 | ||
404 | Built by Makefile: | |
405 | ||
a14f0bf1 | 406 | `Makefile.ninja` |
09e93326 PB |
407 | A Makefile include that bridges to ninja for the actual build. The |
408 | Makefile is mostly a list of targets that Meson included in build.ninja. | |
77d27b92 | 409 | |
a14f0bf1 PB |
410 | `Makefile.mtest` |
411 | The Makefile definitions that let "make check" run tests defined in | |
412 | meson.build. The rules are produced from Meson's JSON description of | |
413 | tests (obtained with "meson introspect --tests") through the script | |
414 | scripts/mtest2make.py. | |
de1da442 MAL |
415 | |
416 | ||
417 | Useful make targets | |
a14f0bf1 | 418 | ------------------- |
de1da442 | 419 | |
a14f0bf1 | 420 | `help` |
de1da442 MAL |
421 | Print a help message for the most common build targets. |
422 | ||
a14f0bf1 | 423 | `print-VAR` |
de1da442 MAL |
424 | Print the value of the variable VAR. Useful for debugging the build |
425 | system. |