]>
Commit | Line | Data |
---|---|---|
476ff2be SL |
1 | The goal of the libc crate is to have CI running everywhere to have the |
2 | strongest guarantees about the definitions that this library contains, and as a | |
3 | result the CI is pretty complicated and also pretty large! Hopefully this can | |
4 | serve as a guide through the sea of scripts in this directory and elsewhere in | |
5 | this project. | |
6 | ||
7 | # Files | |
8 | ||
9 | First up, let's talk about the files in this directory: | |
10 | ||
11 | * `run-travis.sh` - a shell script run by all Travis builders, this is | |
12 | responsible for setting up the rest of the environment such as installing new | |
13 | packages, downloading Rust target libraries, etc. | |
14 | ||
15 | * `run.sh` - the actual script which runs tests for a particular architecture. | |
16 | Called from the `run-travis.sh` script this will run all tests for the target | |
17 | specified. | |
18 | ||
19 | * `cargo-config` - Cargo configuration of linkers to use copied into place by | |
20 | the `run-travis.sh` script before builds are run. | |
21 | ||
22 | * `dox.sh` - script called from `run-travis.sh` on only the linux 64-bit nightly | |
23 | Travis bots to build documentation for this crate. | |
24 | ||
25 | * `landing-page-*.html` - used by `dox.sh` to generate a landing page for all | |
26 | architectures' documentation. | |
27 | ||
28 | * `run-qemu.sh` - see discussion about QEMU below | |
29 | ||
30 | * `mips`, `rumprun` - instructions to build the docker image for each respective | |
31 | CI target | |
32 | ||
33 | # CI Systems | |
34 | ||
35 | Currently this repository leverages a combination of Travis CI and AppVeyor for | |
36 | running tests. The triples tested are: | |
37 | ||
38 | * AppVeyor | |
39 | * `{i686,x86_64}-pc-windows-{msvc,gnu}` | |
40 | * Travis | |
41 | * `{i686,x86_64,mips,aarch64}-unknown-linux-gnu` | |
abe05a73 | 42 | * `{x86_64,aarch64}-unknown-linux-musl` |
476ff2be SL |
43 | * `arm-unknown-linux-gnueabihf` |
44 | * `arm-linux-androideabi` | |
45 | * `{i686,x86_64}-apple-{darwin,ios}` | |
46 | * `x86_64-rumprun-netbsd` | |
47 | * `x86_64-unknown-freebsd` | |
48 | * `x86_64-unknown-openbsd` | |
49 | ||
50 | The Windows triples are all pretty standard, they just set up their environment | |
51 | then run tests, no need for downloading any extra target libs (we just download | |
52 | the right installer). The Intel Linux/OSX builds are similar in that we just | |
53 | download the right target libs and run tests. Note that the Intel Linux/OSX | |
54 | builds are run on stable/beta/nightly, but are the only ones that do so. | |
55 | ||
56 | The remaining architectures look like: | |
57 | ||
58 | * Android runs in a [docker image][android-docker] with an emulator, the NDK, | |
59 | and the SDK already set up. The entire build happens within the docker image. | |
60 | * The MIPS, ARM, and AArch64 builds all use the QEMU userspace emulator to run | |
61 | the generated binary to actually verify the tests pass. | |
62 | * The MUSL build just has to download a MUSL compiler and target libraries and | |
63 | then otherwise runs tests normally. | |
64 | * iOS builds need an extra linker flag currently, but beyond that they're built | |
65 | as standard as everything else. | |
66 | * The rumprun target builds an entire kernel from the test suite and then runs | |
67 | it inside QEMU using the serial console to test whether it succeeded or | |
68 | failed. | |
69 | * The BSD builds, currently OpenBSD and FreeBSD, use QEMU to boot up a system | |
70 | and compile/run tests. More information on that below. | |
71 | ||
72 | [android-docker]: https://github.com/rust-lang/rust-buildbot/blob/master/slaves/android/Dockerfile | |
73 | ||
74 | ## QEMU | |
75 | ||
76 | Lots of the architectures tested here use QEMU in the tests, so it's worth going | |
77 | over all the crazy capabilities QEMU has and the various flavors in which we use | |
78 | it! | |
79 | ||
80 | First up, QEMU has userspace emulation where it doesn't boot a full kernel, it | |
81 | just runs a binary from another architecture (using the `qemu-<arch>` wrappers). | |
82 | We provide it the runtime path for the dynamically loaded system libraries, | |
83 | however. This strategy is used for all Linux architectures that aren't intel. | |
84 | Note that one downside of this QEMU system is that threads are barely | |
85 | implemented, so we're careful to not spawn many threads. | |
86 | ||
87 | For the rumprun target the only output is a kernel image, so we just use that | |
88 | plus the `rumpbake` command to create a full kernel image which is then run from | |
89 | within QEMU. | |
90 | ||
91 | Finally, the fun part, the BSDs. Quite a few hoops are jumped through to get CI | |
92 | working for these platforms, but the gist of it looks like: | |
93 | ||
94 | * Cross compiling from Linux to any of the BSDs seems to be quite non-standard. | |
95 | We may be able to get it working but it might be difficult at that point to | |
96 | ensure that the libc definitions align with what you'd get on the BSD itself. | |
97 | As a result, we try to do compiles within the BSD distro. | |
98 | * On Travis we can't run a VM-in-a-VM, so we resort to userspace emulation | |
99 | (QEMU). | |
100 | * Unfortunately on Travis we also can't use KVM, so the emulation is super slow. | |
101 | ||
102 | With all that in mind, the way BSD is tested looks like: | |
103 | ||
104 | 1. Download a pre-prepared image for the OS being tested. | |
105 | 2. Generate the tests for the OS being tested. This involves running the `ctest` | |
106 | library over libc to generate a Rust file and a C file which will then be | |
107 | compiled into the final test. | |
108 | 3. Generate a disk image which will later be mounted by the OS being tested. | |
109 | This image is mostly just the libc directory, but some modifications are made | |
110 | to compile the generated files from step 2. | |
111 | 4. The kernel is booted in QEMU, and it is configured to detect the libc-test | |
112 | image being available, run the test script, and then shut down afterwards. | |
113 | 5. Look for whether the tests passed in the serial console output of the kernel. | |
114 | ||
115 | There's some pretty specific instructions for setting up each image (detailed | |
116 | below), but the main gist of this is that we must avoid a vanilla `cargo run` | |
117 | inside of the `libc-test` directory (which is what it's intended for) because | |
118 | that would compile `syntex_syntax`, a large library, with userspace emulation. | |
119 | This invariably times out on Travis, so we can't do that. | |
120 | ||
121 | Once all those hoops are jumped through, however, we can be happy that we're | |
122 | testing almost everything! | |
123 | ||
124 | Below are some details of how to set up the initial OS images which are | |
125 | downloaded. Each image must be enabled have input/output over the serial | |
126 | console, log in automatically at the serial console, detect if a second drive in | |
127 | QEMU is available, and if so mount it, run a script (it'll specifically be | |
128 | `run-qemu.sh` in this folder which is copied into the generated image talked | |
129 | about above), and then shut down. | |
130 | ||
131 | ### QEMU setup - FreeBSD | |
132 | ||
133 | 1. Download CD installer (most minimal is fine) | |
134 | 2. `qemu-img create -f qcow2 foo.qcow2 2G` | |
135 | 3. `qemu -cdrom foo.iso -drive if=virtio,file=foo.qcow2 -net nic,model=virtio -net user` | |
136 | 4. run installer | |
137 | 5. `echo 'console="comconsole"' >> /boot/loader.conf` | |
138 | 6. `echo 'autoboot_delay="0"' >> /boot/loader.conf` | |
139 | 7. look at /etc/ttys, see what getty argument is for ttyu0 | |
140 | 8. edit /etc/gettytab, look for ttyu0 argument, prepend `:al=root` to line | |
141 | beneath | |
142 | ||
143 | (note that the current image has a `freebsd` user, but this isn't really | |
144 | necessary) | |
145 | ||
146 | Once that's done, arrange for this script to run at login: | |
147 | ||
148 | ``` | |
149 | #!/bin/sh | |
150 | ||
151 | sudo kldload ext2fs | |
152 | [ -e /dev/vtbd1 ] || exit 0 | |
153 | sudo mount -t ext2fs /dev/vtbd1 /mnt | |
154 | sh /mnt/run.sh /mnt | |
155 | sudo poweroff | |
156 | ``` | |
157 | ||
158 | Helpful links | |
159 | ||
160 | * https://en.wikibooks.org/wiki/QEMU/Images | |
161 | * https://blog.nekoconeko.nl/blog/2015/06/04/creating-an-openstack-freebsd-image.html | |
162 | * https://www.freebsd.org/doc/handbook/serialconsole-setup.html | |
163 | ||
164 | ||
165 | ### QEMU setup - OpenBSD | |
166 | ||
167 | 1. Download CD installer | |
168 | 2. `qemu-img create -f qcow2 foo.qcow2 2G` | |
169 | 3. `qemu -cdrom foo.iso -drive if=virtio,file=foo.qcow2 -net nic,model=virtio -net user` | |
170 | 4. run installer | |
171 | 5. `echo 'set tty com0' >> /etc/boot.conf` | |
172 | 6. `echo 'boot' >> /etc/boot.conf` | |
173 | 7. Modify /etc/ttys, change the `tty00` at the end from 'unknown off' to | |
174 | 'vt220 on secure' | |
175 | 8. Modify same line in /etc/ttys to have `"/root/foo.sh"` as the shell | |
176 | 9. Add this script to `/root/foo.sh` | |
177 | ||
178 | ``` | |
179 | #!/bin/sh | |
180 | exec 1>/dev/tty00 | |
181 | exec 2>&1 | |
182 | ||
183 | if mount -t ext2fs /dev/sd1c /mnt; then | |
184 | sh /mnt/run.sh /mnt | |
185 | shutdown -ph now | |
186 | fi | |
187 | ||
188 | # limited shell... | |
189 | exec /bin/sh < /dev/tty00 | |
190 | ``` | |
191 | ||
192 | 10. `chmod +x /root/foo.sh` | |
193 | ||
194 | Helpful links: | |
195 | ||
196 | * https://en.wikibooks.org/wiki/QEMU/Images | |
197 | * http://www.openbsd.org/faq/faq7.html#SerCon | |
198 | ||
199 | # Questions? | |
200 | ||
201 | Hopefully that's at least somewhat of an introduction to everything going on | |
202 | here, and feel free to ping @alexcrichton with questions! | |
203 |