]>
Commit | Line | Data |
---|---|---|
89606032 XL |
1 | .. contents:: |
2 | ||
3 | Packaging a crate for Debian | |
4 | ============================ | |
d580884e | 5 | |
c9806f32 | 6 | To get set up, run at Debian unstable (recommended):: |
1057693c | 7 | |
4333698e | 8 | apt update && apt install debcargo |
1057693c XL |
9 | |
10 | Then for each new package: | |
23db3623 | 11 | |
8960a81d XL |
12 | To package a new crate, or to update an existing crate |
13 | ------------------------------------------------------ | |
931eabc0 | 14 | |
8960a81d XL |
15 | :: |
16 | ||
17 | ./new-package.sh <rust-crate-name> # or | |
18 | ./update.sh <rust-crate-name> | |
86e10bdd | 19 | |
6c1b3182 XL |
20 | and follow its instructions. |
21 | ||
8960a81d | 22 | Note that ``new-package.sh`` is just a symlink to ``update.sh``, to help newcomers. |
7b976234 | 23 | |
1ff2ded3 XL |
24 | To package a co-installable older version of a crate |
25 | ---------------------------------------------------- | |
7b976234 | 26 | |
6c1b3182 | 27 | To maintain an old version of a crate alongside the latest one, first make sure |
8960a81d | 28 | the latest version is packaged by doing all of the above, then run:: |
6c1b3182 | 29 | |
8960a81d XL |
30 | ./new-package.sh <rust-crate-name> <old-version> # or |
31 | ./update.sh <rust-crate-name> <old-version> | |
86e10bdd | 32 | |
fed17d69 XL |
33 | and follow its instructions. To save time, you can first copy anything relevant |
34 | from ``src/<rust-crate-name>`` to ``src/<rust-crate-name>-<old-version>``, then | |
35 | adapt it as needed. | |
999f9269 | 36 | |
8960a81d XL |
37 | To prepare a release |
38 | -------------------- | |
39 | ||
40 | :: | |
5cbe2f60 | 41 | |
699de4ac SL |
42 | ./release.sh <rust-crate-name> # or |
43 | ./release.sh <rust-crate-name> <old-version> # as appropriate | |
44 | DISTRO=experimental ./release.sh <rust-crate-name> # to target another distro | |
86e10bdd | 45 | |
fed17d69 XL |
46 | This prepares the necessary Debian files in ``build/``, and creates a git |
47 | branch to manage the packaging until it is accepted in Debian itself. You need | |
48 | to run additional commands after this - more specific instructions are given to | |
49 | you about this, by the script after you run it. | |
6c1b3182 | 50 | |
8960a81d XL |
51 | Holding packages at old versions |
52 | -------------------------------- | |
53 | ||
54 | If you need to keep the latest version in Debian at an older version than is | |
55 | released on crates.io, e.g. to upload an important bugfix without being blocked | |
56 | on having to package all the dependencies of the newest version, you can:: | |
57 | ||
58 | REALVER=<old-version> ./update.sh <rust-crate-name> # then | |
59 | REALVER=<old-version> ./release.sh <rust-crate-name> | |
60 | ||
c6709bfe XL |
61 | Repackaging the existing revision |
62 | --------------------------------- | |
63 | ||
64 | In order to build a package A already in ``debcargo-conf/src`` | |
65 | in the exact version which is present here, do the following:: | |
66 | ||
67 | $ ./repackage.sh A | |
68 | $ cd build | |
69 | $ ./build.sh A | |
70 | ||
71 | If this package is already in the archive and you want to recreate that | |
72 | exactly, you will need to use the exact same version of debcargo that was | |
73 | used previously. This version is mentioned in ``debian/changelog``. | |
74 | ||
d580884e | 75 | |
c6709bfe XL |
76 | Build environment |
77 | ================= | |
78 | ||
79 | To set up a suitable build environment for ``./build.sh``:: | |
80 | ||
745a5a63 | 81 | $ sudo apt-get install devscripts reprepro debootstrap sbuild dh-cargo schroot autopkgtest |
c6709bfe XL |
82 | $ sudo sbuild-createchroot --include=eatmydata,ccache,gnupg,dh-cargo,cargo,lintian,perl-openssl-defaults \ |
83 | --chroot-prefix debcargo-unstable unstable \ | |
84 | /srv/chroot/debcargo-unstable-amd64-sbuild http://deb.debian.org/debian | |
85 | ||
1ff2ded3 XL |
86 | An explanation of this, plus more recipes, can be found on the `sbuild wiki |
87 | page <https://wiki.debian.org/sbuild>`_. | |
88 | ||
1e197466 MC |
89 | If you need to pass additional options to sbuild, like "--arch=i386", then set |
90 | the SBUILD_OPTS environment variable. | |
91 | ||
c6709bfe XL |
92 | Normally, ``./build.sh`` will fail early if not all the build dependencies are |
93 | available in your local apt cache. If you are packaging a large dependency tree | |
94 | however, to avoid many round-trips through NEW it is possible to bypass this | |
95 | check and build all the packages together. Suppose package B depends on package | |
96 | A, then you can run something like:: | |
97 | ||
98 | $ export IGNORE_MISSING_BUILD_DEPS=1 | |
99 | $ ./release.sh A | |
100 | $ ( cd build && ./build.sh A ) | |
101 | # push pending and checkout master | |
102 | $ ./release.sh B | |
103 | $ ( cd build && ./build.sh B librust-A*.deb ) | |
104 | ||
105 | The extra arguments after ``./build.sh B <args>`` is extra deb files to pass to | |
106 | sbuild to use as dependencies. In this case, ``librust-A*.deb`` should have | |
e313fb0f BN |
107 | been built by the previous step. Alternatively, use the environment variable |
108 | ``EXTRA_DEBS``, like so: :: | |
109 | ||
110 | $ EXTRA_DEBS=librust-A*.deb ./build.sh B | |
111 | $ EXTRA_DEBS=librust-A.deb,librust-B.deb ./build.sh C | |
c6709bfe XL |
112 | |
113 | After everything is built successfully, you can ``dput`` all of them and then | |
114 | push all the ``pending-*`` branches as normal. | |
115 | ||
116 | ||
89606032 XL |
117 | Repository structure |
118 | ==================== | |
119 | ||
120 | ``pending-*`` branches are managed by ``./release.sh``, so please don't manage | |
121 | them yourself as you will interfere with the working of that script. The | |
122 | intention is that they should only differ from the master branch by 1 commit, | |
123 | i.e. the ``dch -r`` commit created by ``./release.sh``. | |
124 | ||
125 | If you want to create separate non-master branches, that is fine - just don't | |
126 | call them ``pending-*`` and don't run ``./release.sh`` on those branches. If you | |
127 | want to test your crate, instead run:: | |
128 | ||
129 | cd build && [SOURCEONLY=1] ./build.sh <rust-crate-name> [<old-version>] | |
130 | ||
131 | omitting or not omitting the stuff in [] as needed. | |
132 | ||
133 | Like many other Debian git repositories, we don't follow "feature branch" | |
134 | practises here. We generally don't package just 1 or 2 rust crates at a time, | |
135 | but all of its dependencies and sometimes some reverse-dependencies too. So | |
136 | normally we'll be touching a few dozen packages at once. In this context, it's | |
137 | good to merge often, to avoid conflicts with someone else that might also need | |
138 | to touch those too in the next few days. | |
139 | ||
140 | To match a release (i.e. a ``.deb`` or a ``.dsc`` file) to a commit, find the | |
141 | commit message that actually says "Release package X". This will usually be a | |
142 | merge commit. | |
143 | ||
144 | ||
145 | Expert mode & packaging multiple packages | |
146 | ========================================= | |
147 | ||
148 | You should get used to the single-packaging workflow a bit first, including | |
149 | doing a few `test builds <#build-environment>`_ of your package. Otherwise the | |
150 | instructions below may seem a bit opaque. | |
151 | ||
152 | 1. ``rm -rf build/* && sbuild-update -udr debcargo-unstable-amd64-sbuild`` - | |
153 | clears out your build directory, making the subsequent steps a bit faster. | |
154 | 2. ``./update.sh <CRATE>`` for all your relevant packages | |
155 | 3. Do any manual updates. | |
94d1ee96 | 156 | 4. ``cd build`` then ``IGNORE_MISSING_BUILD_DEPS=1 ./build.sh <CRATE> *.deb`` |
89606032 XL |
157 | for all your relevant packages, in dependency order. |
158 | 5. Deal with any issues that come up. | |
159 | 6. Push your updates to our git. | |
160 | 7. Run ``dev/list-rdeps <CRATE> [<CRATE> ...]`` on all the crates you updated. | |
161 | Any reverse-dependencies that are affected, also need to be updated and you | |
162 | should repeat steps 1-7 (including this step) for them as well, until this | |
163 | step lists no new packages that are affected. | |
164 | 8. ``./release.sh <CRATE>`` for all the packages you updated, running the build | |
165 | again if necessary. It may be possible to do this out of dependency order, | |
166 | assuming you didn't have to make significant changes in step (5). If you | |
167 | did, then this step also has to be done in dependency order. | |
168 | 9. Push your ``pending-*`` branches to our git. | |
169 | ||
170 | I like to have 4 shell windows open for this: | |
171 | ||
172 | 1. To do the manual updates. | |
173 | 2. To explore git, to remember what step you're on and to lookup previous | |
174 | reference material. | |
175 | 3. To explore the build directory, e.g. logs and crate source code. | |
94d1ee96 XL |
176 | 4. To run a build. Try to have one running here at all times, for the next |
177 | package you didn't look at yet, to save time waiting. | |
89606032 XL |
178 | |
179 | There are also various scripts in ``dev/*`` that might help you. They should | |
180 | have a couple lines at the top of the source code describing their | |
181 | functionality and some brief usage instructions. | |
182 | ||
183 | Whew, thanks for all your work! | |
184 | ||
185 | ||
7f68b728 XL |
186 | General packaging tips |
187 | ====================== | |
188 | ||
189 | Dependencies on clippy | |
190 | ---------------------- | |
191 | ||
192 | Patch away dependencies on "clippy" unless it is a "real" dependency. Usually | |
193 | crates only use clippy to lint themselves and it is not a "real" dependency | |
194 | in the sense that they actually import clippy's code for what they do. | |
195 | ||
89606032 XL |
196 | If you want to be sure, ``rg clippy`` and check that all the usages of it are |
197 | inside ``cfg_attr`` declarations. If so, then just get rid of it. | |
7f68b728 | 198 | |
738135f9 XL |
199 | OS-specific crates |
200 | ------------------ | |
201 | ||
202 | See redox-syscall for examples on how to deal with these. | |
203 | ||
204 | If this is unclear, ask on IRC. | |
205 | ||
7f68b728 XL |
206 | Architecture-specific crates |
207 | ---------------------------- | |
01f68998 | 208 | |
738135f9 | 209 | This is a bit harder. Usually there are two options: |
95399141 | 210 | |
738135f9 XL |
211 | 1. The crate should build a dummy/no-op version of itself "out-of-the-box" |
212 | on the architectures it doesn't work on. | |
213 | 2. Dependent crates should depend on it with a platform-specific dependency, | |
214 | see https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#platform-specific-dependencies | |
215 | ||
216 | (1) involves less burden for others, both for dependent crates and for us | |
217 | packagers, since we don't have to override d/rules to ignore test failures on | |
218 | non-working architectures. You should communicate to upstream that this is | |
219 | the preferred approach. | |
220 | ||
221 | In the case of (2), the crate should document exactly what conditional should | |
222 | be used, and keep this documentation up-to-date. This allows us to easily | |
223 | determine if dependent crates are using the correct conditional. You will then | |
224 | have to override d/rules for this crate, see src/simd for an example. | |
225 | ||
226 | You should file a bug upstream if the crate does neither (1) nor document the | |
227 | conditions for (2), e.g. https://github.com/hsivonen/simd/issues/25 | |
228 | ||
229 | (Actually the above applies even for "OS-specific crates" but then (2) is | |
230 | obvious so documentation is less necessary, and dependent crates all do it | |
231 | correctly already.) | |
95399141 | 232 | |
c9063851 AK |
233 | Setting collapse_features in debcargo.conf |
234 | ------------------------------------------ | |
235 | ||
236 | Rust and Debian have a two different levels of abstraction when handling dependencies and the | |
237 | relationship between them. In rust the lowest level is a feature, and in Debian it's the binary | |
238 | package. | |
239 | ||
240 | This means that the following dependency chain is not a problem in rust: | |
241 | ||
242 | - crate A with feature AX depends on crate B with feature BY | |
243 | - crate B with feature BX depends on crate A with feature AY | |
244 | ||
245 | This is a perfectly valid situation in the rust+cargo ecosystem. Notice that | |
246 | there is no dependency cycle on the per-feature level, and this is enforced by | |
247 | cargo; but if collapse_features is used then package A+AX+AY would cyclicly | |
248 | depend on package B+BX+BY. | |
249 | ||
250 | This is reflected in the Debian packages by producing `Provides` lines for all combinations | |
251 | of features, and this can become a quite large section. | |
252 | ||
253 | Setting `collapse_features = true` in debcargo.toml removes this behaviour and is recommended, | |
254 | but it can lead to dependency cycles of debian packages, if that happens those must be | |
255 | broken up by having some or all of the packages set this feature to false. | |
256 | ||
a0c73b1a XL |
257 | Changed orig tarballs |
258 | --------------------- | |
259 | ||
260 | Sometimes the orig.tar generated by debcargo might change e.g. if you are using | |
261 | a newer version of debcargo and one of the dependencies relating to generating | |
262 | the tarball was updated and its behaviour changed - compression settings, | |
263 | tarball archive ordering, etc. This will cause your upload to get REJECTED by | |
264 | the Debian FTP archive for having a different orig.tar. In this case, set | |
1ff2ded3 | 265 | ``REUSE_EXISTING_ORIG_TARBALL=1`` when running ``./release.sh``. |
a0c73b1a | 266 | |
7f68b728 XL |
267 | ITPs |
268 | ---- | |
95399141 | 269 | |
7f68b728 | 270 | Don't file ITPs for library crates, but do file them for binary crates. |
01f68998 | 271 | |
1ff2ded3 XL |
272 | Generally we'll be uploading a dozen crates or so at once. Submitting ITPs for |
273 | these is unnecessary since we're the only ones uploading and there is no chance | |
274 | of conflict. It would only be spam for the bug tracker. Please instead | |
275 | co-ordinate uploads on the ``#debian-rust`` IRC channel. | |
01f68998 | 276 | |
0d1e2247 SL |
277 | Testing |
278 | ------- | |
279 | ||
dc94b1d5 XL |
280 | Debian has two types of tests: |
281 | ||
1ff2ded3 XL |
282 | 1. pre-install tests run in ``debian/rules`` |
283 | 2. post-install tests defined in ``debian/tests/control`` | |
dc94b1d5 XL |
284 | |
285 | For Debian rust packages, in (1) we run the crate's test suite with default | |
286 | features but only if there are no dev-dependencies, and in (2) we run the whole | |
1ff2ded3 XL |
287 | test suite with each feature enabled separately plus ``--no-default-features`` |
288 | and ``--all-features``. | |
dc94b1d5 XL |
289 | |
290 | Sometimes, tests require extra tweaks and settings to work. In this case, you | |
291 | can tweak ``debian/rules`` for (1), and for (2) you will simply have to mark | |
292 | the relevant tests as broken using ``test_is_broken = true``. See the existing | |
293 | crate configs for examples. | |
294 | ||
295 | Other times, the tests are simply broken or can't be run in Debian. In this | |
296 | case you should disable the test in (1) by running ``dh_auto_test -- build`` | |
297 | instead of the default ``dh_auto_test -- test --all``, and for (2) again you | |
298 | should mark the relevant tests as broken. | |
b8798cdc SL |
299 | These tests are going to be marked as flaky in autopkgtest, still executed but |
300 | won't fail the autopkgtest run. | |
dc94b1d5 | 301 | |
ef62445b SL |
302 | Currently, using debcargo, it is not possible to add new dependencies as part |
303 | of an autopkgtest run. See https://salsa.debian.org/rust-team/debcargo/-/merge_requests/24 | |
304 | Instead, just override ``debian/tests/control``. See ``src/cbindgen/`` as | |
305 | example. | |
306 | ||
dc94b1d5 XL |
307 | Please note that ``[packages.lib]\ntest_is_broken = true`` will transitively |
308 | disable tests for all combinations of features. Sometimes this is correct e.g. | |
309 | if the test actually breaks for all features. Sometimes this is *not* correct, | |
310 | e.g. if the test only breaks for ``--no-default-features``. In the latter case | |
311 | you should instead patch the crate to ignore those tests when the relevant | |
1ff2ded3 | 312 | features are absent - e.g. ``src/regex-automata/debian/patches/ignore-std-tests.patch``. |
01f68998 | 313 | |
a1872536 XL |
314 | Binary-crate has "required-features" |
315 | ------------------------------------ | |
316 | ||
317 | See ``src/dotenv`` for an example on dealing with this. | |
318 | ||
319 | Binary-crate has conflicting name | |
320 | --------------------------------- | |
321 | ||
322 | See ``src/fd-find`` for an example on dealing with this. | |
323 | ||
053e604c SL |
324 | Updating the dependencies |
325 | ------------------------- | |
326 | ||
ed1eaac5 XL |
327 | In some cases, libraries/programs are forcing an old version of a library as |
328 | dependencies. In order to limit the number of duplicated libraries in the | |
329 | archive, please try to evaluate if a newer version of the dependencies could be | |
330 | used. | |
331 | ||
1ff2ded3 | 332 | To achieve that, after ``./update.sh``, try:: |
ed1eaac5 XL |
333 | |
334 | $ cd build/<package>/ | |
335 | $ rm -rf .pc # sometimes this is necessary due to minor debcargo bug | |
336 | $ quilt push -a | |
337 | $ quilt new relax-dep.diff | |
338 | $ quilt edit Cargo.toml | |
16650581 | 339 | $ quilt header -e --dep3 |
ed1eaac5 XL |
340 | $ quilt refresh |
341 | $ cargo build # check that it works. if it does, then | |
342 | $ cp -R patches ../../src/<package>/debian | |
343 | ||
344 | Suppose you want to change the dependency from 0.3 to 0.5. If the crate builds | |
345 | with no further source changes, then we would change the required version in | |
346 | ``Cargo.toml`` from ``0.3`` to ``>= 0.3, < 0.6`` or something like that. Then | |
347 | the convention is to put all these changes into a single patch called | |
348 | ``relax-dep-versions.patch``. | |
349 | ||
350 | OTOH, if the cargo build fails, and you can fix it up by editing the source | |
351 | code in a minor way to use the new crate API, then: for each crate that needs | |
352 | to be updated, you should instead name the patch ``update-dep-<crate>.patch`` | |
1ff2ded3 XL |
353 | and add both the ``Cargo.toml`` and the source code changes to it. The change |
354 | to ``Cargo.toml`` would then simply say (e.g.) ``0.5`` since the older versions | |
355 | actually don't work, and not the version range from the previous paragraph. | |
e4d0e3d6 | 356 | |
870cacd0 RK |
357 | If you want to make a crate work with an older dependency version than listed |
358 | in ``Cargo.toml`` (for example 0.3 instead of 0.5), you cannot use a flexible | |
359 | version requirement like ``>= 0.3, < 0.6``. Instead you have to specify only | |
360 | the older version, in this example ``0.3`` (`explanation`_). | |
361 | ||
362 | .. _explanation: https://salsa.debian.org/rust-team/debcargo-conf/merge_requests/86#note_135456 | |
363 | ||
16650581 | 364 | Information on patch headers is available in `dep3`_. |
932e534e | 365 | Use (some of) the headers to explain **why** the patch exists. |
16650581 | 366 | |
367 | .. _dep3: https://dep-team.pages.debian.net/deps/dep3/ | |
368 | ||
e4d0e3d6 XL |
369 | Help, something went wrong! |
370 | --------------------------- | |
371 | ||
372 | Sometimes, the error messages are not the most informative. In this case you | |
373 | can try re-running the command with ``RUST_BACKTRACE=1``. If you are using the | |
374 | ``debcargo`` from Debian's own repositories, you should also install the | |
375 | ``debcargo-dbgsym`` package, otherwise the stack trace will be next to useless. | |
376 | Make sure you have the `debug repository <https://wiki.debian.org/HowToGetABacktrace#Installing_the_debugging_symbols>`_ | |
377 | enabled in your APT sources. | |
65c5dc9d GS |
378 | |
379 | ||
380 | Some ramblings | |
381 | -------------- | |
382 | ||
383 | In ``#debian-rust`` came these two blog posts along with the remark of _good read_ | |
8c68fe9c GS |
384 | * https://blog.hackeriet.no/packaging-a-rust-project-for-debian/ |
385 | * https://blog.hackeriet.no/packaging-rust-part-II/ | |
65c5dc9d | 386 | |
16650581 | 387 | Now are they, those two blog posts, parked here. Waiting for better integration. |
901609d8 GS |
388 | |
389 | ||
390 | Developing Rust code using Debian-packaged crates | |
391 | ================================================= | |
392 | ||
393 | While perhaps not the stated intention, the Rust ecosystem in Debian | |
394 | is actually quite usable for developing Rust code in general. Thanks | |
395 | to `source replacement | |
396 | <https://doc.rust-lang.org/cargo/reference/source-replacement.html>`_, | |
397 | Cargo can be configured to use only local, Debian-provided packages by | |
844ab706 GS |
398 | placing something like the following in ``~/.cargo/config.toml`` (for |
399 | user-wide effect) or in a given project's ``.cargo/config.toml``:: | |
901609d8 GS |
400 | |
401 | [net] | |
402 | offline = true | |
403 | ||
404 | [source] | |
405 | ||
406 | [source.crates-io] | |
407 | replace-with = "debian" | |
408 | ||
409 | [source.debian] | |
1e1d8645 | 410 | directory = "/usr/share/cargo/registry" |
901609d8 GS |
411 | |
412 | In this state, Cargo will only look for crates installed as Debian | |
413 | packages on the local system. |