]>
Commit | Line | Data |
---|---|---|
89606032 XL |
1 | .. contents:: |
2 | ||
3 | Packaging a crate for Debian | |
4 | ============================ | |
d580884e | 5 | |
c9806f32 | 6 | To get set up, run at Debian unstable (recommended):: |
1057693c | 7 | |
4333698e | 8 | apt update && apt install debcargo |
1057693c XL |
9 | |
10 | Then for each new package: | |
23db3623 | 11 | |
8960a81d XL |
12 | To package a new crate, or to update an existing crate |
13 | ------------------------------------------------------ | |
931eabc0 | 14 | |
8960a81d XL |
15 | :: |
16 | ||
17 | ./new-package.sh <rust-crate-name> # or | |
18 | ./update.sh <rust-crate-name> | |
86e10bdd | 19 | |
6c1b3182 XL |
20 | and follow its instructions. |
21 | ||
8960a81d | 22 | Note that ``new-package.sh`` is just a symlink to ``update.sh``, to help newcomers. |
7b976234 | 23 | |
1ff2ded3 XL |
24 | To package a co-installable older version of a crate |
25 | ---------------------------------------------------- | |
7b976234 | 26 | |
6c1b3182 | 27 | To maintain an old version of a crate alongside the latest one, first make sure |
8960a81d | 28 | the latest version is packaged by doing all of the above, then run:: |
6c1b3182 | 29 | |
8960a81d XL |
30 | ./new-package.sh <rust-crate-name> <old-version> # or |
31 | ./update.sh <rust-crate-name> <old-version> | |
86e10bdd | 32 | |
fed17d69 XL |
33 | and follow its instructions. To save time, you can first copy anything relevant |
34 | from ``src/<rust-crate-name>`` to ``src/<rust-crate-name>-<old-version>``, then | |
35 | adapt it as needed. | |
999f9269 | 36 | |
570a64b5 SL |
37 | You will need excellent reasons to do this. It should be done only for core crates |
38 | used by many other crates or programs for which the upgrade path is complex. | |
39 | For example, if the API significantly changed and requires a lot of work. | |
40 | ||
41 | Instead, please consider: | |
42 | * Downgrading or upgrading the dependency. | |
43 | * If it doesn't exist, open an issue on the upstream issue tracker to | |
44 | encourage them to upgrade. | |
45 | * If possible/relevant, disable a feature if it uses it. | |
46 | * Wait until upstream upgraded. | |
47 | ||
8960a81d XL |
48 | To prepare a release |
49 | -------------------- | |
50 | ||
51 | :: | |
5cbe2f60 | 52 | |
699de4ac SL |
53 | ./release.sh <rust-crate-name> # or |
54 | ./release.sh <rust-crate-name> <old-version> # as appropriate | |
55 | DISTRO=experimental ./release.sh <rust-crate-name> # to target another distro | |
86e10bdd | 56 | |
fed17d69 XL |
57 | This prepares the necessary Debian files in ``build/``, and creates a git |
58 | branch to manage the packaging until it is accepted in Debian itself. You need | |
59 | to run additional commands after this - more specific instructions are given to | |
60 | you about this, by the script after you run it. | |
6c1b3182 | 61 | |
8960a81d XL |
62 | Holding packages at old versions |
63 | -------------------------------- | |
64 | ||
65 | If you need to keep the latest version in Debian at an older version than is | |
66 | released on crates.io, e.g. to upload an important bugfix without being blocked | |
67 | on having to package all the dependencies of the newest version, you can:: | |
68 | ||
69 | REALVER=<old-version> ./update.sh <rust-crate-name> # then | |
70 | REALVER=<old-version> ./release.sh <rust-crate-name> | |
71 | ||
c6709bfe XL |
72 | Repackaging the existing revision |
73 | --------------------------------- | |
74 | ||
75 | In order to build a package A already in ``debcargo-conf/src`` | |
76 | in the exact version which is present here, do the following:: | |
77 | ||
78 | $ ./repackage.sh A | |
79 | $ cd build | |
80 | $ ./build.sh A | |
81 | ||
82 | If this package is already in the archive and you want to recreate that | |
83 | exactly, you will need to use the exact same version of debcargo that was | |
84 | used previously. This version is mentioned in ``debian/changelog``. | |
85 | ||
d580884e | 86 | |
c6709bfe XL |
87 | Build environment |
88 | ================= | |
89 | ||
90 | To set up a suitable build environment for ``./build.sh``:: | |
91 | ||
745a5a63 | 92 | $ sudo apt-get install devscripts reprepro debootstrap sbuild dh-cargo schroot autopkgtest |
c6709bfe XL |
93 | $ sudo sbuild-createchroot --include=eatmydata,ccache,gnupg,dh-cargo,cargo,lintian,perl-openssl-defaults \ |
94 | --chroot-prefix debcargo-unstable unstable \ | |
95 | /srv/chroot/debcargo-unstable-amd64-sbuild http://deb.debian.org/debian | |
96 | ||
1ff2ded3 XL |
97 | An explanation of this, plus more recipes, can be found on the `sbuild wiki |
98 | page <https://wiki.debian.org/sbuild>`_. | |
99 | ||
1e197466 MC |
100 | If you need to pass additional options to sbuild, like "--arch=i386", then set |
101 | the SBUILD_OPTS environment variable. | |
102 | ||
c6709bfe XL |
103 | Normally, ``./build.sh`` will fail early if not all the build dependencies are |
104 | available in your local apt cache. If you are packaging a large dependency tree | |
105 | however, to avoid many round-trips through NEW it is possible to bypass this | |
106 | check and build all the packages together. Suppose package B depends on package | |
107 | A, then you can run something like:: | |
108 | ||
109 | $ export IGNORE_MISSING_BUILD_DEPS=1 | |
110 | $ ./release.sh A | |
111 | $ ( cd build && ./build.sh A ) | |
112 | # push pending and checkout master | |
113 | $ ./release.sh B | |
114 | $ ( cd build && ./build.sh B librust-A*.deb ) | |
115 | ||
116 | The extra arguments after ``./build.sh B <args>`` is extra deb files to pass to | |
117 | sbuild to use as dependencies. In this case, ``librust-A*.deb`` should have | |
3ceff793 BN |
118 | been built by the previous step. Alternatively, use the environment variable |
119 | ``EXTRA_DEBS``, like so: :: | |
120 | ||
121 | $ EXTRA_DEBS=librust-A*.deb ./build.sh B | |
122 | $ EXTRA_DEBS=librust-A.deb,librust-B.deb ./build.sh C | |
c6709bfe XL |
123 | |
124 | After everything is built successfully, you can ``dput`` all of them and then | |
125 | push all the ``pending-*`` branches as normal. | |
126 | ||
127 | ||
89606032 XL |
128 | Repository structure |
129 | ==================== | |
130 | ||
131 | ``pending-*`` branches are managed by ``./release.sh``, so please don't manage | |
132 | them yourself as you will interfere with the working of that script. The | |
133 | intention is that they should only differ from the master branch by 1 commit, | |
134 | i.e. the ``dch -r`` commit created by ``./release.sh``. | |
135 | ||
136 | If you want to create separate non-master branches, that is fine - just don't | |
137 | call them ``pending-*`` and don't run ``./release.sh`` on those branches. If you | |
138 | want to test your crate, instead run:: | |
139 | ||
140 | cd build && [SOURCEONLY=1] ./build.sh <rust-crate-name> [<old-version>] | |
141 | ||
142 | omitting or not omitting the stuff in [] as needed. | |
143 | ||
144 | Like many other Debian git repositories, we don't follow "feature branch" | |
145 | practises here. We generally don't package just 1 or 2 rust crates at a time, | |
146 | but all of its dependencies and sometimes some reverse-dependencies too. So | |
147 | normally we'll be touching a few dozen packages at once. In this context, it's | |
148 | good to merge often, to avoid conflicts with someone else that might also need | |
149 | to touch those too in the next few days. | |
150 | ||
151 | To match a release (i.e. a ``.deb`` or a ``.dsc`` file) to a commit, find the | |
152 | commit message that actually says "Release package X". This will usually be a | |
153 | merge commit. | |
154 | ||
155 | ||
156 | Expert mode & packaging multiple packages | |
157 | ========================================= | |
158 | ||
159 | You should get used to the single-packaging workflow a bit first, including | |
160 | doing a few `test builds <#build-environment>`_ of your package. Otherwise the | |
161 | instructions below may seem a bit opaque. | |
162 | ||
163 | 1. ``rm -rf build/* && sbuild-update -udr debcargo-unstable-amd64-sbuild`` - | |
164 | clears out your build directory, making the subsequent steps a bit faster. | |
165 | 2. ``./update.sh <CRATE>`` for all your relevant packages | |
166 | 3. Do any manual updates. | |
94d1ee96 | 167 | 4. ``cd build`` then ``IGNORE_MISSING_BUILD_DEPS=1 ./build.sh <CRATE> *.deb`` |
89606032 XL |
168 | for all your relevant packages, in dependency order. |
169 | 5. Deal with any issues that come up. | |
170 | 6. Push your updates to our git. | |
171 | 7. Run ``dev/list-rdeps <CRATE> [<CRATE> ...]`` on all the crates you updated. | |
172 | Any reverse-dependencies that are affected, also need to be updated and you | |
173 | should repeat steps 1-7 (including this step) for them as well, until this | |
174 | step lists no new packages that are affected. | |
175 | 8. ``./release.sh <CRATE>`` for all the packages you updated, running the build | |
176 | again if necessary. It may be possible to do this out of dependency order, | |
177 | assuming you didn't have to make significant changes in step (5). If you | |
178 | did, then this step also has to be done in dependency order. | |
179 | 9. Push your ``pending-*`` branches to our git. | |
180 | ||
181 | I like to have 4 shell windows open for this: | |
182 | ||
183 | 1. To do the manual updates. | |
184 | 2. To explore git, to remember what step you're on and to lookup previous | |
185 | reference material. | |
186 | 3. To explore the build directory, e.g. logs and crate source code. | |
94d1ee96 XL |
187 | 4. To run a build. Try to have one running here at all times, for the next |
188 | package you didn't look at yet, to save time waiting. | |
89606032 XL |
189 | |
190 | There are also various scripts in ``dev/*`` that might help you. They should | |
191 | have a couple lines at the top of the source code describing their | |
192 | functionality and some brief usage instructions. | |
193 | ||
194 | Whew, thanks for all your work! | |
195 | ||
196 | ||
7f68b728 XL |
197 | General packaging tips |
198 | ====================== | |
199 | ||
200 | Dependencies on clippy | |
201 | ---------------------- | |
202 | ||
203 | Patch away dependencies on "clippy" unless it is a "real" dependency. Usually | |
204 | crates only use clippy to lint themselves and it is not a "real" dependency | |
205 | in the sense that they actually import clippy's code for what they do. | |
206 | ||
89606032 XL |
207 | If you want to be sure, ``rg clippy`` and check that all the usages of it are |
208 | inside ``cfg_attr`` declarations. If so, then just get rid of it. | |
7f68b728 | 209 | |
738135f9 XL |
210 | OS-specific crates |
211 | ------------------ | |
212 | ||
213 | See redox-syscall for examples on how to deal with these. | |
214 | ||
215 | If this is unclear, ask on IRC. | |
216 | ||
7f68b728 XL |
217 | Architecture-specific crates |
218 | ---------------------------- | |
01f68998 | 219 | |
738135f9 | 220 | This is a bit harder. Usually there are two options: |
95399141 | 221 | |
738135f9 XL |
222 | 1. The crate should build a dummy/no-op version of itself "out-of-the-box" |
223 | on the architectures it doesn't work on. | |
224 | 2. Dependent crates should depend on it with a platform-specific dependency, | |
225 | see https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#platform-specific-dependencies | |
226 | ||
227 | (1) involves less burden for others, both for dependent crates and for us | |
228 | packagers, since we don't have to override d/rules to ignore test failures on | |
229 | non-working architectures. You should communicate to upstream that this is | |
230 | the preferred approach. | |
231 | ||
232 | In the case of (2), the crate should document exactly what conditional should | |
233 | be used, and keep this documentation up-to-date. This allows us to easily | |
234 | determine if dependent crates are using the correct conditional. You will then | |
235 | have to override d/rules for this crate, see src/simd for an example. | |
236 | ||
237 | You should file a bug upstream if the crate does neither (1) nor document the | |
238 | conditions for (2), e.g. https://github.com/hsivonen/simd/issues/25 | |
239 | ||
240 | (Actually the above applies even for "OS-specific crates" but then (2) is | |
241 | obvious so documentation is less necessary, and dependent crates all do it | |
242 | correctly already.) | |
95399141 | 243 | |
c9063851 AK |
244 | Setting collapse_features in debcargo.conf |
245 | ------------------------------------------ | |
246 | ||
247 | Rust and Debian have a two different levels of abstraction when handling dependencies and the | |
248 | relationship between them. In rust the lowest level is a feature, and in Debian it's the binary | |
249 | package. | |
250 | ||
251 | This means that the following dependency chain is not a problem in rust: | |
252 | ||
253 | - crate A with feature AX depends on crate B with feature BY | |
254 | - crate B with feature BX depends on crate A with feature AY | |
255 | ||
256 | This is a perfectly valid situation in the rust+cargo ecosystem. Notice that | |
257 | there is no dependency cycle on the per-feature level, and this is enforced by | |
258 | cargo; but if collapse_features is used then package A+AX+AY would cyclicly | |
259 | depend on package B+BX+BY. | |
260 | ||
261 | This is reflected in the Debian packages by producing `Provides` lines for all combinations | |
262 | of features, and this can become a quite large section. | |
263 | ||
264 | Setting `collapse_features = true` in debcargo.toml removes this behaviour and is recommended, | |
265 | but it can lead to dependency cycles of debian packages, if that happens those must be | |
266 | broken up by having some or all of the packages set this feature to false. | |
267 | ||
a0c73b1a XL |
268 | Changed orig tarballs |
269 | --------------------- | |
270 | ||
271 | Sometimes the orig.tar generated by debcargo might change e.g. if you are using | |
272 | a newer version of debcargo and one of the dependencies relating to generating | |
273 | the tarball was updated and its behaviour changed - compression settings, | |
274 | tarball archive ordering, etc. This will cause your upload to get REJECTED by | |
275 | the Debian FTP archive for having a different orig.tar. In this case, set | |
1ff2ded3 | 276 | ``REUSE_EXISTING_ORIG_TARBALL=1`` when running ``./release.sh``. |
a0c73b1a | 277 | |
7f68b728 XL |
278 | ITPs |
279 | ---- | |
95399141 | 280 | |
7f68b728 | 281 | Don't file ITPs for library crates, but do file them for binary crates. |
01f68998 | 282 | |
1ff2ded3 XL |
283 | Generally we'll be uploading a dozen crates or so at once. Submitting ITPs for |
284 | these is unnecessary since we're the only ones uploading and there is no chance | |
285 | of conflict. It would only be spam for the bug tracker. Please instead | |
c1a24a1f | 286 | coordinate uploads on the ``#debian-rust`` IRC channel. |
01f68998 | 287 | |
0d1e2247 SL |
288 | Testing |
289 | ------- | |
290 | ||
dc94b1d5 XL |
291 | Debian has two types of tests: |
292 | ||
1ff2ded3 XL |
293 | 1. pre-install tests run in ``debian/rules`` |
294 | 2. post-install tests defined in ``debian/tests/control`` | |
dc94b1d5 XL |
295 | |
296 | For Debian rust packages, in (1) we run the crate's test suite with default | |
297 | features but only if there are no dev-dependencies, and in (2) we run the whole | |
1ff2ded3 XL |
298 | test suite with each feature enabled separately plus ``--no-default-features`` |
299 | and ``--all-features``. | |
dc94b1d5 XL |
300 | |
301 | Sometimes, tests require extra tweaks and settings to work. In this case, you | |
302 | can tweak ``debian/rules`` for (1), and for (2) you will simply have to mark | |
303 | the relevant tests as broken using ``test_is_broken = true``. See the existing | |
304 | crate configs for examples. | |
305 | ||
306 | Other times, the tests are simply broken or can't be run in Debian. In this | |
307 | case you should disable the test in (1) by running ``dh_auto_test -- build`` | |
308 | instead of the default ``dh_auto_test -- test --all``, and for (2) again you | |
309 | should mark the relevant tests as broken. | |
b8798cdc SL |
310 | These tests are going to be marked as flaky in autopkgtest, still executed but |
311 | won't fail the autopkgtest run. | |
dc94b1d5 | 312 | |
ef62445b SL |
313 | Currently, using debcargo, it is not possible to add new dependencies as part |
314 | of an autopkgtest run. See https://salsa.debian.org/rust-team/debcargo/-/merge_requests/24 | |
315 | Instead, just override ``debian/tests/control``. See ``src/cbindgen/`` as | |
316 | example. | |
317 | ||
dc94b1d5 XL |
318 | Please note that ``[packages.lib]\ntest_is_broken = true`` will transitively |
319 | disable tests for all combinations of features. Sometimes this is correct e.g. | |
320 | if the test actually breaks for all features. Sometimes this is *not* correct, | |
321 | e.g. if the test only breaks for ``--no-default-features``. In the latter case | |
322 | you should instead patch the crate to ignore those tests when the relevant | |
1ff2ded3 | 323 | features are absent - e.g. ``src/regex-automata/debian/patches/ignore-std-tests.patch``. |
01f68998 | 324 | |
a1872536 XL |
325 | Binary-crate has "required-features" |
326 | ------------------------------------ | |
327 | ||
328 | See ``src/dotenv`` for an example on dealing with this. | |
329 | ||
330 | Binary-crate has conflicting name | |
331 | --------------------------------- | |
332 | ||
333 | See ``src/fd-find`` for an example on dealing with this. | |
334 | ||
053e604c SL |
335 | Updating the dependencies |
336 | ------------------------- | |
337 | ||
ed1eaac5 XL |
338 | In some cases, libraries/programs are forcing an old version of a library as |
339 | dependencies. In order to limit the number of duplicated libraries in the | |
340 | archive, please try to evaluate if a newer version of the dependencies could be | |
341 | used. | |
342 | ||
1ff2ded3 | 343 | To achieve that, after ``./update.sh``, try:: |
ed1eaac5 XL |
344 | |
345 | $ cd build/<package>/ | |
346 | $ rm -rf .pc # sometimes this is necessary due to minor debcargo bug | |
347 | $ quilt push -a | |
348 | $ quilt new relax-dep.diff | |
349 | $ quilt edit Cargo.toml | |
16650581 | 350 | $ quilt header -e --dep3 |
ed1eaac5 XL |
351 | $ quilt refresh |
352 | $ cargo build # check that it works. if it does, then | |
353 | $ cp -R patches ../../src/<package>/debian | |
354 | ||
355 | Suppose you want to change the dependency from 0.3 to 0.5. If the crate builds | |
356 | with no further source changes, then we would change the required version in | |
357 | ``Cargo.toml`` from ``0.3`` to ``>= 0.3, < 0.6`` or something like that. Then | |
358 | the convention is to put all these changes into a single patch called | |
359 | ``relax-dep-versions.patch``. | |
360 | ||
361 | OTOH, if the cargo build fails, and you can fix it up by editing the source | |
362 | code in a minor way to use the new crate API, then: for each crate that needs | |
363 | to be updated, you should instead name the patch ``update-dep-<crate>.patch`` | |
1ff2ded3 XL |
364 | and add both the ``Cargo.toml`` and the source code changes to it. The change |
365 | to ``Cargo.toml`` would then simply say (e.g.) ``0.5`` since the older versions | |
366 | actually don't work, and not the version range from the previous paragraph. | |
e4d0e3d6 | 367 | |
870cacd0 RK |
368 | If you want to make a crate work with an older dependency version than listed |
369 | in ``Cargo.toml`` (for example 0.3 instead of 0.5), you cannot use a flexible | |
370 | version requirement like ``>= 0.3, < 0.6``. Instead you have to specify only | |
371 | the older version, in this example ``0.3`` (`explanation`_). | |
372 | ||
373 | .. _explanation: https://salsa.debian.org/rust-team/debcargo-conf/merge_requests/86#note_135456 | |
374 | ||
16650581 | 375 | Information on patch headers is available in `dep3`_. |
932e534e | 376 | Use (some of) the headers to explain **why** the patch exists. |
16650581 | 377 | |
378 | .. _dep3: https://dep-team.pages.debian.net/deps/dep3/ | |
379 | ||
e4d0e3d6 XL |
380 | Help, something went wrong! |
381 | --------------------------- | |
382 | ||
383 | Sometimes, the error messages are not the most informative. In this case you | |
384 | can try re-running the command with ``RUST_BACKTRACE=1``. If you are using the | |
385 | ``debcargo`` from Debian's own repositories, you should also install the | |
386 | ``debcargo-dbgsym`` package, otherwise the stack trace will be next to useless. | |
387 | Make sure you have the `debug repository <https://wiki.debian.org/HowToGetABacktrace#Installing_the_debugging_symbols>`_ | |
388 | enabled in your APT sources. | |
65c5dc9d GS |
389 | |
390 | ||
391 | Some ramblings | |
392 | -------------- | |
393 | ||
394 | In ``#debian-rust`` came these two blog posts along with the remark of _good read_ | |
8c68fe9c GS |
395 | * https://blog.hackeriet.no/packaging-a-rust-project-for-debian/ |
396 | * https://blog.hackeriet.no/packaging-rust-part-II/ | |
65c5dc9d | 397 | |
16650581 | 398 | Now are they, those two blog posts, parked here. Waiting for better integration. |
901609d8 GS |
399 | |
400 | ||
401 | Developing Rust code using Debian-packaged crates | |
402 | ================================================= | |
403 | ||
404 | While perhaps not the stated intention, the Rust ecosystem in Debian | |
405 | is actually quite usable for developing Rust code in general. Thanks | |
406 | to `source replacement | |
407 | <https://doc.rust-lang.org/cargo/reference/source-replacement.html>`_, | |
408 | Cargo can be configured to use only local, Debian-provided packages by | |
844ab706 GS |
409 | placing something like the following in ``~/.cargo/config.toml`` (for |
410 | user-wide effect) or in a given project's ``.cargo/config.toml``:: | |
901609d8 GS |
411 | |
412 | [net] | |
413 | offline = true | |
414 | ||
415 | [source] | |
416 | ||
417 | [source.crates-io] | |
418 | replace-with = "debian" | |
419 | ||
420 | [source.debian] | |
1e1d8645 | 421 | directory = "/usr/share/cargo/registry" |
901609d8 GS |
422 | |
423 | In this state, Cargo will only look for crates installed as Debian | |
424 | packages on the local system. |