]>
Commit | Line | Data |
---|---|---|
89606032 XL |
1 | .. contents:: |
2 | ||
3 | Packaging a crate for Debian | |
4 | ============================ | |
d580884e | 5 | |
c9806f32 | 6 | To get set up, run at Debian unstable (recommended):: |
1057693c | 7 | |
4333698e | 8 | apt update && apt install debcargo |
1057693c XL |
9 | |
10 | Then for each new package: | |
23db3623 | 11 | |
8960a81d XL |
12 | To package a new crate, or to update an existing crate |
13 | ------------------------------------------------------ | |
931eabc0 | 14 | |
8960a81d XL |
15 | :: |
16 | ||
17 | ./new-package.sh <rust-crate-name> # or | |
18 | ./update.sh <rust-crate-name> | |
86e10bdd | 19 | |
6c1b3182 XL |
20 | and follow its instructions. |
21 | ||
8960a81d | 22 | Note that ``new-package.sh`` is just a symlink to ``update.sh``, to help newcomers. |
7b976234 | 23 | |
1ff2ded3 XL |
24 | To package a co-installable older version of a crate |
25 | ---------------------------------------------------- | |
7b976234 | 26 | |
6c1b3182 | 27 | To maintain an old version of a crate alongside the latest one, first make sure |
8960a81d | 28 | the latest version is packaged by doing all of the above, then run:: |
6c1b3182 | 29 | |
8960a81d XL |
30 | ./new-package.sh <rust-crate-name> <old-version> # or |
31 | ./update.sh <rust-crate-name> <old-version> | |
86e10bdd | 32 | |
fed17d69 XL |
33 | and follow its instructions. To save time, you can first copy anything relevant |
34 | from ``src/<rust-crate-name>`` to ``src/<rust-crate-name>-<old-version>``, then | |
35 | adapt it as needed. | |
999f9269 | 36 | |
8960a81d XL |
37 | To prepare a release |
38 | -------------------- | |
39 | ||
40 | :: | |
5cbe2f60 | 41 | |
699de4ac SL |
42 | ./release.sh <rust-crate-name> # or |
43 | ./release.sh <rust-crate-name> <old-version> # as appropriate | |
44 | DISTRO=experimental ./release.sh <rust-crate-name> # to target another distro | |
86e10bdd | 45 | |
fed17d69 XL |
46 | This prepares the necessary Debian files in ``build/``, and creates a git |
47 | branch to manage the packaging until it is accepted in Debian itself. You need | |
48 | to run additional commands after this - more specific instructions are given to | |
49 | you about this, by the script after you run it. | |
6c1b3182 | 50 | |
8960a81d XL |
51 | Holding packages at old versions |
52 | -------------------------------- | |
53 | ||
54 | If you need to keep the latest version in Debian at an older version than is | |
55 | released on crates.io, e.g. to upload an important bugfix without being blocked | |
56 | on having to package all the dependencies of the newest version, you can:: | |
57 | ||
58 | REALVER=<old-version> ./update.sh <rust-crate-name> # then | |
59 | REALVER=<old-version> ./release.sh <rust-crate-name> | |
60 | ||
c6709bfe XL |
61 | Repackaging the existing revision |
62 | --------------------------------- | |
63 | ||
64 | In order to build a package A already in ``debcargo-conf/src`` | |
65 | in the exact version which is present here, do the following:: | |
66 | ||
67 | $ ./repackage.sh A | |
68 | $ cd build | |
69 | $ ./build.sh A | |
70 | ||
71 | If this package is already in the archive and you want to recreate that | |
72 | exactly, you will need to use the exact same version of debcargo that was | |
73 | used previously. This version is mentioned in ``debian/changelog``. | |
74 | ||
d580884e | 75 | |
c6709bfe XL |
76 | Build environment |
77 | ================= | |
78 | ||
79 | To set up a suitable build environment for ``./build.sh``:: | |
80 | ||
745a5a63 | 81 | $ sudo apt-get install devscripts reprepro debootstrap sbuild dh-cargo schroot autopkgtest |
c6709bfe XL |
82 | $ sudo sbuild-createchroot --include=eatmydata,ccache,gnupg,dh-cargo,cargo,lintian,perl-openssl-defaults \ |
83 | --chroot-prefix debcargo-unstable unstable \ | |
84 | /srv/chroot/debcargo-unstable-amd64-sbuild http://deb.debian.org/debian | |
85 | ||
1ff2ded3 XL |
86 | An explanation of this, plus more recipes, can be found on the `sbuild wiki |
87 | page <https://wiki.debian.org/sbuild>`_. | |
88 | ||
1e197466 MC |
89 | If you need to pass additional options to sbuild, like "--arch=i386", then set |
90 | the SBUILD_OPTS environment variable. | |
91 | ||
c6709bfe XL |
92 | Normally, ``./build.sh`` will fail early if not all the build dependencies are |
93 | available in your local apt cache. If you are packaging a large dependency tree | |
94 | however, to avoid many round-trips through NEW it is possible to bypass this | |
95 | check and build all the packages together. Suppose package B depends on package | |
96 | A, then you can run something like:: | |
97 | ||
98 | $ export IGNORE_MISSING_BUILD_DEPS=1 | |
99 | $ ./release.sh A | |
100 | $ ( cd build && ./build.sh A ) | |
101 | # push pending and checkout master | |
102 | $ ./release.sh B | |
103 | $ ( cd build && ./build.sh B librust-A*.deb ) | |
104 | ||
105 | The extra arguments after ``./build.sh B <args>`` is extra deb files to pass to | |
106 | sbuild to use as dependencies. In this case, ``librust-A*.deb`` should have | |
107 | been built by the previous step. | |
108 | ||
109 | After everything is built successfully, you can ``dput`` all of them and then | |
110 | push all the ``pending-*`` branches as normal. | |
111 | ||
112 | ||
89606032 XL |
113 | Repository structure |
114 | ==================== | |
115 | ||
116 | ``pending-*`` branches are managed by ``./release.sh``, so please don't manage | |
117 | them yourself as you will interfere with the working of that script. The | |
118 | intention is that they should only differ from the master branch by 1 commit, | |
119 | i.e. the ``dch -r`` commit created by ``./release.sh``. | |
120 | ||
121 | If you want to create separate non-master branches, that is fine - just don't | |
122 | call them ``pending-*`` and don't run ``./release.sh`` on those branches. If you | |
123 | want to test your crate, instead run:: | |
124 | ||
125 | cd build && [SOURCEONLY=1] ./build.sh <rust-crate-name> [<old-version>] | |
126 | ||
127 | omitting or not omitting the stuff in [] as needed. | |
128 | ||
129 | Like many other Debian git repositories, we don't follow "feature branch" | |
130 | practises here. We generally don't package just 1 or 2 rust crates at a time, | |
131 | but all of its dependencies and sometimes some reverse-dependencies too. So | |
132 | normally we'll be touching a few dozen packages at once. In this context, it's | |
133 | good to merge often, to avoid conflicts with someone else that might also need | |
134 | to touch those too in the next few days. | |
135 | ||
136 | To match a release (i.e. a ``.deb`` or a ``.dsc`` file) to a commit, find the | |
137 | commit message that actually says "Release package X". This will usually be a | |
138 | merge commit. | |
139 | ||
140 | ||
141 | Expert mode & packaging multiple packages | |
142 | ========================================= | |
143 | ||
144 | You should get used to the single-packaging workflow a bit first, including | |
145 | doing a few `test builds <#build-environment>`_ of your package. Otherwise the | |
146 | instructions below may seem a bit opaque. | |
147 | ||
148 | 1. ``rm -rf build/* && sbuild-update -udr debcargo-unstable-amd64-sbuild`` - | |
149 | clears out your build directory, making the subsequent steps a bit faster. | |
150 | 2. ``./update.sh <CRATE>`` for all your relevant packages | |
151 | 3. Do any manual updates. | |
94d1ee96 | 152 | 4. ``cd build`` then ``IGNORE_MISSING_BUILD_DEPS=1 ./build.sh <CRATE> *.deb`` |
89606032 XL |
153 | for all your relevant packages, in dependency order. |
154 | 5. Deal with any issues that come up. | |
155 | 6. Push your updates to our git. | |
156 | 7. Run ``dev/list-rdeps <CRATE> [<CRATE> ...]`` on all the crates you updated. | |
157 | Any reverse-dependencies that are affected, also need to be updated and you | |
158 | should repeat steps 1-7 (including this step) for them as well, until this | |
159 | step lists no new packages that are affected. | |
160 | 8. ``./release.sh <CRATE>`` for all the packages you updated, running the build | |
161 | again if necessary. It may be possible to do this out of dependency order, | |
162 | assuming you didn't have to make significant changes in step (5). If you | |
163 | did, then this step also has to be done in dependency order. | |
164 | 9. Push your ``pending-*`` branches to our git. | |
165 | ||
166 | I like to have 4 shell windows open for this: | |
167 | ||
168 | 1. To do the manual updates. | |
169 | 2. To explore git, to remember what step you're on and to lookup previous | |
170 | reference material. | |
171 | 3. To explore the build directory, e.g. logs and crate source code. | |
94d1ee96 XL |
172 | 4. To run a build. Try to have one running here at all times, for the next |
173 | package you didn't look at yet, to save time waiting. | |
89606032 XL |
174 | |
175 | There are also various scripts in ``dev/*`` that might help you. They should | |
176 | have a couple lines at the top of the source code describing their | |
177 | functionality and some brief usage instructions. | |
178 | ||
179 | Whew, thanks for all your work! | |
180 | ||
181 | ||
7f68b728 XL |
182 | General packaging tips |
183 | ====================== | |
184 | ||
185 | Dependencies on clippy | |
186 | ---------------------- | |
187 | ||
188 | Patch away dependencies on "clippy" unless it is a "real" dependency. Usually | |
189 | crates only use clippy to lint themselves and it is not a "real" dependency | |
190 | in the sense that they actually import clippy's code for what they do. | |
191 | ||
89606032 XL |
192 | If you want to be sure, ``rg clippy`` and check that all the usages of it are |
193 | inside ``cfg_attr`` declarations. If so, then just get rid of it. | |
7f68b728 | 194 | |
738135f9 XL |
195 | OS-specific crates |
196 | ------------------ | |
197 | ||
198 | See redox-syscall for examples on how to deal with these. | |
199 | ||
200 | If this is unclear, ask on IRC. | |
201 | ||
7f68b728 XL |
202 | Architecture-specific crates |
203 | ---------------------------- | |
01f68998 | 204 | |
738135f9 | 205 | This is a bit harder. Usually there are two options: |
95399141 | 206 | |
738135f9 XL |
207 | 1. The crate should build a dummy/no-op version of itself "out-of-the-box" |
208 | on the architectures it doesn't work on. | |
209 | 2. Dependent crates should depend on it with a platform-specific dependency, | |
210 | see https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#platform-specific-dependencies | |
211 | ||
212 | (1) involves less burden for others, both for dependent crates and for us | |
213 | packagers, since we don't have to override d/rules to ignore test failures on | |
214 | non-working architectures. You should communicate to upstream that this is | |
215 | the preferred approach. | |
216 | ||
217 | In the case of (2), the crate should document exactly what conditional should | |
218 | be used, and keep this documentation up-to-date. This allows us to easily | |
219 | determine if dependent crates are using the correct conditional. You will then | |
220 | have to override d/rules for this crate, see src/simd for an example. | |
221 | ||
222 | You should file a bug upstream if the crate does neither (1) nor document the | |
223 | conditions for (2), e.g. https://github.com/hsivonen/simd/issues/25 | |
224 | ||
225 | (Actually the above applies even for "OS-specific crates" but then (2) is | |
226 | obvious so documentation is less necessary, and dependent crates all do it | |
227 | correctly already.) | |
95399141 | 228 | |
a0c73b1a XL |
229 | Changed orig tarballs |
230 | --------------------- | |
231 | ||
232 | Sometimes the orig.tar generated by debcargo might change e.g. if you are using | |
233 | a newer version of debcargo and one of the dependencies relating to generating | |
234 | the tarball was updated and its behaviour changed - compression settings, | |
235 | tarball archive ordering, etc. This will cause your upload to get REJECTED by | |
236 | the Debian FTP archive for having a different orig.tar. In this case, set | |
1ff2ded3 | 237 | ``REUSE_EXISTING_ORIG_TARBALL=1`` when running ``./release.sh``. |
a0c73b1a | 238 | |
7f68b728 XL |
239 | ITPs |
240 | ---- | |
95399141 | 241 | |
7f68b728 | 242 | Don't file ITPs for library crates, but do file them for binary crates. |
01f68998 | 243 | |
1ff2ded3 XL |
244 | Generally we'll be uploading a dozen crates or so at once. Submitting ITPs for |
245 | these is unnecessary since we're the only ones uploading and there is no chance | |
246 | of conflict. It would only be spam for the bug tracker. Please instead | |
247 | co-ordinate uploads on the ``#debian-rust`` IRC channel. | |
01f68998 | 248 | |
0d1e2247 SL |
249 | Testing |
250 | ------- | |
251 | ||
dc94b1d5 XL |
252 | Debian has two types of tests: |
253 | ||
1ff2ded3 XL |
254 | 1. pre-install tests run in ``debian/rules`` |
255 | 2. post-install tests defined in ``debian/tests/control`` | |
dc94b1d5 XL |
256 | |
257 | For Debian rust packages, in (1) we run the crate's test suite with default | |
258 | features but only if there are no dev-dependencies, and in (2) we run the whole | |
1ff2ded3 XL |
259 | test suite with each feature enabled separately plus ``--no-default-features`` |
260 | and ``--all-features``. | |
dc94b1d5 XL |
261 | |
262 | Sometimes, tests require extra tweaks and settings to work. In this case, you | |
263 | can tweak ``debian/rules`` for (1), and for (2) you will simply have to mark | |
264 | the relevant tests as broken using ``test_is_broken = true``. See the existing | |
265 | crate configs for examples. | |
266 | ||
267 | Other times, the tests are simply broken or can't be run in Debian. In this | |
268 | case you should disable the test in (1) by running ``dh_auto_test -- build`` | |
269 | instead of the default ``dh_auto_test -- test --all``, and for (2) again you | |
270 | should mark the relevant tests as broken. | |
b8798cdc SL |
271 | These tests are going to be marked as flaky in autopkgtest, still executed but |
272 | won't fail the autopkgtest run. | |
dc94b1d5 | 273 | |
ef62445b SL |
274 | Currently, using debcargo, it is not possible to add new dependencies as part |
275 | of an autopkgtest run. See https://salsa.debian.org/rust-team/debcargo/-/merge_requests/24 | |
276 | Instead, just override ``debian/tests/control``. See ``src/cbindgen/`` as | |
277 | example. | |
278 | ||
dc94b1d5 XL |
279 | Please note that ``[packages.lib]\ntest_is_broken = true`` will transitively |
280 | disable tests for all combinations of features. Sometimes this is correct e.g. | |
281 | if the test actually breaks for all features. Sometimes this is *not* correct, | |
282 | e.g. if the test only breaks for ``--no-default-features``. In the latter case | |
283 | you should instead patch the crate to ignore those tests when the relevant | |
1ff2ded3 | 284 | features are absent - e.g. ``src/regex-automata/debian/patches/ignore-std-tests.patch``. |
01f68998 | 285 | |
a1872536 XL |
286 | Binary-crate has "required-features" |
287 | ------------------------------------ | |
288 | ||
289 | See ``src/dotenv`` for an example on dealing with this. | |
290 | ||
291 | Binary-crate has conflicting name | |
292 | --------------------------------- | |
293 | ||
294 | See ``src/fd-find`` for an example on dealing with this. | |
295 | ||
053e604c SL |
296 | Updating the dependencies |
297 | ------------------------- | |
298 | ||
ed1eaac5 XL |
299 | In some cases, libraries/programs are forcing an old version of a library as |
300 | dependencies. In order to limit the number of duplicated libraries in the | |
301 | archive, please try to evaluate if a newer version of the dependencies could be | |
302 | used. | |
303 | ||
1ff2ded3 | 304 | To achieve that, after ``./update.sh``, try:: |
ed1eaac5 XL |
305 | |
306 | $ cd build/<package>/ | |
307 | $ rm -rf .pc # sometimes this is necessary due to minor debcargo bug | |
308 | $ quilt push -a | |
309 | $ quilt new relax-dep.diff | |
310 | $ quilt edit Cargo.toml | |
16650581 | 311 | $ quilt header -e --dep3 |
ed1eaac5 XL |
312 | $ quilt refresh |
313 | $ cargo build # check that it works. if it does, then | |
314 | $ cp -R patches ../../src/<package>/debian | |
315 | ||
316 | Suppose you want to change the dependency from 0.3 to 0.5. If the crate builds | |
317 | with no further source changes, then we would change the required version in | |
318 | ``Cargo.toml`` from ``0.3`` to ``>= 0.3, < 0.6`` or something like that. Then | |
319 | the convention is to put all these changes into a single patch called | |
320 | ``relax-dep-versions.patch``. | |
321 | ||
322 | OTOH, if the cargo build fails, and you can fix it up by editing the source | |
323 | code in a minor way to use the new crate API, then: for each crate that needs | |
324 | to be updated, you should instead name the patch ``update-dep-<crate>.patch`` | |
1ff2ded3 XL |
325 | and add both the ``Cargo.toml`` and the source code changes to it. The change |
326 | to ``Cargo.toml`` would then simply say (e.g.) ``0.5`` since the older versions | |
327 | actually don't work, and not the version range from the previous paragraph. | |
e4d0e3d6 | 328 | |
870cacd0 RK |
329 | If you want to make a crate work with an older dependency version than listed |
330 | in ``Cargo.toml`` (for example 0.3 instead of 0.5), you cannot use a flexible | |
331 | version requirement like ``>= 0.3, < 0.6``. Instead you have to specify only | |
332 | the older version, in this example ``0.3`` (`explanation`_). | |
333 | ||
334 | .. _explanation: https://salsa.debian.org/rust-team/debcargo-conf/merge_requests/86#note_135456 | |
335 | ||
16650581 | 336 | Information on patch headers is available in `dep3`_. |
932e534e | 337 | Use (some of) the headers to explain **why** the patch exists. |
16650581 | 338 | |
339 | .. _dep3: https://dep-team.pages.debian.net/deps/dep3/ | |
340 | ||
e4d0e3d6 XL |
341 | Help, something went wrong! |
342 | --------------------------- | |
343 | ||
344 | Sometimes, the error messages are not the most informative. In this case you | |
345 | can try re-running the command with ``RUST_BACKTRACE=1``. If you are using the | |
346 | ``debcargo`` from Debian's own repositories, you should also install the | |
347 | ``debcargo-dbgsym`` package, otherwise the stack trace will be next to useless. | |
348 | Make sure you have the `debug repository <https://wiki.debian.org/HowToGetABacktrace#Installing_the_debugging_symbols>`_ | |
349 | enabled in your APT sources. | |
65c5dc9d GS |
350 | |
351 | ||
352 | Some ramblings | |
353 | -------------- | |
354 | ||
355 | In ``#debian-rust`` came these two blog posts along with the remark of _good read_ | |
8c68fe9c GS |
356 | * https://blog.hackeriet.no/packaging-a-rust-project-for-debian/ |
357 | * https://blog.hackeriet.no/packaging-rust-part-II/ | |
65c5dc9d | 358 | |
16650581 | 359 | Now are they, those two blog posts, parked here. Waiting for better integration. |
901609d8 GS |
360 | |
361 | ||
362 | Developing Rust code using Debian-packaged crates | |
363 | ================================================= | |
364 | ||
365 | While perhaps not the stated intention, the Rust ecosystem in Debian | |
366 | is actually quite usable for developing Rust code in general. Thanks | |
367 | to `source replacement | |
368 | <https://doc.rust-lang.org/cargo/reference/source-replacement.html>`_, | |
369 | Cargo can be configured to use only local, Debian-provided packages by | |
844ab706 GS |
370 | placing something like the following in ``~/.cargo/config.toml`` (for |
371 | user-wide effect) or in a given project's ``.cargo/config.toml``:: | |
901609d8 GS |
372 | |
373 | [net] | |
374 | offline = true | |
375 | ||
376 | [source] | |
377 | ||
378 | [source.crates-io] | |
379 | replace-with = "debian" | |
380 | ||
381 | [source.debian] | |
1e1d8645 | 382 | directory = "/usr/share/cargo/registry" |
901609d8 GS |
383 | |
384 | In this state, Cargo will only look for crates installed as Debian | |
385 | packages on the local system. |