bors [Thu, 18 Jun 2020 17:14:16 +0000 (17:14 +0000)]
Auto merge of #8363 - alexcrichton:less-git-data, r=ehuss
Cut down on data fetch from git dependencies
Currently Cargo pretty heavily over-approximates data fetch for git
dependencies. For the index it fetches precisely one branch, but for all
other git dependencies Cargo will fetch all branches and all tags all
the time. In each of these situations, however, Cargo knows if one
branch is desired or if only one tag is desired.
This commit updates Cargo's fetching logic to plumb the desired
`GitReference` all the way down to `fetch`. In that one location we then
determine what to fetch. Namely if a branch or tag is explicitly
selected then we only fetch that one reference from the remote, cutting
down on the amount of traffic to the git remote.
Additionally a bugfix included here is that the GitHub fast path for
checking if a repository is up-to-date now works for non-`master`-based
branch dependencies.
Alex Crichton [Tue, 16 Jun 2020 16:02:01 +0000 (09:02 -0700)]
Update how locked git commits are fetched
This commit refactors various logic of the git source internals to
ensure that if we have a locked revision that we plumb the desired
branch/tag all the way through to the `fetch`. Previously we'd switch to
`Rev` very early on, but the fetching logic for `Rev` is very eager and
fetches too much, so instead we only resolve the locked revision later
on.
Internally this does some various refactoring to try to make various
bits and pieces of logic a bit easyer to grok, although it's still
perhaps not the cleanest implementation.
Alex Crichton [Mon, 15 Jun 2020 20:36:23 +0000 (13:36 -0700)]
Cut down on data fetch from git dependencies
Currently Cargo pretty heavily over-approximates data fetch for git
dependencies. For the index it fetches precisely one branch, but for all
other git dependencies Cargo will fetch all branches and all tags all
the time. In each of these situations, however, Cargo knows if one
branch is desired or if only one tag is desired.
This commit updates Cargo's fetching logic to plumb the desired
`GitReference` all the way down to `fetch`. In that one location we then
determine what to fetch. Namely if a branch or tag is explicitly
selected then we only fetch that one reference from the remote, cutting
down on the amount of traffic to the git remote.
Additionally a bugfix included here is that the GitHub fast path for
checking if a repository is up-to-date now works for non-`master`-based
branch dependencies.
bors [Mon, 15 Jun 2020 14:38:34 +0000 (14:38 +0000)]
Auto merge of #8359 - ehuss:doctest-xcompile-linker, r=alexcrichton
Support linker with -Zdoctest-xcompile.
This adds support for `-Clinker` with `-Zdoctest-xcompile`.
I'm not entirely sure how `-Zdoctest-xcompile` was supposed to work without setting the linker. I tested this with std on arm-unknown-linux-gnueabihf with qemu. It seems to work (although it was quite slow).
bors [Mon, 15 Jun 2020 14:05:10 +0000 (14:05 +0000)]
Auto merge of #8358 - ehuss:fix-target-host-doctest, r=alexcrichton
Fix doctests not running with --target=HOST.
There was a regression in #8167 where `cargo test --target=$HOST` stopped running doctests. This caused doctests to silently stop running in rust-lang/rust (https://github.com/rust-lang/rust/issues/73286). This PR restores the original behavior where `--target=$HOST` behaves as-if it is a normal host test.
There was a discussion about this at https://github.com/rust-lang/cargo/pull/8167#pullrequestreview-402196323, but I think I let it slip through the cracks.
bors [Thu, 11 Jun 2020 22:13:37 +0000 (22:13 +0000)]
Auto merge of #8353 - ehuss:fix-missing-readme, r=alexcrichton
Fix failure with missing readme.
#8277 added implicit README support, but it also rejected parsing any manifest where the README was missing. This causes a problem because the README is often missing in many registry packages (for various reasons).
This removes the validation at parsing time. Cargo has historically not had hard enforcement at the parsing stage. Whether or not the readme exists has always been enforced during publishing. I have added some extra context to the error message, and added a test to that effect.
bors [Thu, 11 Jun 2020 15:34:15 +0000 (15:34 +0000)]
Auto merge of #8349 - ehuss:fix-lto, r=alexcrichton
Some LTO fixes.
This reworks the LTO computation a little to address a few issues:
- `cargo build` in a project with both a lib and bin would not engage the optimization introduced in #8192 where the lib *should* be compiled with `-C linker-plugin-lto` (bitcode only). This happened because the old code was starting root units as `Lto::None`. The solution here is to conditionally choose the starting Lto for roots.
- A project with a dylib dependency would fail to build. It was building the dylib with `-C linker-plugin-lto` which is not valid.
- A project with a bin/lib would build the lib differently based on whether or not it was selected. This changes it so that the lib is built the same. See `lto::between_builds`, where the second build the lib is now fresh.
- Tests/benchmarks of a `lib` target will now support LTO.
- Treats example libs a little more consistently as regular libs.
I scattered some comments throughout, hopefully it's not too difficult to follow.
bors [Tue, 9 Jun 2020 20:03:14 +0000 (20:03 +0000)]
Auto merge of #8277 - tverghis:default-readme, r=alexcrichton
Default values for `readme` if not specified
If the a value for `readme` is not specified in Cargo.toml, we will now check for the existence of files named `README.md`, `README.txt` or `README`. If one does exist, the name of that file will be defaulted in to the manifest for the project.
This behavior can be suppressed if `readme` is set to `false`.
bors [Mon, 8 Jun 2020 14:19:43 +0000 (14:19 +0000)]
Auto merge of #8267 - drmikehenry:prefix, r=alexcrichton
Support `{prefix}` and `{lowerprefix}` markers in `config.json` `dl` key
Hello,
The crates.io-index Git repository uses a nice directory structure to keep individual directory sizes under control.
When mirroring crates.io, it's useful to store crate files in a similar directory structure for the same reasons.
Cargo provides "markers" for use in the `dl` key of the `config.json` file in crates.io-index to allow flexibility in mapping a crate's name and version into a URL for the crate. The marker `{crate}` is replaced by the crate's name, and the marker `{version}` is replaced with the crate's version. The default URL template is `https://crates.io/api/v1/crates/{crate}/{version}/download`.
Currently, if a mirror of crates.io stores crates in a directory structure similar to that of crates.io-index, it's up to the server to construct the directory name from the crate name. This eliminates trivial web servers and `file:` URLs from hosting such a tree of crates.
This pull requests adds two new markers for the `dl` key in `config.json`, allowing Cargo to supply the directory name as part of the URL. The marker `{lowerprefix}` is the same directory name used within crates.io-index; it is calculated from the crate name converted to lowercase. The marker `{prefix}` is similar, but it uses the crate name as-is (without case conversion), which is useful for supporting older versions of Cargo that lack these markers; for example, nginx rewrite rules can easily construct `{prefix}` but can't perform case-conversion to construct `{lowerprefix}`. These new markers will provide implementation flexibility and simplicity for crate mirror servers.
bors [Fri, 5 Jun 2020 16:39:10 +0000 (16:39 +0000)]
Auto merge of #8331 - ehuss:1.45-beta-backport, r=Eh2406
1.45 beta backports
Beta backports for:
* #8290 — Fix fingerprinting for lld on Windows with dylib.
* #8329 — Don't hash executable filenames on apple platforms. (fix macos backtraces)
bors [Fri, 5 Jun 2020 14:31:21 +0000 (14:31 +0000)]
Auto merge of #8329 - ehuss:apple-no-hash, r=alexcrichton
Don't hash executable filenames on apple platforms.
Due to some recent changes to the backtrace crate, backtraces on apple platforms haven't been working (they are missing line/filename information). The reason is that previously libbacktrace would hunt through the directory for any matching file in the `.dSYM` directory. The new implementation expects a file matching the executable name exactly (which no longer includes the hash because Cargo renames it).
The solution here is to not include a hash in the executable filename. This matches the behavior on Windows which does it for a similar reason (paths are embedded in pdb files).
The downside is that switching between different settings (like different features) causes Cargo to rebuild the binary each time. I don't think this is a particularly common use case, at least I've not heard any complaints about this behavior on Windows.
bors [Wed, 27 May 2020 19:33:35 +0000 (19:33 +0000)]
Auto merge of #8290 - ehuss:fix-lld-freshness, r=alexcrichton
Fix fingerprinting for lld on Windows with dylib.
This fixes an issue where if `lld` is used on Windows, dynamic libraries will never be treated as "fresh". This is a regression from #8210 where Cargo is expecting export files to be created, but lld does not create these.
The solution is to ignore "Auxiliary" files in fingerprinting, which AFAIK aren't really needed (only the primary output files really matter).
bors [Fri, 5 Jun 2020 14:31:21 +0000 (14:31 +0000)]
Auto merge of #8329 - ehuss:apple-no-hash, r=alexcrichton
Don't hash executable filenames on apple platforms.
Due to some recent changes to the backtrace crate, backtraces on apple platforms haven't been working (they are missing line/filename information). The reason is that previously libbacktrace would hunt through the directory for any matching file in the `.dSYM` directory. The new implementation expects a file matching the executable name exactly (which no longer includes the hash because Cargo renames it).
The solution here is to not include a hash in the executable filename. This matches the behavior on Windows which does it for a similar reason (paths are embedded in pdb files).
The downside is that switching between different settings (like different features) causes Cargo to rebuild the binary each time. I don't think this is a particularly common use case, at least I've not heard any complaints about this behavior on Windows.
Passing a relative path to Workspace now bails with proper message.
Previously, this failure will return an unhelpful warning.
This commit adds an error message saying that the argument for
`manifest_path` must be an absolute path.
bors [Wed, 3 Jun 2020 15:03:10 +0000 (15:03 +0000)]
Auto merge of #8319 - alexcrichton:fix-access, r=Eh2406
Fix an accidental raw access of field
The manifest has a few different ways of specifying whether a crate is a
procedural macro, and there's a `TomlTarget::proc_macro()` method to
unify these various lines. Unfortunately though we had a bug where one
location forgot to call the method and read the raw field! This led to
surprising behavior where the different ways to specify a proc macro
would have subtly different changes in behavior. The fix here in this PR
is to ensure that we access the property always via the method.
Alex Crichton [Wed, 3 Jun 2020 14:38:21 +0000 (07:38 -0700)]
Fix an accidental raw access of field
The manifest has a few different ways of specifying whether a crate is a
procedural macro, and there's a `TomlTarget::proc_macro()` method to
unify these various lines. Unfortunately though we had a bug where one
location forgot to call the method and read the raw field! This led to
surprising behavior where the different ways to specify a proc macro
would have subtly different changes in behavior. The fix here in this PR
is to ensure that we access the property always via the method.
bors [Tue, 2 Jun 2020 16:35:38 +0000 (16:35 +0000)]
Auto merge of #8307 - pickfire:alias-help, r=alexcrichton
Show alias in help message
Improve feature discovery of help message
Inspired by x.py help https://github.com/rust-lang/rust/issues/71357
Improves https://github.com/rust-lang/cargo/issues/6104
Ivan Tham [Tue, 2 Jun 2020 12:52:00 +0000 (20:52 +0800)]
Show alias in help message
Improve feature discovery of help message
Inspired by x.py help https://github.com/rust-lang/rust/issues/71357
Improves https://github.com/rust-lang/cargo/issues/6104
bors [Mon, 1 Jun 2020 22:35:00 +0000 (22:35 +0000)]
Auto merge of #8297 - mjarkk:warn-when-using-hash-in-git-url, r=ehuss
Warn if using hash in git URL, Fixes #8241
This fixes an issue where if the user wants to set the git rev but doesn't know how and as results tries to set the ref in the url hash as also shown when downloading the dependency.
Now cargo returns a warning notifying the user about the correct way to set the ref.
bors [Mon, 1 Jun 2020 16:01:10 +0000 (16:01 +0000)]
Auto merge of #8274 - Eh2406:8249-repro, r=alexcrichton
reset lockfile information between resolutions
#8249 pointed out that some kind of lockfile data was leaking between calls to the resolver. @ehuss made a reproducing test case. This PR resets the `LockedMap` data structure when calling `register_previous_locks`.
bors [Sat, 30 May 2020 17:08:19 +0000 (17:08 +0000)]
Auto merge of #8287 - ehuss:rustdoc-map, r=alexcrichton
Add support for rustdoc root URL mappings.
This adds an experimental configuration setting to allow Cargo to pass the `--extern-html-root-url` flag to rustdoc. This flag allows rustdoc to link to other locations when a dependency is not locally documented. See the documentation in `unstable.md` for more details.
There are some known issues with this implementation:
* Rustdoc doesn't seem to know much about renamed dependencies. The links it generates are to the package name, not the renamed name. The code is written to pass in package names, but if there are multiple dependencies to the same package, it won't work properly.
* Similarly, if there are multiple versions of the same package within the dep graph, rustdoc will only link to one of them. To fix this, Cargo would need to pass metadata info into rustdoc (such as the package version).
* If a dependency is built with different features than what is on docs.rs, some links may break.
* This explodes the command-line length significantly. Before stabilizing, we may want to consider addressing that. I'm not sure if it would make sense to change rustdoc's interface, or to use response files?
* This does not pass mappings for transitive dependencies. This normally isn't an issue, but can arise for re-exports (see the `alt_registry` test for an example). I'm not sure if this is a bug in rustdoc or not (there is a large number of issues regarding reexports and rustdoc). Cargo could include these, but this would make the command-line length even longer. Not sure what to do here.
* The config value does not support environment variables. This would be very difficult to support, because Cargo doesn't retain the registry name in `SourceId`. I looked into fixing that, but it is very difficult, and hard to make it reliable.
I have tried to consider future changes in this design, to ensure it doesn't make them more difficult:
* Single-tab browsing. This would be a mode where the std docs are merged with the local crate's docs so that the std docs are shown in the same place (and included in the index). This could be expressed with something like `doc.extern-map.std = "include"` or something like that. (Or maybe just use build-std?)
* Direct-dependencies only. Often transitive dependencies aren't that interesting, and take up a lot of space in the output, and clog the search index. Some users want the ability to (locally) document their package + direct dependencies only. I think this could be implemented with some kind of command-line flag, perhaps with a config setting in the `[doc]` table. `--extern-html-root-url` flag will automatically handle second-level dependencies.
* Manual-exclusions. Sometimes there are specific dependencies that are very expensive to document locally, but you still want everything else. I think this could be implemented with a command-line flag (`--exclude winapi`?), and the rustdoc-map feature would automatically link those excluded crates' items to docs.rs. This could also be added to the `[doc]` table.
We can also consider at any time to change the defaults (such as making `crates-io = "https://docs.rs"` the default). It could also potentially auto-detect `std = "local"`, although rustdoc could do the same internally.
bors [Wed, 27 May 2020 19:33:35 +0000 (19:33 +0000)]
Auto merge of #8290 - ehuss:fix-lld-freshness, r=alexcrichton
Fix fingerprinting for lld on Windows with dylib.
This fixes an issue where if `lld` is used on Windows, dynamic libraries will never be treated as "fresh". This is a regression from #8210 where Cargo is expecting export files to be created, but lld does not create these.
The solution is to ignore "Auxiliary" files in fingerprinting, which AFAIK aren't really needed (only the primary output files really matter).
bors [Tue, 26 May 2020 19:04:00 +0000 (19:04 +0000)]
Auto merge of #8286 - ehuss:fix-close-output, r=alexcrichton
Fix several issues with close_output test.
This test had several mistakes in it:
* On Windows, the test took several minutes to run. The sleep ran very slowly, and wasn't even necessary.
* On Windows, there is an output filename collision with `foo.pdb` because it is used for both the DLL and the executable. (Note: I'm not sure, maybe we should not uplift proc-macros?)
* The output wasn't being verified (I forgot that `lines_match` returns a bool, and does not panic). Someday in the future I would like to use a general-purpose line match/differ that is easier to use (not just on `ProcessBuilder`).
bors [Sat, 23 May 2020 19:19:52 +0000 (19:19 +0000)]
Auto merge of #8269 - ehuss:tree-all-targets, r=alexcrichton
Fix panic with `cargo tree --target=all -Zfeatures=all`
When `cargo tree --target=all` was used with the new feature resolver, this would cause a panic because the feature resolver doesn't know about the "all" behavior, and would filter out packages that don't match.
I don't feel like this is a particularly elegant solution, but I'm uncertain of how to make it better.
bors [Sat, 23 May 2020 18:38:03 +0000 (18:38 +0000)]
Auto merge of #8272 - ehuss:fix-llvm-tools, r=alexcrichton
Fix nightly tests with llvm-tools.
https://github.com/rust-lang/rust/pull/72000 removed `libLLVM-10-rust-1.45.0-nightly.so` from the default install. Several of the plugins tests fail without this, so make sure it gets installed.
Michael Henry [Fri, 22 May 2020 15:15:24 +0000 (11:15 -0400)]
Support `{prefix}` and `{lowerprefix}` markers in `config.json` `dl` key.
These new markers allow Cargo to supply a directory name (similar to
that used in crates.io-index) as part of a crate's download URL,
enabling simpler hosting of crates. Previously, a `file` URL would need
to put all crates into a single huge directory (such as `/srv/crates/`),
e.g.:
bors [Thu, 21 May 2020 14:29:50 +0000 (14:29 +0000)]
Auto merge of #8248 - ehuss:patch-err-help, r=alexcrichton
Provide better error messages for a bad `patch`.
This attempts to provide more user-friendly error messages for some situations with a bad `patch`. This is a follow-up to #8243.
I think this more or less covers all the issues from #4678. I imagine there are other corner cases, but those will need to wait for another day. The main one I can think of is when the patch location is missing required features. Today you get a "blah was not used in the crate graph." warning, with some suggestions added in #6470, but it doesn't actually check if there is a feature mismatch.
bors [Wed, 20 May 2020 20:48:44 +0000 (20:48 +0000)]
Auto merge of #8022 - illicitonion:trywithout, r=ehuss
Try installing exact versions before updating
When an exact version is being installed, if we already have that
version from the index, we don't need to update the index before
installing it. Don't do this if it's not an exact version, because the
update may find us a newer version.
This is particularly useful for scripts which unconditionally run
`cargo install some-crate --version=1.2.3`. Before install-update, I
wrote a crate to do this
(https://crates.io/crates/cargo-ensure-installed) which I'm trying to
replace with just `cargo install`, but the extra latency of updating the
index for a no-op is noticeable.
projects / cargo.git / log