bors [Tue, 17 Mar 2020 14:23:20 +0000 (14:23 +0000)]
Auto merge of #8004 - ehuss:1.42-bump, r=alexcrichton
Bump stable version to 0.43.1
There was a hiccup where 0.43.0 was published to crates.io missing a change (#7848), see #7994 for details. It is not super critical, but in rare cases the bug can cause cargo used as a library to hang. I think bumping the version and re-publishing is relatively low-effort and low-risk.
This also includes backports to appease CI: #7883 #7906 #7955.
bors [Thu, 20 Feb 2020 03:25:54 +0000 (03:25 +0000)]
Auto merge of #7906 - ehuss:macos-10.15, r=alexcrichton
Switch azure to macOS 10.15.
Switches CI to the macOS 10.15 image. Since 32-bit support is no longer available, this changes how cross-compile testing works. I decided to use `x86_64-apple-ios` as a cross target, since it can easily build/link on macOS. `cargo run` won't work without a simulator, so some of the tests are restructured to check if `cargo run` is allowed. If you do have a simulator, it should Just Work. CI doesn't seem to be configured with a simulator installed, and I didn't bother to look if that would be possible (the simulators tend to be several gigabytes in size).
An alternative approach would be to use wasm as a cross target, which is also fairly easy to support. But wasm is a sufficiently different target that it can cause some issues in some tests, and is a bit harder to run as an executable.
This also adds some more help text on how to configure cross-compile tests.
Rustup is now installed on macOS by default, so no need to install it. Unfortunately self-updates are not allowed, but hopefully that won't be an issue.
bors [Sun, 26 Jan 2020 01:15:51 +0000 (01:15 +0000)]
Auto merge of #7829 - Mark-Simulacrum:fix-progress-panics, r=ehuss
Store maximum queue length
Previously, the queue length was constantly decreasing as we built crates, which
meant that we were incorrectly displaying the progress bar. In debug builds,
this even led to panics (due to underflow on subtraction).
Not sure if we can add a test case for this. I have made the panic unconditional on release/debug though by explicitly checking that current is less than the maximum for the progress bar.
Mark Rousskov [Sat, 25 Jan 2020 19:20:40 +0000 (14:20 -0500)]
Store maximum queue length
Previously, the queue length was constantly decreasing as we built crates, which
meant that we were incorrectly displaying the progress bar. In debug builds,
this even led to panics (due to underflow on subtraction).
bors [Fri, 24 Jan 2020 18:26:23 +0000 (18:26 +0000)]
Auto merge of #7826 - eddyb:recursion-limit-diagnostic, r=Eh2406
test: allow some flexibility in check::error_from_deep_recursion's expected diagnostic.
This should unblock https://github.com/rust-lang/rust/pull/68407, by loosening the expected output pattern.
As per https://github.com/rust-lang/rust/pull/68407#issuecomment-578189644, this is the change in the diagnostic:
```diff
-recursion limit reached while expanding the macro `m`
+recursion limit reached while expanding `m!`
```
Ideally I would use something like this regex:
```
recursion limit reached while expanding (the macro `m`|`m!`)
```
but AFAIK these tests don't support regexes.
There were no tests for `cargo owner -a/-r` and `cargo yank --undo`. However, These commands in tests had empty response. So I also update response handling with reference to https://github.com/rust-lang/cargo/pull/3301/commits/7dd0f932a864d97bef5da26a0e148fca3f06d448.
bors [Wed, 22 Jan 2020 22:42:20 +0000 (22:42 +0000)]
Auto merge of #7731 - Mark-Simulacrum:chatty-jobserver, r=alexcrichton
Scalable jobserver for rustc
This refactors the job queue code for support of [per-rustc process jobservers](https://github.com/rust-lang/rust/pull/67398). Furthermore, it also cleans up the code and refactors the main loop to be more amenable to understanding (splitting into methods and such).
Assignment of tokens to either rustc "threads" or processes is dedicated to the main loop, which proceeds in a strict "least recently requested" fashion among both thread and process token requests. Specifically, we will first allocate tokens to all pending process token requests (i.e., high-level units of work), and then (in per-rustc jobserver mode) follow up by assigning any remaining tokens to rustcs, again in the order that requests came into cargo (first request served first).
It's not quite clear that that model is good (no modeling or so has been done). On the other hand this strategy should mean that long-running crates will get more thread tokens once we bottom out in terms of rustc parallelism than short-running crates, which means that crates like syn which start early on but finish pretty late should hopefully get more parallelism nicely (without any more complex heuristics).
One plausible change that may be worth exploring is making the assignment prefer earlier rustc's, globally, rather than first attempting to spawn new crates and only then increasing parallelism for old crates. syn for example frequently gets compiled in the early storm of dozens of crates so is somewhat unlikely to have parallelism, until fairly late in its compilation.
We also currently conflate under this model the rayon threads and codegen threads. Eventually inside rustc those will probably(?) also be just one thing, and the rustc side of this implementation provides no information as to what the token request is for so we can't do better here yet.
Mark Rousskov [Mon, 20 Jan 2020 17:50:04 +0000 (12:50 -0500)]
Refactor to_send_clients to use a BTreeMap
This is both a performance optimization (avoiding O(n) shifting from the
beginning), and communicates intent in a nicer way overall.
It is plausible that we will eventually want to tie this data structure to
something like the DependencyQueue, i.e., to get more information on which rustc
to give tokens to. An old rustc with a very late dependency edge is less
important than one we'll need sooner, probably.
Mark Rousskov [Thu, 16 Jan 2020 01:35:51 +0000 (20:35 -0500)]
Do not send acquired tokens to waiting rustc threads
This removes the ad-hoc token re-send in the message processing; this sort of
decision should be left up to the main loop which manages tokens.
Notably, the behavior change here is that new tokens will go solely to spawning
new rustc *processes* rather than increasing rustc internal parallelism, unless
we can't spawn new processes.
Otherwise, before this commit, we may be saturating a single rustc with tokens
rather than creating lots of rustcs that can work in parallel. In particular in
the beginning of a build, it's likely that this is worse (i.e., crates are small
and rustc internal parallelism is not at that point all that helpful) since it
severely limits the benefits of pipelining and generally makes the build
nearly serial.
Mark Rousskov [Thu, 16 Jan 2020 01:24:05 +0000 (20:24 -0500)]
Split waiting for an event out of the primary drainer
This has the slight behavior change where we won't ask for new dependencies and
so forth if no events have been received but I believe that there's no activity
that can happen if an event hasn't occurred (i.e., no state change has occurred)
so there's no need for us to actually do anything in practice.
To make sure we still record CPU usage and such sufficiently often that is also
moved into the inner "waiting for events" loop.
bors [Wed, 22 Jan 2020 00:59:36 +0000 (00:59 +0000)]
Auto merge of #7819 - ehuss:fix-replay-newlines, r=alexcrichton
Fix cache replay including extra newlines.
The compiler output cache replay was changed in #7737 to use `BufReader::read_line` instead of `str::lines`. `read_line`, unlike `lines`, includes the trailing line ending. The code is written assuming that the line endings are stripped, so make sure they are stripped here, too.
This only happens for non-JSON messages, like `RUSTC_LOG`.
bors [Tue, 21 Jan 2020 16:15:39 +0000 (16:15 +0000)]
Auto merge of #7798 - jnbr:dylib_path, r=alexcrichton
Fix wrong directories in host_libdir.
This fixes a regression from #7482 where the sysroot_target_libdir leaks into the host libdir. This can cause problems when the dynamic linker does not ignore the target libraries but tries to load them instead. This happens for example when building on x86_64-musl for aarch64-musl.
bors [Mon, 20 Jan 2020 20:33:50 +0000 (20:33 +0000)]
Auto merge of #7815 - rust-lang:dependabot/cargo/humantime-2.0.0, r=Eh2406
Update humantime requirement from 1.2.0 to 2.0.0
Updates the requirements on [humantime](https://github.com/tailhook/humantime) to permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/tailhook/humantime/commit/d478f8a7878edba8a609a287776e27fc16d7009b"><code>d478f8a</code></a> Version bumped to v2.0.0</li>
<li><a href="https://github.com/tailhook/humantime/commit/49f11fdc2a59746085d2457cb46bce204dec746a"><code>49f11fd</code></a> Another improvement of the error message</li>
<li><a href="https://github.com/tailhook/humantime/commit/8a13b047ca0dc731e0c515b1db544fe1fd75bd7d"><code>8a13b04</code></a> Nicer error message for plain numeric duration</li>
<li><a href="https://github.com/tailhook/humantime/commit/edfa493e8cb8217aa5a0cc638398f60d67648c93"><code>edfa493</code></a> vagga.yaml: upgrade rust to 1.31.0</li>
<li><a href="https://github.com/tailhook/humantime/commit/8b8d748566c85a73b2f940e755dc0160c93f465a"><code>8b8d748</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/tailhook/humantime/issues/13">#13</a> from gh0st42/master</li>
<li><a href="https://github.com/tailhook/humantime/commit/da7723529fa30222b73561fad0ed7ce47d9a15a4"><code>da77235</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/tailhook/humantime/issues/12">#12</a> from koushiro/works</li>
<li><a href="https://github.com/tailhook/humantime/commit/227d7e591dd76bbfbca6a236d2e8661ddfd16d63"><code>227d7e5</code></a> Made crate unsafe free and forbid unsafe</li>
<li><a href="https://github.com/tailhook/humantime/commit/b7da4ab6ad24f68660cf92ede8dc88d7400a39fa"><code>b7da4ab</code></a> Downgrade MSRV to 1.31</li>
<li><a href="https://github.com/tailhook/humantime/commit/4a406825951039c6c3414b76c855dc38384d874d"><code>4a40682</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/tailhook/humantime/issues/11">#11</a> from koushiro/apply-rustfmt-and-clippy</li>
<li><a href="https://github.com/tailhook/humantime/commit/f00dbbae37ac281c919fbf0d56caf5900f1e0085"><code>f00dbba</code></a> Apply suggestions</li>
<li>Additional commits viewable in <a href="https://github.com/tailhook/humantime/compare/v1.2.0...v2.0.0">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
Updates the requirements on [humantime](https://github.com/tailhook/humantime) to permit the latest version.
- [Release notes](https://github.com/tailhook/humantime/releases)
- [Commits](https://github.com/tailhook/humantime/compare/v1.2.0...v2.0.0)
bors [Wed, 15 Jan 2020 00:31:05 +0000 (00:31 +0000)]
Auto merge of #7774 - giraffate:update_credentials, r=ehuss
Load credentials only when needed
Credentials are always loaded, even if these are not used. If
access to confidential files such as credentials is not given,
`cargo build` fails despite not using credentials.
This fixes a regression from #7475 where the sysroot_target_libdir leaks into
the host libdir. This can cause problems when the dynamic linker does
not ignore the target libraries but tries to load them instead. This
happens for example when building on x86_64-musl for aarch64-musl.
bors [Mon, 13 Jan 2020 21:37:15 +0000 (21:37 +0000)]
Auto merge of #7750 - ehuss:named-config-profiles, r=alexcrichton
Add named config profiles.
This adds support for named config profiles. Previously, only `dev` and `release` were allowed in config files, it now supports all profile names. I think it would be strange to have arbitrarily named profiles in `Cargo.toml`, but not allow them in config. This is a deviation from the RFC, but RFC 2282 was written before named profiles which I think changes the landscape.
This diff is a little large due to some refactoring to make it work well. Overview of the changes:
- Removed `ProfileKind` and only use an `InternedString` to track the name of the profile. I didn't feel like the enum carried its cognitive weight, and it seems to simplify some things.
- `Profiles` is no longer stored in the manifest. There was no need to do a bunch of processing for each manifest. `Manifest` now only retains the low-level `TomlProfiles`. A single `Profiles` now lives in `BuildContext`.
- The profile name requested by the user is no longer passed around. It is given to `Profiles::new` and retained inside `Profiles`.
- `Profiles::get_profile` no longer follows the priority stack and inheritance each time a profile is requested. Instead, the profile is computed once (in `Profile::new`) and merged into a single profile. This simplifies getting a profile, and makes it easier to deal with getting the config values in one place.
- I switched profile names to be `InternedString` instead of `String`. There's not a strong reason to do this, other than it seemed a little strange to be creating lots of `String`s.
- I also added `PartialEq<str>` for `InternedString`. It has come up a few times in the past, and it seems useful. I'm not sure if it was excluded intentionally?
- The validation that the profile exists is now done in one place (`Profiles::new`).
- I removed the back-compatibility for the `overrides` key (which was renamed to `package` back in October).
Notes:
- Some of the error messages aren't as good as before, because they don't tell you where the error is located (`Cargo.toml` or `.cargo/config`). This is because the location data is lost by the time validation is done. Hopefully it will be obvious from the profile name and error message. I tried to improve error messages wherever I could.
- There are more calls to `clone()` than I would like, but they are kinda hard to avoid. Should be fewer than before.
- I noticed a bug with `-Zpanic-abort-tests` not supporting named profiles. I'll fix that separately.
- I think this fixes some bugs where package overrides in config weren't merging properly with package overrides defined in `Cargo.toml`.
I struggle when there are multiple types with the same name in the same code base. I think this makes it a little clearer what the type is.
I was tempted to also rename `registry::Kind`, but I could not think of a good name. That file is particularly hard for me to understand (locked vs normal sources, abstract trait, etc.), so I don't feel comfortable changing it. It's also localized in one file, so not as important.
Alex Crichton [Wed, 8 Jan 2020 22:51:49 +0000 (14:51 -0800)]
Use `context` to create a chain of errors
There's an existing bug (#7782) in Cargo which exacerbates the issue
here but in general having a stack of errors is a bit easier to read and
work with than having a big long error message.
bors [Wed, 8 Jan 2020 22:28:54 +0000 (22:28 +0000)]
Auto merge of #7779 - ehuss:fix-cargo-lock-ignore, r=alexcrichton
Fix .gitignore of Cargo.lock in a subdirectory.
The code for checking if `Cargo.lock` is ignored was erroneously assuming it was at the root of the git repo. This would cause a problem if `Cargo.lock` is in `.gitignore` in a subdirectory.
Fixes issue noted in https://github.com/rust-lang/cargo/issues/7705#issuecomment-572027382
bors [Wed, 8 Jan 2020 17:50:44 +0000 (17:50 +0000)]
Auto merge of #7776 - alexcrichton:anyhow, r=ehuss
Migrate from the `failure` crate to `anyhow`
The `anyhow` crate interoperates with the `std::error::Error` trait
rather than a custom `Fail` trait, and this is the general trend of
error handling in Rust as well.
Note that this is mostly mechanical (sed) and intended to get the test
suite passing. As usual there's still more idiomatic cleanup that can
happen, but that's left to later commits.
Alex Crichton [Tue, 7 Jan 2020 22:30:15 +0000 (14:30 -0800)]
Migrate from the `failure` crate to `anyhow`
The `anyhow` crate interoperates with the `std::error::Error` trait
rather than a custom `Fail` trait, and this is the general trend of
error handling in Rust as well.
Note that this is mostly mechanical (sed) and intended to get the test
suite passing. As usual there's still more idiomatic cleanup that can
happen, but that's left to later commits.
bors [Tue, 7 Jan 2020 15:25:14 +0000 (15:25 +0000)]
Auto merge of #7771 - moxian:needless-borrow, r=alexcrichton
Fix several needless_borrow clippy lints.
Fixes several [needless_borrow](https://rust-lang.github.io/rust-clippy/v0.0.212/index.html#needless_borrow) clippy lints.
I've only fixed this kind of lint errors and not others, since for some reason these are the only ones that show as red squigglies in my editor of choice.
Takayuki Nakata [Tue, 7 Jan 2020 14:02:33 +0000 (23:02 +0900)]
Load credentials only when needed
Credentials are always loaded, even if these are not used. If
access to confidential files such as credentials is not given,
`cargo build` fails despite not using credentials.
bors [Mon, 6 Jan 2020 19:11:37 +0000 (19:11 +0000)]
Auto merge of #7758 - jblazquez:fix-windows-uwp-dynamic-linking, r=alexcrichton
Fix dynamic linking for Windows UWP MSVC targets
When creating a dynamic library, the MSVC linker generates an import library (.lib) next to the .dll file. Cargo has explicit knowledge of this and includes those generated .dll.lib on the list of files generated by a Cargo invocation.
However, the check to see if those import libraries must be included is too strict and doesn't match any Windows targets that don't end in `pc-windows-msvc`. For example, https://github.com/rust-lang/rust/pull/63155 added several new Windows targets for targeting UWP called `*-uwp-windows-msvc`. The end result is that the sysroot for these UWP toolchains don't contain a `std-XXX.dll.lib` file and thus any executable that uses `-C prefer-dynamic` will fail to link because the `std` library is not linked at all.
This change relaxes the test and makes Cargo know about those import libraries for all Windows MSVC targets.
#7649 caused an unfortunate regression where the `CARGO_TARGET_triple_LINKER` environment variable stopped working. I did not realize `serde(flatten)` caused serde to switch to `deserialize_map` which does not support environment variables.
The solution here is to essentially revert back to how the `[target]` table used to be loaded by loading each key individually.
This also removes the `ar` field which is not used by `rustc`.