]> git.proxmox.com Git - cargo.git/log
cargo.git
4 years agoAuto merge of #7857 - ehuss:fix-build-script-dupe, r=alexcrichton
bors [Fri, 7 Feb 2020 15:35:03 +0000 (15:35 +0000)]
Auto merge of #7857 - ehuss:fix-build-script-dupe, r=alexcrichton

Fix BuildScriptOutput when a build script is run multiple times.

When I implemented profile build overrides, I created a scenario where a build script could run more than once for different scenarios.  See the test for an example.

However, the `BuildScriptOutputs` map did not take this into consideration.  This caused multiple build script runs to stomp on the output of previous runs.  This is further exacerbated by the new feature resolver in #7820 where build scripts can run with different features enabled.

The solution is to make the map key unique for the Unit it is running for.  Since this map must be updated on different threads, `Unit` cannot be used as a key, so I chose just using the metadata hash which is hopefully unique.  Most of this patch is involved with the fussiness of getting the correct metadata hash (we want the RunCustomBuild unit's hash).  I also added some checks to avoid collisions and assert assumptions.

4 years agoAuto merge of #7875 - ehuss:update-jobserver, r=alexcrichton
bors [Thu, 6 Feb 2020 20:44:26 +0000 (20:44 +0000)]
Auto merge of #7875 - ehuss:update-jobserver, r=alexcrichton

Update jobserver.

Keep in sync with rust-lang/rust (https://github.com/rust-lang/rust/pull/68663), so that local users and lib users get the same version.

4 years agoAuto merge of #7874 - ehuss:update-tar, r=Eh2406
bors [Thu, 6 Feb 2020 19:38:30 +0000 (19:38 +0000)]
Auto merge of #7874 - ehuss:update-tar, r=Eh2406

Update tar.

Updates to the latest tar.  rust-lang/rust is currently on 0.4.20.  The change I'm most interested in is fixing the TarError cause/source, so that Cargo reports a better error message.  Compare:

```
Caused by:
  failed to unpack `/…/registry/src/github.com-1ecc6299db9ec823/curl-sys-0.4.25/curl/docs/libcurl/libcurl-multi.3`
```

to the new version:

```
Caused by:
  failed to unpack `curl-sys-0.4.25/curl/docs/libcurl/libcurl-multi.3` into `/…/registry/src/github.com-1ecc6299db9ec823/curl-sys-0.4.25/curl/docs/libcurl/libcurl-multi.3`

Caused by:
  No space left on device (os error 28)
```

4 years agoUpdate jobserver.
Eric Huss [Thu, 6 Feb 2020 19:30:07 +0000 (11:30 -0800)]
Update jobserver.

4 years agoUpdate tar.
Eric Huss [Thu, 6 Feb 2020 19:29:07 +0000 (11:29 -0800)]
Update tar.

4 years agoAuto merge of #7872 - ehuss:timings-error, r=Eh2406
bors [Thu, 6 Feb 2020 18:57:21 +0000 (18:57 +0000)]
Auto merge of #7872 - ehuss:timings-error, r=Eh2406

Emit report on error with Ztimings.

Previously the report was not saved on error.

I'm not actually sure this is all that useful.  I was using it to gather a picture of what was being built (I wasn't actually interested in the timing data).  There might be better ways to accomplish what I wanted, but it's a small change, so probably doesn't hurt.

Fixes #7413.

4 years agoEmit report on error with Ztimings.
Eric Huss [Thu, 6 Feb 2020 18:45:22 +0000 (10:45 -0800)]
Emit report on error with Ztimings.

4 years agoAuto merge of #7827 - Kinrany:7656-format-placeholder-code-when-generating-a-crate...
bors [Wed, 5 Feb 2020 16:50:49 +0000 (16:50 +0000)]
Auto merge of #7827 - Kinrany:7656-format-placeholder-code-when-generating-a-crate, r=ehuss

Format placeholder code when generating a crate

When generating source files in `cargo new` and `cargo init`, try to run `rustfmt` CLI on them.

If it fails, log the error and proceed.

Tests:
* Works for `cargo init --lib`
* No changes in behavior if `rustfmt` is missing

Closes #7656

4 years agoDo not run `formats_source` if `rustfmt` is not available
Kinrany [Wed, 5 Feb 2020 07:09:43 +0000 (10:09 +0300)]
Do not run `formats_source` if `rustfmt` is not available

Generalized `clippy_is_available` and renamed as `command_is_available`.

No checks in `ignores_failure_to_format_source`, it's not supposed to
use `rustfmt` even if it's available

4 years agoAuto merge of #7865 - ehuss:fix-rebuild_sub_package_then_while_package, r=alexcrichton
bors [Wed, 5 Feb 2020 04:16:45 +0000 (04:16 +0000)]
Auto merge of #7865 - ehuss:fix-rebuild_sub_package_then_while_package, r=alexcrichton

Fix rebuild_sub_package_then_while_package on HFS.

This test was flaky on HFS ([azure failure](https://dev.azure.com/rust-lang/cargo/_build/results?buildId=20144&view=logs&j=a5e52b91-c83f-5429-4a68-c246fc63a4f7&t=d4864165-4be3-5e34-b483-a6b05303aa68&l=2018)), resulting in this error:

```
   Compiling foo v0.0.1 (/Users/runner/runners/2.164.7/work/1/s/target/cit/t750/foo)
error[E0460]: found possibly newer version of crate `b` which `a` depends on
 --> src/lib.rs:1:1
  |
1 | extern crate a; extern crate b; pub fn toplevel() {}
  | ^^^^^^^^^^^^^^^
  |
  = note: perhaps that crate needs to be recompiled?
  = note: the following crate versions were found:
          crate `b`: /Users/runner/runners/2.164.7/work/1/s/target/cit/t750/foo/target/debug/deps/libb-98160c67a5811c37.rlib
          crate `b`: /Users/runner/runners/2.164.7/work/1/s/target/cit/t750/foo/target/debug/deps/libb-98160c67a5811c37.rmeta
          crate `a`: /Users/runner/runners/2.164.7/work/1/s/target/cit/t750/foo/target/debug/deps/liba-7d2b9ccd932a36e9.rmeta
```

There are two race-condition bugs here.

Race 1: The second cargo build command (`cargo build -pb`) would sometimes not build, because it thought `b` is fresh.  This can happen when the first build finishes and changing `b/src/lib.rs` happen within the same second. (#5918)  The test silently ignored this failure, this is not the cause of the CI failure, though.

Race 2: The first and second build commands work as expected.  The third build command fails because it thinks both `a` and `b` are "fresh".  However, `b` was just rebuilt, and `a` depends on it, so `a` should have been rebuilt.  It thinks `a` is fresh because `a`'s mtime is equal to `b` when `b` finishes compiling within the same second that the first build finished.

The solution here is to make sure the second step happens well after the first.  The underlying problem is #5918.

4 years agoAuto merge of #7855 - ehuss:fix-required-features-renamed, r=alexcrichton
bors [Wed, 5 Feb 2020 02:31:09 +0000 (02:31 +0000)]
Auto merge of #7855 - ehuss:fix-required-features-renamed, r=alexcrichton

Fix required-features using renamed dependencies.

The dep_name/feat_name syntax in required-features should use the "name in toml" for dep_name, not the actual package name.  Otherwise, it is impossible to actually enable the feature (because the `--features` flag expects a "name in toml").

4 years agoFix rebuild_sub_package_then_while_package on HFS.
Eric Huss [Wed, 5 Feb 2020 01:42:09 +0000 (17:42 -0800)]
Fix rebuild_sub_package_then_while_package on HFS.

4 years agoAuto merge of #7860 - ehuss:fix-std-collision, r=alexcrichton
bors [Tue, 4 Feb 2020 22:23:31 +0000 (22:23 +0000)]
Auto merge of #7860 - ehuss:fix-std-collision, r=alexcrichton

Fix build-std collisions.

`build-std` unit filenames can collide with user dependencies in some situations.  In particular, `cc` as a build-dependency is likely to be exactly the same as a user's dep.  This would result in the `cc` crate being built twice, but with the same filename, causing a collision.

Other dependencies typically never collide because they have the `rustc-dep-of-std` feature, but build-dependencies do not.

The solution here is to include `is_std` in the metadata hash.

Fixes #7859.

4 years agoRemove likely brittle test.
Eric Huss [Tue, 4 Feb 2020 22:22:29 +0000 (14:22 -0800)]
Remove likely brittle test.

This is too dependent on the particulars of libstd.

4 years agoMerge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate
Kinrany [Tue, 4 Feb 2020 19:46:44 +0000 (22:46 +0300)]
Merge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate

4 years agoInstall rustfmt for testing in CI
Kinrany [Tue, 4 Feb 2020 19:45:46 +0000 (22:45 +0300)]
Install rustfmt for testing in CI

4 years agoFix build-std collisions.
Eric Huss [Mon, 3 Feb 2020 17:54:14 +0000 (09:54 -0800)]
Fix build-std collisions.

4 years agoFix BuildScriptOutput when a build script is run multiple times.
Eric Huss [Sat, 1 Feb 2020 19:38:05 +0000 (11:38 -0800)]
Fix BuildScriptOutput when a build script is run multiple times.

4 years agoFix required-features using renamed dependencies.
Eric Huss [Sun, 2 Feb 2020 17:59:15 +0000 (09:59 -0800)]
Fix required-features using renamed dependencies.

The dep_name/feat_name syntax in required-features should use the
"name in toml" for dep_name, not the actual package name.

4 years agoAuto merge of #7837 - ehuss:fix-global-opt-alias, r=alexcrichton
bors [Fri, 31 Jan 2020 21:56:58 +0000 (21:56 +0000)]
Auto merge of #7837 - ehuss:fix-global-opt-alias, r=alexcrichton

Fix using global options before an alias.

Options before an alias were being ignored (like `cargo -v b`).  The solution is to extract those global options before expanding an alias, and then merging it later.

An alternative to this is to try to avoid discarding the options during expansion, but I couldn't figure out a way to get the position of the subcommand in the argument list.  Clap only provides a way to get the arguments *following* the subcommand.

I also cleaned up some of the code in `Config::configure`, which was carrying some weird baggage from previous refactorings.

Fixes #7834

4 years agoFix using global options before an alias.
Eric Huss [Sun, 26 Jan 2020 23:02:37 +0000 (15:02 -0800)]
Fix using global options before an alias.

4 years agoAuto merge of #7851 - ehuss:version-bump, r=alexcrichton
bors [Fri, 31 Jan 2020 20:26:19 +0000 (20:26 +0000)]
Auto merge of #7851 - ehuss:version-bump, r=alexcrichton

Bump to 0.44.0, update changelog

4 years agoUpdate changelog for 1.42.
Eric Huss [Fri, 31 Jan 2020 20:05:32 +0000 (12:05 -0800)]
Update changelog for 1.42.

4 years agoBump to 0.44.0.
Eric Huss [Fri, 31 Jan 2020 20:05:14 +0000 (12:05 -0800)]
Bump to 0.44.0.

4 years agoAuto merge of #7823 - ehuss:stabilize-config-profile, r=alexcrichton
bors [Fri, 31 Jan 2020 11:50:31 +0000 (11:50 +0000)]
Auto merge of #7823 - ehuss:stabilize-config-profile, r=alexcrichton

Stabilize config-profile.

This is a proposal to stabilize config-profiles. This feature was proposed in [RFC 2282](https://github.com/rust-lang/rfcs/pull/2282) and implemented in #5506. Tracking issue is rust-lang/rust#48683.

This is intended to land in 1.43 which will reach the stable channel on April 23rd.

This is a fairly straightforward extension of profiles where the exact same syntax from `Cargo.toml` can be specified in a config file. Environment variables are supported for everything except the `package` override table, where we do not support the ability to read arbitrary keys in the environment name.

4 years agoMerge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate
Kinrany [Thu, 30 Jan 2020 22:40:01 +0000 (01:40 +0300)]
Merge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate

4 years agoLog rustfmt output if it fails; also do not check that rustfmt exists
Kinrany [Thu, 30 Jan 2020 22:36:30 +0000 (01:36 +0300)]
Log rustfmt output if it fails; also do not check that rustfmt exists

Worst case is that the logs will have multiple errors caused by missing
rustfmt, so checking separately that rustfmt exists is unnecessary.

4 years agoAuto merge of #7844 - spastorino:fix-infinite-loop, r=alexcrichton
bors [Thu, 30 Jan 2020 07:54:51 +0000 (07:54 +0000)]
Auto merge of #7844 - spastorino:fix-infinite-loop, r=alexcrichton

Swap std::sync::mpsc channel with crossbeam_channel

Hoping it closes #7840

r? @Mark-Simulacrum

4 years agoAuto merge of #7847 - rust-lang:dependabot/cargo/pretty_env_logger-0.4, r=alexcrichton
bors [Thu, 30 Jan 2020 06:51:41 +0000 (06:51 +0000)]
Auto merge of #7847 - rust-lang:dependabot/cargo/pretty_env_logger-0.4, r=alexcrichton

Update pretty_env_logger requirement from 0.3 to 0.4

Updates the requirements on [pretty_env_logger](https://github.com/seanmonstar/pretty-env-logger) to permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/fa4e28537f153a20e655349556ec982e3657a848"><code>fa4e285</code></a> v0.4.0</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/693b5e70880ada9fcca6c10766f5c55f656980e8"><code>693b5e7</code></a> Remove chrono dependency</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/28c5ad0cbbc206b661f1e35784fc2a2d2d11a9ed"><code>28c5ad0</code></a> env_logger: 0.6.2 -&gt; 0.7.0</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/0288e4ed4b730239c401b46b78b818fa3875171b"><code>0288e4e</code></a> fixes env goof in README.md</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/76d9fc760608eff42bbf7deb77ad2cab2cd1172e"><code>76d9fc7</code></a> Make env_logger dependency public</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/8ddffae2c556108640af9d4faf95d51f773f2623"><code>8ddffae</code></a> v0.3.1</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/04c1aa50e1f07fa2be33ea6866c05e39a073f015"><code>04c1aa5</code></a> require latest env_logger</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/ce8c2f12fb72f634f8467f2e5e4fc4a19edafa7c"><code>ce8c2f1</code></a> fix deprecated calls</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/67d2e7d68b3449229c4876bb0813bed50d65b207"><code>67d2e7d</code></a> timestamps with milliseconds</li>
<li><a href="https://github.com/seanmonstar/pretty-env-logger/commit/27eaa2bc1bb666b0888027c106d00cc0ab50d2bc"><code>27eaa2b</code></a> fix with_builder_1 example (<a href="https://github-redirect.dependabot.com/seanmonstar/pretty-env-logger/issues/25">#25</a>)</li>
<li>See full diff in <a href="https://github.com/seanmonstar/pretty-env-logger/compare/v0.3.0...v0.4.0">compare view</a></li>
</ul>
</details>
<br />

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)

</details>

4 years agoUpdate pretty_env_logger requirement from 0.3 to 0.4
dependabot-preview[bot] [Thu, 30 Jan 2020 05:12:20 +0000 (05:12 +0000)]
Update pretty_env_logger requirement from 0.3 to 0.4

Updates the requirements on [pretty_env_logger](https://github.com/seanmonstar/pretty-env-logger) to permit the latest version.
- [Release notes](https://github.com/seanmonstar/pretty-env-logger/releases)
- [Commits](https://github.com/seanmonstar/pretty-env-logger/compare/v0.3.0...v0.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
4 years agoSwap std::sync::mpsc channel with crossbeam_channel
Santiago Pastorino [Tue, 28 Jan 2020 15:35:08 +0000 (12:35 -0300)]
Swap std::sync::mpsc channel with crossbeam_channel

Switching this hoping it closes #7840

4 years agoFix tests on Linux/MacOS
Kinrany [Mon, 27 Jan 2020 21:46:56 +0000 (00:46 +0300)]
Fix tests on Linux/MacOS

The tests were failing with error:
"could not determine the current user, please set $USER"

4 years agoAuto merge of #7810 - giraffate:support_out_dir_in_build_section, r=alexcrichton
bors [Mon, 27 Jan 2020 19:41:32 +0000 (19:41 +0000)]
Auto merge of #7810 - giraffate:support_out_dir_in_build_section, r=alexcrichton

Support out-dir in build section of Cargo configuration file

Fixed #7555.

4 years agoAuto merge of #7768 - chrisduerr:install-workspaces-from-git, r=ehuss
bors [Mon, 27 Jan 2020 18:13:10 +0000 (18:13 +0000)]
Auto merge of #7768 - chrisduerr:install-workspaces-from-git, r=ehuss

Search for root manifest with ephemeral workspaces

Fixes #5495.

This seems like it's too simple to just work like this, but after trying a few different things, this was the only solution which worked reliably for me.

I've verified that no `/target` is present in the actual checkout location, the target directory used is actually the one created in `/tmp`.

I've also verified that both workspaces and "normal" packages still install through git and that a normal `cargo install --path` works too (though that doesn't use ephemeral workspaces anyways).

4 years agoFix typo.
Eric Huss [Mon, 27 Jan 2020 18:12:22 +0000 (10:12 -0800)]
Fix typo.

4 years agoMerge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate
Kinrany [Mon, 27 Jan 2020 16:43:03 +0000 (19:43 +0300)]
Merge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate

4 years agoAdd tests
Kinrany [Mon, 27 Jan 2020 16:42:20 +0000 (19:42 +0300)]
Add tests

Test 1: init a library crate with a `rustfmt.toml` config file in it.
Expect the generated source files to be formatted according to the
config file.

Test 2: same as test 1, but with missing `rustfmt`. Expect `cargo init`
to ignore the absence of `rustfmt` and generate source files
with default formatting.

4 years agoDeduplicate warnings about missing rustfmt
Kinrany [Sun, 26 Jan 2020 20:56:44 +0000 (23:56 +0300)]
Deduplicate warnings about missing rustfmt

There's now exactly one warning per new crate instead of one per
source file.

Also warnings are being logged now, not printed to console.

4 years agoAuto merge of #7836 - Mark-Simulacrum:clean-error, r=ehuss
bors [Sun, 26 Jan 2020 18:27:29 +0000 (18:27 +0000)]
Auto merge of #7836 - Mark-Simulacrum:clean-error, r=ehuss

Polish code to clarify meaning

This preserves the same behavior, but the previous code looked like it might
loop, but in fact never did.

4 years agoPolish code to clarify meaning
Mark Rousskov [Sun, 26 Jan 2020 17:16:01 +0000 (12:16 -0500)]
Polish code to clarify meaning

This preserves the same behavior, but the previous code looked like it might
loop, but in fact never did.

4 years agoAuto merge of #7829 - Mark-Simulacrum:fix-progress-panics, r=ehuss
bors [Sun, 26 Jan 2020 01:15:51 +0000 (01:15 +0000)]
Auto merge of #7829 - Mark-Simulacrum:fix-progress-panics, r=ehuss

Store maximum queue length

Previously, the queue length was constantly decreasing as we built crates, which
meant that we were incorrectly displaying the progress bar. In debug builds,
this even led to panics (due to underflow on subtraction).

Not sure if we can add a test case for this. I have made the panic unconditional on release/debug though by explicitly checking that current is less than the maximum for the progress bar.

Fixes https://github.com/rust-lang/cargo/pull/7731#issuecomment-578358824.

4 years agoStore maximum queue length
Mark Rousskov [Sat, 25 Jan 2020 19:20:40 +0000 (14:20 -0500)]
Store maximum queue length

Previously, the queue length was constantly decreasing as we built crates, which
meant that we were incorrectly displaying the progress bar. In debug builds,
this even led to panics (due to underflow on subtraction).

4 years agoMerge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate
Kinrany [Sat, 25 Jan 2020 03:19:59 +0000 (06:19 +0300)]
Merge branch 'master' into 7656-format-placeholder-code-when-generating-a-crate

4 years agoLog entry 2: first implementation
kinrany [Sat, 25 Jan 2020 02:41:22 +0000 (05:41 +0300)]
Log entry 2: first implementation

I think I found the correct place in the algorithm where rustfmt should
be called.

I'm not sure if using `std::process::Command` and converting errors into
warnings is the correct way here.

Also the warning should only be printed once. Unless
`config.shell().warn()` deduplicates warnings, right now there will be
one warning per source file.

So I should probably check that `rustfmt` exists only once and disable
formatting if it doesn't.

4 years agoRefactor code
Christian Duerr [Fri, 24 Jan 2020 21:08:32 +0000 (22:08 +0100)]
Refactor code

4 years agoAuto merge of #7826 - eddyb:recursion-limit-diagnostic, r=Eh2406
bors [Fri, 24 Jan 2020 18:26:23 +0000 (18:26 +0000)]
Auto merge of #7826 - eddyb:recursion-limit-diagnostic, r=Eh2406

test: allow some flexibility in check::error_from_deep_recursion's expected diagnostic.

This should unblock https://github.com/rust-lang/rust/pull/68407, by loosening the expected output pattern.

As per https://github.com/rust-lang/rust/pull/68407#issuecomment-578189644, this is the change in the diagnostic:
```diff
-recursion limit reached while expanding the macro `m`
+recursion limit reached while expanding `m!`
```

Ideally I would use something like this regex:
```
recursion limit reached while expanding (the macro `m`|`m!`)
```
but AFAIK these tests don't support regexes.

4 years agotest: allow some flexibility in check::error_from_deep_recursion's expected diagnostic.
Eduard-Mihai Burtescu [Fri, 24 Jan 2020 16:40:34 +0000 (18:40 +0200)]
test: allow some flexibility in check::error_from_deep_recursion's expected diagnostic.

4 years agoStabilize config-profile.
Eric Huss [Thu, 23 Jan 2020 20:05:02 +0000 (12:05 -0800)]
Stabilize config-profile.

4 years agoAuto merge of #7818 - giraffate:add_some_tests, r=alexcrichton
bors [Thu, 23 Jan 2020 15:15:49 +0000 (15:15 +0000)]
Auto merge of #7818 - giraffate:add_some_tests, r=alexcrichton

Add tests for `cargo owner -a/-r` / `cargo yank --undo`

Follow up 5e15286.

There were no tests for `cargo owner -a/-r` and `cargo yank --undo`. However, These commands in tests had empty response. So I also update response handling with reference to https://github.com/rust-lang/cargo/pull/3301/commits/7dd0f932a864d97bef5da26a0e148fca3f06d448.

4 years agoRevert updating a doc
Takayuki Nakata [Thu, 23 Jan 2020 00:13:13 +0000 (09:13 +0900)]
Revert updating a doc

4 years agoAuto merge of #7731 - Mark-Simulacrum:chatty-jobserver, r=alexcrichton
bors [Wed, 22 Jan 2020 22:42:20 +0000 (22:42 +0000)]
Auto merge of #7731 - Mark-Simulacrum:chatty-jobserver, r=alexcrichton

Scalable jobserver for rustc

This refactors the job queue code for support of [per-rustc process jobservers](https://github.com/rust-lang/rust/pull/67398). Furthermore, it also cleans up the code and refactors the main loop to be more amenable to understanding (splitting into methods and such).

Assignment of tokens to either rustc "threads" or processes is dedicated to the main loop, which proceeds in a strict "least recently requested" fashion among both thread and process token requests. Specifically, we will first allocate tokens to all pending process token requests (i.e., high-level units of work), and then (in per-rustc jobserver mode) follow up by assigning any remaining tokens to rustcs, again in the order that requests came into cargo (first request served first).

It's not quite clear that that model is good (no modeling or so has been done). On the other hand this strategy should mean that long-running crates will get more thread tokens once we bottom out in terms of rustc parallelism than short-running crates, which means that crates like syn which start early on but finish pretty late should hopefully get more parallelism nicely (without any more complex heuristics).

One plausible change that may be worth exploring is making the assignment prefer earlier rustc's, globally, rather than first attempting to spawn new crates and only then increasing parallelism for old crates. syn for example frequently gets compiled in the early storm of dozens of crates so is somewhat unlikely to have parallelism, until fairly late in its compilation.

We also currently conflate under this model the rayon threads and codegen threads. Eventually inside rustc those will probably(?) also be just one thing, and the rustc side of this implementation provides no information as to what the token request is for so we can't do better here yet.

4 years agoMove token truncation to just before waiting
Mark Rousskov [Wed, 22 Jan 2020 19:24:43 +0000 (14:24 -0500)]
Move token truncation to just before waiting

This also moves and enhances the message logging the state before blocking.

4 years agoStop threading Client through into the NeedsToken message
Mark Rousskov [Mon, 20 Jan 2020 18:03:05 +0000 (13:03 -0500)]
Stop threading Client through into the NeedsToken message

4 years agoElaborate on some documentation
Mark Rousskov [Mon, 20 Jan 2020 17:53:39 +0000 (12:53 -0500)]
Elaborate on some documentation

4 years agoRefactor to_send_clients to use a BTreeMap
Mark Rousskov [Mon, 20 Jan 2020 17:50:04 +0000 (12:50 -0500)]
Refactor to_send_clients to use a BTreeMap

This is both a performance optimization (avoiding O(n) shifting from the
beginning), and communicates intent in a nicer way overall.

It is plausible that we will eventually want to tie this data structure to
something like the DependencyQueue, i.e., to get more information on which rustc
to give tokens to. An old rustc with a very late dependency edge is less
important than one we'll need sooner, probably.

4 years agoUse an expect instead of directly panicking
Mark Rousskov [Mon, 20 Jan 2020 17:47:56 +0000 (12:47 -0500)]
Use an expect instead of directly panicking

4 years agoRefactor rustc thread granting loop
Mark Rousskov [Mon, 20 Jan 2020 17:39:18 +0000 (12:39 -0500)]
Refactor rustc thread granting loop

4 years agoReintroduce crossbeam threads
Mark Rousskov [Fri, 17 Jan 2020 01:16:36 +0000 (20:16 -0500)]
Reintroduce crossbeam threads

Turns out, these are actually necessary -- we will deadlock otherwise, though
it's not entirely obvious why.

4 years agoPop thread token requests from the front
Mark Rousskov [Thu, 16 Jan 2020 23:28:03 +0000 (18:28 -0500)]
Pop thread token requests from the front

This ensures we have a first come first served ordering for both thread and
process tokens.

4 years agoAdd some commentary
Mark Rousskov [Thu, 16 Jan 2020 23:25:02 +0000 (18:25 -0500)]
Add some commentary

4 years agoMove transient state to separate struct
Mark Rousskov [Thu, 16 Jan 2020 23:01:01 +0000 (18:01 -0500)]
Move transient state to separate struct

4 years agoTake JobQueue by-value in drain_the_queue
Mark Rousskov [Thu, 16 Jan 2020 22:44:50 +0000 (17:44 -0500)]
Take JobQueue by-value in drain_the_queue

4 years agoDocument ordering constraint on providing thread tokens
Mark Rousskov [Thu, 16 Jan 2020 02:02:37 +0000 (21:02 -0500)]
Document ordering constraint on providing thread tokens

4 years agoRefactor rustc_tokens to a HashMap
Mark Rousskov [Thu, 16 Jan 2020 01:45:43 +0000 (20:45 -0500)]
Refactor rustc_tokens to a HashMap

4 years agoDo not send acquired tokens to waiting rustc threads
Mark Rousskov [Thu, 16 Jan 2020 01:35:51 +0000 (20:35 -0500)]
Do not send acquired tokens to waiting rustc threads

This removes the ad-hoc token re-send in the message processing; this sort of
decision should be left up to the main loop which manages tokens.

Notably, the behavior change here is that new tokens will go solely to spawning
new rustc *processes* rather than increasing rustc internal parallelism, unless
we can't spawn new processes.

Otherwise, before this commit, we may be saturating a single rustc with tokens
rather than creating lots of rustcs that can work in parallel. In particular in
the beginning of a build, it's likely that this is worse (i.e., crates are small
and rustc internal parallelism is not at that point all that helpful) since it
severely limits the benefits of pipelining and generally makes the build
nearly serial.

4 years agoSplit waiting for an event out of the primary drainer
Mark Rousskov [Thu, 16 Jan 2020 01:24:05 +0000 (20:24 -0500)]
Split waiting for an event out of the primary drainer

This has the slight behavior change where we won't ask for new dependencies and
so forth if no events have been received but I believe that there's no activity
that can happen if an event hasn't occurred (i.e., no state change has occurred)
so there's no need for us to actually do anything in practice.

To make sure we still record CPU usage and such sufficiently often that is also
moved into the inner "waiting for events" loop.

4 years agoSplit out event handler into separate function
Mark Rousskov [Thu, 16 Jan 2020 01:18:41 +0000 (20:18 -0500)]
Split out event handler into separate function

4 years agoSplit out granting rustc token requests
Mark Rousskov [Thu, 16 Jan 2020 01:10:46 +0000 (20:10 -0500)]
Split out granting rustc token requests

4 years agoSplit out work spawning
Mark Rousskov [Thu, 16 Jan 2020 01:07:17 +0000 (20:07 -0500)]
Split out work spawning

4 years agoMove local variables to struct fields
Mark Rousskov [Thu, 16 Jan 2020 01:03:58 +0000 (20:03 -0500)]
Move local variables to struct fields

This will facilitate splitting drain_the_queue into methods

4 years agoDrop crossbeam scopes from job queue
Mark Rousskov [Wed, 15 Jan 2020 23:17:50 +0000 (18:17 -0500)]
Drop crossbeam scopes from job queue

These were unused for a long time (comment added in April 2019) and add
some complexity to API design.

4 years agoIntroduce newtype wrapping JobId
Mark Rousskov [Wed, 15 Jan 2020 22:22:25 +0000 (17:22 -0500)]
Introduce newtype wrapping JobId

4 years agoGate Cargo changes behind -Zjobserver-per-rustc
Mark Rousskov [Tue, 14 Jan 2020 22:13:48 +0000 (17:13 -0500)]
Gate Cargo changes behind -Zjobserver-per-rustc

4 years agoAdd documentation about the job queue and rustc
Mark Rousskov [Mon, 23 Dec 2019 19:45:33 +0000 (14:45 -0500)]
Add documentation about the job queue and rustc

4 years agoRecord the amount of rustc internal parallelism
Mark Rousskov [Sat, 21 Dec 2019 02:10:31 +0000 (21:10 -0500)]
Record the amount of rustc internal parallelism

4 years agoCommunicate jobserver information with each rustc
Mark Rousskov [Sat, 21 Dec 2019 01:11:10 +0000 (20:11 -0500)]
Communicate jobserver information with each rustc

4 years agoWire up methods for informing job queue of rustc jobserver state
Mark Rousskov [Thu, 19 Dec 2019 18:40:29 +0000 (13:40 -0500)]
Wire up methods for informing job queue of rustc jobserver state

4 years agoAuto merge of #7819 - ehuss:fix-replay-newlines, r=alexcrichton
bors [Wed, 22 Jan 2020 00:59:36 +0000 (00:59 +0000)]
Auto merge of #7819 - ehuss:fix-replay-newlines, r=alexcrichton

Fix cache replay including extra newlines.

The compiler output cache replay was changed in #7737 to use `BufReader::read_line` instead of `str::lines`. `read_line`, unlike `lines`, includes the trailing line ending. The code is written assuming that the line endings are stripped, so make sure they are stripped here, too.

This only happens for non-JSON messages, like `RUSTC_LOG`.

4 years agoRevert fixes of response handling and fix tests
Takayuki Nakata [Wed, 22 Jan 2020 00:19:59 +0000 (09:19 +0900)]
Revert fixes of response handling and fix tests

4 years agoFix cache replay including extra newlines.
Eric Huss [Tue, 21 Jan 2020 20:02:29 +0000 (12:02 -0800)]
Fix cache replay including extra newlines.

4 years agoLog entry 1
kinrany [Tue, 21 Jan 2020 20:12:22 +0000 (23:12 +0300)]
Log entry 1

Today I learned that `rustfmt` is now a part of `rustup`. So I need
to actually look it up in PATH instead of using `cargo-fmt`.

I also learned that `mk` is the file that does the main work and is
shared by `cargo-new` and `cargo-init`, as expected.

Printing inside `mk` produces output as expected.

4 years agoAuto merge of #7798 - jnbr:dylib_path, r=alexcrichton
bors [Tue, 21 Jan 2020 16:15:39 +0000 (16:15 +0000)]
Auto merge of #7798 - jnbr:dylib_path, r=alexcrichton

Fix wrong directories in host_libdir.

This fixes a regression from #7482 where the sysroot_target_libdir leaks into the host libdir. This can cause problems when the dynamic linker does not ignore the target libraries but tries to load them instead. This happens for example when building on x86_64-musl for aarch64-musl.

4 years agoUpdate response handling to add tests for `cargo yank --undo`
Takayuki Nakata [Tue, 21 Jan 2020 00:09:39 +0000 (09:09 +0900)]
Update response handling to add tests for `cargo yank --undo`

Follow up 5e152863f.

4 years agoUpdate response handling to add tests for `cargo owner -a/-r`
Takayuki Nakata [Mon, 20 Jan 2020 09:11:40 +0000 (18:11 +0900)]
Update response handling to add tests for `cargo owner -a/-r`

Follow up 5e152863f.

4 years agoAuto merge of #7815 - rust-lang:dependabot/cargo/humantime-2.0.0, r=Eh2406
bors [Mon, 20 Jan 2020 20:33:50 +0000 (20:33 +0000)]
Auto merge of #7815 - rust-lang:dependabot/cargo/humantime-2.0.0, r=Eh2406

Update humantime requirement from 1.2.0 to 2.0.0

Updates the requirements on [humantime](https://github.com/tailhook/humantime) to permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/tailhook/humantime/commit/d478f8a7878edba8a609a287776e27fc16d7009b"><code>d478f8a</code></a> Version bumped to v2.0.0</li>
<li><a href="https://github.com/tailhook/humantime/commit/49f11fdc2a59746085d2457cb46bce204dec746a"><code>49f11fd</code></a> Another improvement of the error message</li>
<li><a href="https://github.com/tailhook/humantime/commit/8a13b047ca0dc731e0c515b1db544fe1fd75bd7d"><code>8a13b04</code></a> Nicer error message for plain numeric duration</li>
<li><a href="https://github.com/tailhook/humantime/commit/edfa493e8cb8217aa5a0cc638398f60d67648c93"><code>edfa493</code></a> vagga.yaml: upgrade rust to 1.31.0</li>
<li><a href="https://github.com/tailhook/humantime/commit/8b8d748566c85a73b2f940e755dc0160c93f465a"><code>8b8d748</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/tailhook/humantime/issues/13">#13</a> from gh0st42/master</li>
<li><a href="https://github.com/tailhook/humantime/commit/da7723529fa30222b73561fad0ed7ce47d9a15a4"><code>da77235</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/tailhook/humantime/issues/12">#12</a> from koushiro/works</li>
<li><a href="https://github.com/tailhook/humantime/commit/227d7e591dd76bbfbca6a236d2e8661ddfd16d63"><code>227d7e5</code></a> Made crate unsafe free and forbid unsafe</li>
<li><a href="https://github.com/tailhook/humantime/commit/b7da4ab6ad24f68660cf92ede8dc88d7400a39fa"><code>b7da4ab</code></a> Downgrade MSRV to 1.31</li>
<li><a href="https://github.com/tailhook/humantime/commit/4a406825951039c6c3414b76c855dc38384d874d"><code>4a40682</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/tailhook/humantime/issues/11">#11</a> from koushiro/apply-rustfmt-and-clippy</li>
<li><a href="https://github.com/tailhook/humantime/commit/f00dbbae37ac281c919fbf0d56caf5900f1e0085"><code>f00dbba</code></a> Apply suggestions</li>
<li>Additional commits viewable in <a href="https://github.com/tailhook/humantime/compare/v1.2.0...v2.0.0">compare view</a></li>
</ul>
</details>
<br />

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)

</details>

4 years agoAuto merge of #7817 - ehuss:fix-doc-target-test, r=Eh2406
bors [Mon, 20 Jan 2020 20:12:59 +0000 (20:12 +0000)]
Auto merge of #7817 - ehuss:fix-doc-target-test, r=Eh2406

Fix doc_target test which no longer works on stable/beta.

https://github.com/rust-lang/rust/pull/67989 changed it so that `#![feature]` requires nightly.

4 years agoFix doc_target test which no longer works on stable/beta.
Eric Huss [Mon, 20 Jan 2020 19:16:53 +0000 (11:16 -0800)]
Fix doc_target test which no longer works on stable/beta.

4 years agoUpdate humantime requirement from 1.2.0 to 2.0.0
dependabot-preview[bot] [Mon, 20 Jan 2020 05:27:58 +0000 (05:27 +0000)]
Update humantime requirement from 1.2.0 to 2.0.0

Updates the requirements on [humantime](https://github.com/tailhook/humantime) to permit the latest version.
- [Release notes](https://github.com/tailhook/humantime/releases)
- [Commits](https://github.com/tailhook/humantime/compare/v1.2.0...v2.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
4 years agoFormat code
Christian Duerr [Mon, 20 Jan 2020 00:17:21 +0000 (01:17 +0100)]
Format code

4 years agoRemove tempdir after install
Christian Duerr [Mon, 20 Jan 2020 00:15:02 +0000 (01:15 +0100)]
Remove tempdir after install

4 years agoKeep existing package with git install
Christian Duerr [Sun, 19 Jan 2020 23:40:56 +0000 (00:40 +0100)]
Keep existing package with git install

4 years agoUse non-ephemeral workspace
Christian Duerr [Tue, 14 Jan 2020 00:05:30 +0000 (01:05 +0100)]
Use non-ephemeral workspace

4 years agoTest that git install reads virtual manifest
Christian Duerr [Mon, 6 Jan 2020 21:36:33 +0000 (22:36 +0100)]
Test that git install reads virtual manifest

4 years agoFix failing test
Christian Duerr [Mon, 6 Jan 2020 21:02:26 +0000 (22:02 +0100)]
Fix failing test

4 years agoSearch for root manifest with ephemeral workspaces
Christian Duerr [Mon, 6 Jan 2020 16:04:16 +0000 (17:04 +0100)]
Search for root manifest with ephemeral workspaces

Fixes #5495.

4 years agoAuto merge of #7814 - ehuss:fix-em-dash, r=Eh2406
bors [Sat, 18 Jan 2020 20:25:48 +0000 (20:25 +0000)]
Auto merge of #7814 - ehuss:fix-em-dash, r=Eh2406

Fix some erroneous em-dashes in man pages.

Unfortunately asciidoctor has a built-in substitution pass that cannot be disabled.

Fixes #7807.

4 years agoAuto merge of #7808 - matthiaskrgr:clippy_v10, r=ehuss
bors [Sat, 18 Jan 2020 19:14:12 +0000 (19:14 +0000)]
Auto merge of #7808 - matthiaskrgr:clippy_v10, r=ehuss

fix some clippy warnings

4 years agoFix some erroneous em-dashes in man pages.
Eric Huss [Sat, 18 Jan 2020 19:02:18 +0000 (11:02 -0800)]
Fix some erroneous em-dashes in man pages.

4 years agoSupport out-dir in build section of Cargo configuration file
Takayuki Nakata [Fri, 17 Jan 2020 13:26:58 +0000 (22:26 +0900)]
Support out-dir in build section of Cargo configuration file

Fixed #7555.

4 years agofix some clippy warnings
Matthias Krüger [Fri, 17 Jan 2020 11:19:12 +0000 (12:19 +0100)]
fix some clippy warnings