Auto merge of #8500 - alexcrichton:build-override-opt-level-0, r=Eh2406
Build host dependencies with opt-level 0 by default
This commit updates Cargo's build of host dependencies to build them
with optimization level 0 by default instead of matching the profile of
the final binary.
Since Cargo's inception build dependencies have, by default, been built
in a profile that largely matches the profile of the final target
artifact. Build dependencies, however, rarely actually need to be
optimized and are often executing very small tasks, which means that
optimizing them often wastes a lot of build time. A great example of
this is procedural macros where `syn` and friends are pretty heavyweight
to optimize, and the amount of Rust code they're parsing is typically
quite small, so the time spent optimizing rarely comes as a benefit.
The goal of this PR is to improve build times on average in the
community by not spending time optimizing build dependencies (build
scripts, procedural macros, and their transitive dependencies). The PR
will not be a universal win for everyone, however. There's some
situations where your build time may actually increase:
* In some cases build scripts and procedural macros can take quite a
long time to run!
* Cargo may not build dependencies more than once if they're shared with
the main build. This only applies to builds without `--target` where
the same crate is used in the final binary as in a build script.
In these cases, however, the `build-override` profile has existed for
some time know and allows giving a knob to tweak this behavior. For
example to get back the previous build behavior of Cargo you would
specify, in `Cargo.toml`:
There are two notable features we would like to add in the future which
would make the impact of a change like this smaller, but they're not
implemented at this time (nor do we have concrete plans to implement
them). First we would like crates to have a way of specifying they
should be optimized by default, despite default profile options. Often
crates, like lalrpop historically, have abysmal performance in debug
mode and almost always (even in debug builds) want to be built in
release mode. The second feature is that ideally crate authors would be
able to tell Cargo to minimize the number of crates built, unifying
profiles where possible to avoid double-compiling crates.
At this time though the Cargo team feels that the benefit of changing
the defaults is well worth this change. Neither today nor directly after
this change will be a perfect world, but it's hoped that this change
makes things less bad!
Alex Crichton [Fri, 17 Jul 2020 19:39:41 +0000 (12:39 -0700)]
Build host dependencies with opt-level 0 by default
This commit updates Cargo's build of host dependencies to build them
with optimization level 0 by default instead of matching the profile of
the final binary.
Since Cargo's inception build dependencies have, by default, been built
in a profile that largely matches the profile of the final target
artifact. Build dependencies, however, rarely actually need to be
optimized and are often executing very small tasks, which means that
optimizing them often wastes a lot of build time. A great example of
this is procedural macros where `syn` and friends are pretty heavyweight
to optimize, and the amount of Rust code they're parsing is typically
quite small, so the time spent optimizing rarely comes as a benefit.
The goal of this PR is to improve build times on average in the
community by not spending time optimizing build dependencies (build
scripts, procedural macros, and their transitive dependencies). The PR
will not be a universal win for everyone, however. There's some
situations where your build time may actually increase:
* In some cases build scripts and procedural macros can take quite a
long time to run!
* Cargo may not build dependencies more than once if they're shared with
the main build. This only applies to builds without `--target` where
the same crate is used in the final binary as in a build script.
In these cases, however, the `build-override` profile has existed for
some time know and allows giving a knob to tweak this behavior. For
example to get back the previous build behavior of Cargo you would
specify, in `Cargo.toml`:
There are two notable features we would like to add in the future which
would make the impact of a change like this smaller, but they're not
implemented at this time (nor do we have concrete plans to implement
them). First we would like crates to have a way of specifying they
should be optimized by default, despite default profile options. Often
crates, like lalrpop historically, have abysmal performance in debug
mode and almost always (even in debug builds) want to be built in
release mode. The second feature is that ideally crate authors would be
able to tell Cargo to minimize the number of crates built, unifying
profiles where possible to avoid double-compiling crates.
At this time though the Cargo team feels that the benefit of changing
the defaults is well worth this change. Neither today nor directly after
this change will be a perfect world, but it's hoped that this change
makes things less bad!
Auto merge of #8497 - alexcrichton:fix-rebuild-on-rename, r=ehuss
Fix freshness checks for build scripts on renamed dirs
This commit fixes an issue in Cargo where when an entire project
directory is renamed (preserving the target directory) then path
dependencies with build scripts would have their build scripts rereun
when building again. The problem with this was that when a build script
doesn't print `rerun-if-changed` Cargo's conservative fingerprint listed
an absolute path in it, which was intended to be a relative path.
The fix here is to use a relative path in the fingerprint to ensure that
it's not the reason a rebuild happens when directories are renamed.
Alex Crichton [Fri, 17 Jul 2020 17:07:06 +0000 (10:07 -0700)]
Fix freshness checks for build scripts on renamed dirs
This commit fixes an issue in Cargo where when an entire project
directory is renamed (preserving the target directory) then path
dependencies with build scripts would have their build scripts rereun
when building again. The problem with this was that when a build script
doesn't print `rerun-if-changed` Cargo's conservative fingerprint listed
an absolute path in it, which was intended to be a relative path.
The fix here is to use a relative path in the fingerprint to ensure that
it's not the reason a rebuild happens when directories are renamed.
Auto merge of #8490 - alexcrichton:std-customize-features, r=ehuss
Add a `-Zbuild-std-features` flag
This flag is intended to pair with `-Zbuild-std` as necessary to
configure the features that libstd is built with. This is highly
unlikely to ever be stabilized in any form (unlike `-Zbuild-std` which
we'd like to stabilize at some point), but can be useful for
experimenting with the standard library. For example today it can be
used to test changes to binary size by disabling backtraces.
My intention is that we won't need a `--no-default-features` equivalent
for libstd, where after rust-lang/rust#74377 is merged we can
unconditionally specify default features are disabled but the default
set of features lists `default`. That way if users want to override the
list *and* include the default feature, they can just be sure to include
`default`.
Alex Crichton [Wed, 15 Jul 2020 18:44:41 +0000 (11:44 -0700)]
Add a `-Zbuild-std-features` flag
This flag is intended to pair with `-Zbuild-std` as necessary to
configure the features that libstd is built with. This is highly
unlikely to ever be stabilized in any form (unlike `-Zbuild-std` which
we'd like to stabilize at some point), but can be useful for
experimenting with the standard library. For example today it can be
used to test changes to binary size by disabling backtraces.
My intention is that we won't need a `--no-default-features` equivalent
for libstd, where after rust-lang/rust#74377 is merged we can
unconditionally specify default features are disabled but the default
set of features lists `default`. That way if users want to override the
list *and* include the default feature, they can just be sure to include
`default`.
Auto merge of #8495 - matthiaskrgr:clippy_v15, r=Eh2406
clippy cleanups
Fixes a couple of clippy warnings.
Ignores clippy::collapsible_if warnings in the future (iirc there were not desired)
clippy::redundant_clone is enabled by default by clippy already.
Arlo Siemsen [Wed, 15 Jul 2020 17:43:34 +0000 (10:43 -0700)]
Make `cargo metadata` output deterministic
Uses BTreeMap instead of HashMap for the `cargo metadata` command.
The change did not cause a measurable performance impact for
running `cargo metadata` on `cargo` itself.
This commit switches our CI from Azure Pipelines to GitHub Actions. The intention here is to follow the "idiomatic" provider of CI for rust-lang, and otherwise GitHub Actions is better integrated with GitHub's UI right now too.
I'll need to tweak bors to actually `@bors: r+` this to have it successfully get merged, but I think it'd be good to get some review first.
Auto merge of #8393 - ludumipsum:unstable_flags_in_config, r=alexcrichton
Allow configuring unstable flags via config file
# Summary
This fixes #8127 by mapping the `unstable` key in `.cargo/config` to Z flags.
It should have no impact on stable/beta cargo, and on nightlies it gives folks the ability to configure Z flags for an entire project or workspace. This is meant to make it easier to try things like the [new features resolver](https://github.com/rust-lang/cargo/issues/8088) behavior, mtime-on-use, build-std, or timing reports across a whole project.
I've also included a small (but entirely independent) ergonomics change -- treating dashes and underscores identically in Z flags. That's along for the ride in this PR as the last commit, and is included here because it makes for more idiomatic toml file keys (`print_im_a_teapot = yes` vs `print-im-a-teapot = yes`). Please let me know if y'all would prefer that be in a separate PR, or not happen at all.
# Test Plan
Apologies if I've missed anything -- this is my first cargo contrib and I've tried to hew to the contributing guide. If I've slipped up, please let me know how and I'll fix it.
NB. My linux machine doesn't have multilib set up, so I disabled cross tests.
* `cargo test` passes for each commit in the stack
* I formatted each commit in the stack with `rustfmt`
* New tests are included alongside the relevant change for each change
* I've validated each test by locally undoing the code change they support and confirming failure.
* The CLI wins, for both enable and disabling Z flags, as you'd expect.
Keys in `unstable` which do not correspond with a Z flag will trigger an error indicating the invalid flag came from a config file read:
```
Invalid [unstable] entry in Cargo config
Caused by:
unknown `-Z` flag specified: an-invalid-flag
```
If you'd like to see a test case which isn't represented here, I'm happy to add it. Just let me know.
# Documentation
I've included commits in this stack updating the only docs page that seemed relevant to me, skimming the book -- `unstable.md`.
Alex Berghage [Thu, 9 Jul 2020 18:53:42 +0000 (12:53 -0600)]
Add tests verifying overlapping prefixes and defaults
These tests demonstrate the current failure mode around
overlapping env keys and inner structs. To some extent this
is a limitation of mapping on to environment variables with
an underscore as both the namespace separator and the
substitute for dashes in flag names in that the mapping is
not strictly one-to-one.
Auto merge of #8427 - davidtwco:terminal-width, r=ehuss
Add support for rustc's `-Z terminal-width`.
This PR continues the work started in #7315, adding support for rustc's `-Z terminal-width` flag, which is used to trim diagnostic output to fit within the current terminal and was added in rust-lang/rust#63402 (with JSON emitter support in rust-lang/rust#73763).
At the time of writing, rust-lang/rust#73763 isn't in nightly, so the test added in this PR will fail, but it should pass tomorrow (I've confirmed that it works with a local rustc build).
Auto merge of #8473 - alexcrichton:fix-fingerprint-loc, r=ehuss
Avoid colliding with older Cargo fingerprint changes
The fingerprint format Cargo stores changed recently in a way that
older Cargos cannot understand. Unfortunately though older Cargos are
colliding on some compilation units trying to read the new format and
they're bailing out. This commit fixes this issue by ensuring that the
location for fingerprint metadata is different in older Cargos and newer
Cargos.
Fingerprint metadata is always stored in a location with a hash in the
file name. This hash typically includes the hash of rustc's version
information itself, but for units which don't have a `Metadata` it's a
much simpler hash which is much more likely to collide with other
versions of Cargo. The fix in this commit is to extract the metadata
version that we're hashing to a constant, and then also hash it for
generating a filesystem location to house fingerprint data for a unit
that has no `Metadata`.
Alex Crichton [Thu, 9 Jul 2020 14:48:18 +0000 (07:48 -0700)]
Avoid colliding with older Cargo fingerprint changes
The fingerprint format Cargo stores changed recently in a way that
older Cargos cannot understand. Unfortunately though older Cargos are
colliding on some compilation units trying to read the new format and
they're bailing out. This commit fixes this issue by ensuring that the
location for fingerprint metadata is different in older Cargos and newer
Cargos.
Fingerprint metadata is always stored in a location with a hash in the
file name. This hash typically includes the hash of rustc's version
information itself, but for units which don't have a `Metadata` it's a
much simpler hash which is much more likely to collide with other
versions of Cargo. The fix in this commit is to extract the metadata
version that we're hashing to a constant, and then also hash it for
generating a filesystem location to house fingerprint data for a unit
that has no `Metadata`.
Alex Berghage [Thu, 9 Jul 2020 03:14:47 +0000 (21:14 -0600)]
Revert workaround and patch Deserializer
This patch changes how ConfigMapAccess iterates k/v pairs when
deserializing structs.
Previously we produced keys for exactly the set of fields needed
for a struct, and errored out in the deserializer if we can't find
anything for that field.
This patch makes us produces keys from the union of two sets:
1. All fields that are both needed for the struct and can be found
in the environment.
2. All fields in the config table.
This change allows serde's codegen to handle both missing and
unknown fields via the usual derive annotations (default or
deny_unknown_fields respectively)
David Wood [Tue, 30 Jun 2020 19:29:09 +0000 (20:29 +0100)]
Make `-Z terminal-width` opt-in.
This commit modifies the parsing of `-Z terminal-width` so that it can
optionally take a value and only uses `accurate_err_width` when the user
does not provide a value - therefore making the emission of `-Z
terminal-width` opt-in.
Alex Berghage [Wed, 8 Jul 2020 05:35:22 +0000 (23:35 -0600)]
Drop broken test asserting invalid configs error on nightly
The current behavior, with the default-false workaround in place,
is not able to identify extra members of the `unstable` table, so
it can't error on unexpected members.
I'll add this test back in with a Deserializer refactor to clean up
the extra logic added to CliUnstable.
Tests are currently failing, looks like there's something in the
Deserializer impl that's forcing field-missing errors even when
the serde `default` annotation is applied.
Alex Berghage [Sun, 21 Jun 2020 20:19:37 +0000 (14:19 -0600)]
Allow setting unstable options in .cargo/config
Obviously this only works with nightlies and all
that, but if you're e.g. testing resolver v2, want
to always have Ztiming on in your workspace, or
something like that, this makes it easier to do.
Auto merge of #8453 - sourcefrog:name-length-limit-8452, r=alexcrichton
Write GNU tar files, supporting long names.
Fixes #8452
If I understand the previous bugs correctly, long trees packaged using Cargo with this patch will be misinterpreted by Cargo from before Jan 2016. (Without this patch, they can't be written at all.)
To me that seems like long enough ago that it's safe to land this now.
Auto merge of #8446 - ehuss:fix-usize-32, r=alexcrichton
Fix overflow error on 32-bit.
This fails to compile on 32-bit platforms with an overflow error ("attempt to shift right by 32_i32 which would overflow").
I think it would be highly unlikely for any value to be in the billions. Alternatively it can be rewritten to something like `assert!(val <= u32::MAX as usize);`.
Exclude the target directory from backups using CACHEDIR.TAG
This patch follows the lead of #4386 (which excludes target directories
from Time Machine backups) and is motived by the same reasons listen
in #3884. CACHEDIR.TAG is an OS-independent mechanism supported by Borg,
restic, GNU Tar and other backup/archiving solutions.
See https://bford.info/cachedir/ for more information about the
specification. This has been discussed in Rust Internals earlier this
year[1] and it seems like it's an uncontroversial improvement so I went
ahead with the patch.
One thing I'm wondering is whether this should maybe cover the whole main target directory (right now it applies to `target/debug`, `target/release` etc. but not to target root).
Jakub Stasiak [Wed, 1 Jul 2020 20:40:31 +0000 (22:40 +0200)]
Exclude whole target/ from backups
This is following the discussion on GitHub. The doc tests are no longer
necessary because Layout::new() creates CACHEDIR.TAG directly in target
root, no doc-specific code is necessary anymore.
bors [Tue, 30 Jun 2020 14:16:08 +0000 (14:16 +0000)]
Auto merge of #8432 - rust-lang:dependabot/cargo/core-foundation-0.9.0, r=alexcrichton
Update core-foundation requirement from 0.7.0 to 0.9.0
Updates the requirements on [core-foundation](https://github.com/servo/core-foundation-rs) to permit the latest version.
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/servo/core-foundation-rs/commit/a57ca7f976858f56ab159656703a2857fa947870"><code>a57ca7f</code></a> Major version update for core-foundation-sys and all other dependencies.</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/aa69b91def2137addf61f27fb94077d8318dad4f"><code>aa69b91</code></a> Add missing major version changes.</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/7487580958a60d5a51228bdc211a4f53dfe4b4ca"><code>7487580</code></a> Publish core-foundation-sys 0.7.2.</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/40a787f0e6a2f7b3874f4f1b3a6b9fc5a3a165ff"><code>40a787f</code></a> Auto merge of <a href="https://github-redirect.dependabot.com/servo/core-foundation-rs/issues/397">#397</a> - servo:major-bump, r=jdm</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/56d8b6c161ad3db0adc59853951163658b3e97b1"><code>56d8b6c</code></a> Major version change for core-foundation, core-graphics, io-surface, core-tex...</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/37540073110db955230c67f87e50fb55a9a4447c"><code>3754007</code></a> Auto merge of <a href="https://github-redirect.dependabot.com/servo/core-foundation-rs/issues/395">#395</a> - bryanburgers:timezone-name, r=jdm</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/a4e29832190de6d58bd59ccd52289476689cd995"><code>a4e2983</code></a> Add <code>CFTimeZone::name</code></li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/749c0855ad2764ffcc4fef6fed40670f421ae186"><code>749c085</code></a> Auto merge of <a href="https://github-redirect.dependabot.com/servo/core-foundation-rs/issues/394">#394</a> - anp:releases, r=jdm</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/9c264d290423c10d0801d2acdb66c17ec9f8755c"><code>9c264d2</code></a> Bump core-foundation(-sys) to 0.7.1 for release.</li>
<li><a href="https://github.com/servo/core-foundation-rs/commit/d196c0c4c8641fadcdb4553b77e3491f76a136ff"><code>d196c0c</code></a> Publish core-graphics 0.19.2.</li>
<li>Additional commits viewable in <a href="https://github.com/servo/core-foundation-rs/compare/core-foundation-v0.7.0...core-foundation-v0.9.0">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
Update core-foundation requirement from 0.7.0 to 0.9.0
Updates the requirements on [core-foundation](https://github.com/servo/core-foundation-rs) to permit the latest version.
- [Release notes](https://github.com/servo/core-foundation-rs/releases)
- [Commits](https://github.com/servo/core-foundation-rs/compare/core-foundation-v0.7.0...core-foundation-v0.9.0)
bors [Tue, 30 Jun 2020 02:40:18 +0000 (02:40 +0000)]
Auto merge of #8421 - alexcrichton:read-env-dep, r=ehuss
Parse `# env-dep` directives in dep-info files
This commit updates Cargo's parsing of rustc's dep-info files to account
for changes made upstream in rust-lang/rust#71858. This means that if
`env!` or `option_env!` is used in crate files Cargo will correctly
rebuild the crate if the env var changes.
Alex Crichton [Fri, 26 Jun 2020 19:30:42 +0000 (12:30 -0700)]
Parse `# env-dep` directives in dep-info files
This commit updates Cargo's parsing of rustc's dep-info files to account
for changes made upstream in rust-lang/rust#71858. This means that if
`env!` or `option_env!` is used in crate files Cargo will correctly
rebuild the crate if the env var changes.
bors [Thu, 25 Jun 2020 18:47:08 +0000 (18:47 +0000)]
Auto merge of #8409 - alexcrichton:git-instead-of, r=Eh2406
Improve git error messages a bit
This commit is targeted at further improving the error messages
generated from git errors. For authentication errors the actual URL
fetched is now printed out as well if it's different from the original
URL. This should help handle `insteadOf` logic where SSH urls are used
instead of HTTPS urls and users can know to track that down.
Otherwise the logic about recommending `net.git-fetch-with-cli` was
tweaked a bit and moved to the same location as the rest of our error
reporting.
Note that a change piggy-backed here as well is that `Caused by:` errors
are now automatically all tabbed over a bit instead of only having the
first line tabbed over. This required a good number of tests to be
updated, but it's just an updated in renderings.
Alex Crichton [Thu, 25 Jun 2020 15:25:52 +0000 (08:25 -0700)]
Improve git error messages a bit
This commit is targeted at further improving the error messages
generated from git errors. For authentication errors the actual URL
fetched is now printed out as well if it's different from the original
URL. This should help handle `insteadOf` logic where SSH urls are used
instead of HTTPS urls and users can know to track that down.
Otherwise the logic about recommending `net.git-fetch-with-cli` was
tweaked a bit and moved to the same location as the rest of our error
reporting.
Note that a change piggy-backed here as well is that `Caused by:` errors
are now automatically all tabbed over a bit instead of only having the
first line tabbed over. This required a good number of tests to be
updated, but it's just an updated in renderings.