bors [Thu, 6 Nov 2014 00:02:30 +0000 (00:02 +0000)]
auto merge of #799 : alexcrichton/cargo/moar-parallel, r=brson
When building unit tests for a library, we don't need the library itself to be
built beforehand, so the two can be built in parallel. If the crate takes awhile
to compile, this shows some excellent wall-time speedups.
bors [Wed, 5 Nov 2014 23:02:41 +0000 (23:02 +0000)]
auto merge of #802 : alexcrichton/cargo/issue-801, r=brson
Previously all features were traversed when adding information to the lockfile,
but the traversal forgot to add optional dependencies that did not have a
corresponding feature.
bors [Wed, 5 Nov 2014 22:35:35 +0000 (22:35 +0000)]
auto merge of #803 : alexcrichton/cargo/fix-some-tests, r=alexcrichton
Right now we're accidentally not actually leveraging this function well in some
tests due to this sequence of events:
1. The tests run at time X, building artifacts that remember the fingerprint is
at time X.
2. The entire project is moved back one hour to (X-1).
3. A new file is created, at time X (second-level resolution on some systems).
4. On a rebuild, the maximum mtime is still X (due to the new file).
For this reason there are some more calls to move_into_the_past() to push files
back another hour after they've been created to make sure the maximum mtime is
(X-1), or something different than X.
Alex Crichton [Wed, 5 Nov 2014 22:00:38 +0000 (14:00 -0800)]
More aggresively move_into_the_past for tests
Right now we're accidentally not actually leveraging this function well in some
tests due to this sequence of events:
1. The tests run at time X, building artifacts that remember the fingerprint is
at time X.
2. The entire project is moved back one hour to (X-1).
3. A new file is created, at time X (second-level resolution on some systems).
4. On a rebuild, the maximum mtime is still X (due to the new file).
For this reason there are some more calls to move_into_the_past() to push files
back another hour after they've been created to make sure the maximum mtime is
(X-1), or something different than X.
Alex Crichton [Wed, 5 Nov 2014 03:40:06 +0000 (19:40 -0800)]
Build lib tests and libraries in parallel
When building unit tests for a library, we don't need the library itself to be
built beforehand, so the two can be built in parallel. If the crate takes awhile
to compile, this shows some excellent wall-time speedups.
bors [Wed, 5 Nov 2014 21:13:08 +0000 (21:13 +0000)]
auto merge of #792 : alexcrichton/cargo/build-cmd, r=brson
This series of commits (based on https://github.com/rust-lang/cargo/pull/763) is an implementation of the recent [Cargo RFC](https://github.com/rust-lang/rfcs/blob/master/text/0403-cargo-build-command.md). This should implement all portions of the RFC, but there's a lot so an extra set of eyes would be nice!
I haven't added documentation for it all yet, but I would like to do so before landing (starting with https://github.com/rust-lang/cargo/pull/749). Otherwise I've been migrating all of the existing cargo dependencies away from the build command to a build script, and the progress can be seen with these repositories:
Alex Crichton [Wed, 5 Nov 2014 19:13:04 +0000 (11:13 -0800)]
Be sure to lock purely optional dependencies
Previously all features were traversed when adding information to the lockfile,
but the traversal forgot to add optional dependencies that did not have a
corresponding feature.
Alex Crichton [Fri, 31 Oct 2014 22:51:13 +0000 (15:51 -0700)]
Refine the dependency graph for build scripts
Build scripts can immediately start building as soon as all build dependencies
are available and not need to wait for normal dependencies. This commit also
includes a number of refactorings and reorganizations to tidy up how build
scripts are processed.
One primary piece of state introduced in this commit is a shared Arc<Mutex<T>>
which contains information about the processed build scripts as compilation
continues. Compilation commands will draw information from this state and build
scripts will feed information back into this state to ensure it's up to date.
Alex Crichton [Fri, 31 Oct 2014 17:49:04 +0000 (10:49 -0700)]
Implement the `links` manifest key
This commit adds the `links` manifest key as a string of a C library which is
being linked to. This is passed as an argument to the build command when not
overridden. The implementation of overrides will come soon!
bors [Fri, 31 Oct 2014 20:44:47 +0000 (20:44 +0000)]
auto merge of #773 : alexcrichton/cargo/update-all-packages-from-a-git-repo, r=brson
With the recent resolve rewrite, `cargo update -p foo` would only update one
package in a git repository, even if the repository provided many packages.
Cargo does not currently support depending on the same git repository source
with different precise revisions, and this would cause errors down the line
depending on what happened.
This adds a fix to the `resolve_with_previous` method to ensure that any
non-registry sources being updated will not have any locked packages inside them
which would result in an invalid lockfile.
Alex Crichton [Wed, 29 Oct 2014 18:42:45 +0000 (11:42 -0700)]
Be sure to update all packages from a git source
With the recent resolve rewrite, `cargo update -p foo` would only update one
package in a git repository, even if the repository provided many packages.
Cargo does not currently support depending on the same git repository source
with different precise revisions, and this would cause errors down the line
depending on what happened.
This adds a fix to the `resolve_with_previous` method to ensure that any
non-registry sources being updated will not have any locked packages inside them
which would result in an invalid lockfile.
Alex Crichton [Tue, 28 Oct 2014 15:32:54 +0000 (08:32 -0700)]
Update curl-rust to handle SSL config for linuxes
Right now the cargo built on the snapshot bot isn't able to talk to the registry
due to failing to validate the SSL certificate. This is likely due to the
certificate path being hardcoded to something that's compatible with CentOS
(which isn't what I'm running locally).
I've modified curl-rust to use openssl-static-sys to probe the system for where
certificates are located and inform handles by default about the found
locations. This is the same strategy that git2-rs uses to inform openssl about
where the certificates are located.
Tomas Sedovic [Mon, 27 Oct 2014 21:49:20 +0000 (22:49 +0100)]
Add `--name` and `--example` to cargo run
This lets us compile and run examples using `cargo run --example NAME`.
Selecting the other binary targets is now done using the `--name` flag.
`cargo run` falls back to the old behaviour (running the only bin target
in the project, failing if there are more) in neither `--name` nor
`--example` are present.
bors [Mon, 27 Oct 2014 23:49:37 +0000 (23:49 +0000)]
auto merge of #767 : alexcrichton/cargo/issue-637, r=wycats
Each dependency itself already has the precise source listed, and we're
guaranteed that for each (source, name) pair that there is exactly one
dependency, so there is always a way to rebuild the precise dependency graph
after the fact by looking up the (source, name) pair in the hash map.
This should help with some of the readability concerns in #637 because the git
SHA that a source is locked to is now only mentioned once in a lockfile.
This commit does preserve, however, the mention of the version in each
dependency line which will likely never go away. This means that for
registry-based packages will still run into the same lockfile merge conflict
troubles, there will just be more readable versions than git hashes.
It should also be noted that this will alter all currently generated lockfiles
as any dependencies mentioned will lose the hashes mentioned afterwards. This
will likely cause somewhat of a transitionary pain as this version of cargo
propagates throughout.
Alex Crichton [Mon, 27 Oct 2014 18:54:48 +0000 (11:54 -0700)]
Don't encode precise in lockfile dep pointers
Each dependency itself already has the precise source listed, and we're
guaranteed that for each (source, name) pair that there is exactly one
dependency, so there is always a way to rebuild the precise dependency graph
after the fact by looking up the (source, name) pair in the hash map.
This should help with some of the readability concerns in #637 because the git
SHA that a source is locked to is now only mentioned once in a lockfile.
This commit does preserve, however, the mention of the version in each
dependency line which will likely never go away. This means that for
registry-based packages will still run into the same lockfile merge conflict
troubles, there will just be more readable versions than git hashes.
It should also be noted that this will alter all currently generated lockfiles
as any dependencies mentioned will lose the hashes mentioned afterwards. This
will likely cause somewhat of a transitionary pain as this version of cargo
propagates throughout.
bors [Mon, 27 Oct 2014 22:43:26 +0000 (22:43 +0000)]
auto merge of #758 : alexcrichton/cargo/issue-751, r=brson
This ends up killing two birds with one stone! The rationale behind this is that
the example and bin namespaces are not the same, and we don't mix metadata into
either filename, so the outputs need to be in different locations.
bors [Mon, 27 Oct 2014 20:41:25 +0000 (20:41 +0000)]
auto merge of #725 : alexcrichton/cargo/registry-fixes, r=brson
This PR contains a laundry list of improvements when using the registry as a source, and should be one of the last steps necessary for whipping cargo into shape to using the registry. Some of the highlights include:
* All APIs have been updated to the registry's current interface
* Lockfiles are now respected with registry sources
* Conservatively updating dependencies should work
* The network shouldn't be touched unless absolutely necessary
* Lockfiles actually keep versions locked when using a newer registry with more versions
* A new standalone lib was added for interoperating with the registry (HTTP-request-wise)
* Packages are now verified before being published to the registry (this can be opted out of)
* `cargo upload` was renamed to `cargo publish`
The commit series is intended to be individually reviewable and each commit should compile and pass all tests independently (but still needs to be applied in order).
Alex Crichton [Fri, 24 Oct 2014 16:18:19 +0000 (09:18 -0700)]
Build examples into `target/examples`
This ends up killing two birds with one stone! The rationale behind this is that
the example and bin namespaces are not the same, and we don't mix metadata into
either filename, so the outputs need to be in different locations.
Alex Crichton [Thu, 23 Oct 2014 19:21:08 +0000 (12:21 -0700)]
Respect yanks in the registry
In general the semantics of a yank are that the code itself is not removed from
the registry, but rather packages are no longer allowed to depend on the
version. A yank is normally done to remove broken code or perhaps secrets, but
actually deleting code means that all packages depending on the yanked version
all of a sudden break. For these reasons a yank does not actually delete code,
but only flags the version as yanked in the index.
Yanked packages are therefore able to be depended upon if a lockfile points at a
yanked version, but are not allowed to become new dependencies of packages.
Implementation-wise, the following changes were made:
* SourceIds originating from a lockfile for registries will have a precise
version listed (just a generic string "locked")
* Dependencies which use precise source ids are allowed to read yanked versions
* Dependencies without a precise source id are not allowed to use yanked
versions
When using a lockfile (or a previous instance of resolve), all operations will
rewrite dependencies to have the precise source ids where applicable, meaning
the locked versions have access to yanked versions, but the unlocked versions do
not.
Alex Crichton [Thu, 23 Oct 2014 19:12:03 +0000 (12:12 -0700)]
Fix a flaky test relying on nondeterminsm
When updating a source with multiple packages, the registry will lazily discover
both the precise and imprecise versions of the source at some point. Previously
the source would be updated depending on which was discovered first, but this
commit adds logic to understand that if an imprecise version is discovered after
a precise version then the imprecise version should be favored (because it will
always trigger an update).
This needs to understand, however, that if `cargo update --precise` is used that
those sources should not be updated again, so the sources treated in
`add_sources` are considered "special" in the sense that they're locked and will
not be updated again.
Alex Crichton [Thu, 23 Oct 2014 17:38:44 +0000 (10:38 -0700)]
Integrate the lockfile and registry-based deps
This commit radically changes the approach of how lockfiles are handled in the
package registry and how resolve interacts with it. Previously "using a
lockfile" entailed just adding all of the precise sources to the registry and
relying on them not being updated to remain locked. This strategy does not work
out well for the registry, however, as one source can provide many many packages
and it may need to be updated for other reasons.
This new strategy is to rewrite instances of `Summary` in an on-demand fashion
to mention locked versions and sources wherever possible. This will ensure that
any relevant `Dependency` will end up having an exact version requirement as
well as a precise source to originate from (if possible). This rewriting is
performed in a few locations:
1. The top-level package has its dependencies rewritten to their precise
variants if the dependency still matches the precise variant. This covers the
case where a dependency was updated the the lockfile now needs to be updated.
2. Any `Summary` returned from the package registry which matches a locked
`PackageId` will unconditionally have all of its dependencies rewritten to
their precise variants. This is done because any previously locked package
must remain locked during resolution.
3. Any `Summary` which points at a package which was not previously locked still
has its dependencies modified to point at any matching locked package. This
is done to ensure that updates are as conservative as possible.
There are still two outstanding problems with lockfiles and the registry which
this commit does not attempt to solve:
* Yanked versions are not respected
* The registry is still unconditionally updated on all compiles.
Alex Crichton [Thu, 23 Oct 2014 05:18:49 +0000 (22:18 -0700)]
Restructure resolve/lockfile ops
Move functionality from cargo_fetch and cargo_generate_lockfile into dedicated
lockfile/resolve modules. The plan is to expand the resolve module significantly
to deal with the lockfile that's loaded.
Alex Crichton [Wed, 22 Oct 2014 22:21:34 +0000 (15:21 -0700)]
Don't seed Registry with known source ids
In the normal case this isn't necessary due to the Registry dynamically
discovering sources when dependencies are queried for. Consequently there's no
need to register all these sources ahead-of-time, and it reduces the cognitive
load when thinking about sources and updates.
Alex Crichton [Wed, 22 Oct 2014 22:05:49 +0000 (15:05 -0700)]
Remove the hokey add_lockfile_sources
This method shouldn't be necessary as it should be possible to simply add all
sources known in the lockfile to a package registry, and
core::{resolve, registry} should take care of weeding them out if necessary.
In the process of doing so, this actually ends up fixing a problem with
rewriting a dependency to a new one which shares transitive deps. Previously all
the transitive deps were updated, but now the transitive deps remain locked at
where they were previously locked.