ARCHITECTURE.md

   1 # Cargo Architecture
   2
   3 This document gives a high level overview of Cargo internals. You may
   4 find it useful if you want to contribute to Cargo or if you are
   5 interested in the inner workings of Cargo.
   6
   7 The purpose of Cargo is to formalize a canonical Rust workflow, by automating
   8 the standard tasks associated with distributing software. Cargo simplifies
   9 structuring a new project, adding dependencies, writing and running unit tests,
  10 and more.
  11
  12
  13 ## Subcommands
  14
  15 Cargo is a single binary composed of a set of [`clap`] subcommands. All subcommands live in
  16 `src/bin/cargo/commands` directory. `src/bin/cargo/main.rs` is the entry point.
  17
  18 Each subcommand, such as [`src/bin/cargo/commands/build.rs`], has its own API
  19 interface, similarly to Git's, parsing command line options, reading the
  20 configuration files, discovering the Cargo project in the current directory and
  21 delegating the actual implementation to one
  22 of the functions in [`src/cargo/ops/mod.rs`]. This short file is a good
  23 place to find out about most of the things that Cargo can do.
  24 Subcommands are designed to pipe to one another, and custom subcommands make
  25 Cargo easy to extend and attach tools to.
  26
  27 [`clap`]: https://clap.rs/
  28 [`src/bin/cargo/commands/build.rs`]: src/bin/cargo/commands/build.rs
  29 [`src/cargo/ops/mod.rs`]: src/cargo/ops/mod.rs
  30
  31
  32 ## Important Data Structures
  33
  34 There are some important data structures which are used throughout
  35 Cargo.
  36
  37 [`Config`] is available almost everywhere and holds "global"
  38 information, such as `CARGO_HOME` or configuration from
  39 `.cargo/config` files. The [`shell`] method of [`Config`] is the entry
  40 point for printing status messages and other info to the console.
  41
  42 [`Workspace`] is the description of the workspace for the current
  43 working directory. Each workspace contains at least one
  44 [`Package`]. Each package corresponds to a single `Cargo.toml`, and may
  45 define several [`Target`]s, such as the library, binaries, integration
  46 test or examples. Targets are crates (each target defines a crate
  47 root, like `src/lib.rs` or `examples/foo.rs`) and are what is actually
  48 compiled by `rustc`.
  49
  50 A typical package defines the single library target and several
  51 auxiliary ones. Packages are a unit of dependency in Cargo, and when
  52 package `foo` depends on package `bar`, that means that each target
  53 from `foo` needs the library target from `bar`.
  54
  55 [`PackageId`] is the unique identifier of a (possibly remote)
  56 package. It consist of three components: name, version and source
  57 id. Source is the place where the source code for package comes
  58 from. Typical sources are crates.io, a git repository or a folder on
  59 the local hard drive.
  60
  61 [`Resolve`] is the representation of a directed acyclic graph of package
  62 dependencies, which uses [`PackageId`]s for nodes. This is the data
  63 structure that is saved to the lock file. If there is no lock file,
  64 Cargo constructs a resolve by finding a graph of packages which
  65 matches declared dependency specification according to semver.
  66
  67 [`Config`]: https://docs.rs/cargo/latest/cargo/util/config/struct.Config.html
  68 [`shell`]: https://docs.rs/cargo/latest/cargo/util/config/struct.Config.html#method.shell
  69 [`Workspace`]: https://docs.rs/cargo/latest/cargo/core/struct.Workspace.html
  70 [`Package`]: https://docs.rs/cargo/latest/cargo/core/package/struct.Package.html
  71 [`Target`]: https://docs.rs/cargo/latest/cargo/core/manifest/struct.Target.html
  72 [`PackageId`]: https://docs.rs/cargo/latest/cargo/core/package_id/struct.PackageId.html
  73 [`Resolve`]: https://docs.rs/cargo/latest/cargo/core/struct.Resolve.html
  74
  75
  76 ## Persistence
  77
  78 Cargo is a non-daemon command line application, which means that all
  79 the information used by Cargo must be persisted on the hard drive. The
  80 main sources of information are `Cargo.toml` and `Cargo.lock` files,
  81 `.cargo/config` configuration files and the globally shared registry
  82 of packages downloaded from crates.io, usually located at
  83 `~/.cargo/registry`. See [`src/cargo/sources/registry`] for the specifics of
  84 the registry storage format.
  85
  86 [`src/cargo/sources/registry`]: src/cargo/sources/registry
  87
  88
  89 ## Concurrency
  90
  91 Cargo is mostly single threaded. The only concurrency inside a single
  92 instance of Cargo happens during compilation, when several instances
  93 of `rustc` are invoked in parallel to build independent
  94 targets. However there can be several different instances of Cargo
  95 process running concurrently on the system. Cargo guarantees that this
  96 is always safe by using file locks when accessing potentially shared
  97 data like the registry or the target directory.
  98
  99
 100 ## Tests
 101
 102 Cargo has an impressive test suite located in the `tests` folder. Most
 103 of the test are integration: a project structure with `Cargo.toml` and
 104 rust source code is created in a temporary directory, `cargo` binary
 105 is invoked via `std::process::Command` and then stdout and stderr are
 106 verified against the expected output. To simplify testing, several
 107 macros of the form `[MACRO]` are used in the expected output. For
 108 example, `[..]` matches any string.
 109
 110 To see stdout and stderr streams of the subordinate process, add `.stream()`
 111 call to the built-up `Execs`:
 112
 113 ```rust
 114 // Before
 115 p.cargo("run").run();
 116
 117 // After
 118 p.cargo("run").stream().run();
 119 ```
 120
 121 Alternatively to build and run a custom version of cargo simply run `cargo build`
 122 and execute `target/debug/cargo`. Note that `+nightly`/`+stable` (and variants),
 123 being [rustup] features, won't work when executing the locally
 124 built cargo binary directly, you have to instead build with `cargo +nightly build`
 125 and run with `rustup run` (e.g `rustup run nightly
 126 <path-to-cargo>/target/debug/cargo <args>..`) (or set the `RUSTC` env var to point
 127 to nightly rustc).
 128
 129 [rustup]: https://rustup.rs/
 130
 131
 132 ## Logging
 133
 134 Cargo uses [`env_logger`], so you can set
 135 `CARGO_LOG` environment variable to get the logs. This is useful both for diagnosing
 136 bugs in stable Cargo and for local development. Cargo also has internal hierarchical
 137 profiling infrastructure, which is activated via `CARGO_PROFILE` variable
 138
 139 ```
 140 # Outputs all logs with levels debug and higher
 141 $ CARGO_LOG=debug cargo generate-lockfile
 142
 143 # Don't forget that you can filter by module as well
 144 $ CARGO_LOG=cargo::core::resolver=trace cargo generate-lockfile
 145
 146 # Output first three levels of profiling info
 147 $ CARGO_PROFILE=3 cargo generate-lockfile
 148 ```
 149
 150 [`env_logger`]: https://docs.rs/env_logger/*/env_logger/