src/doc/rustc-guide/src/codegen/updating-llvm.md

   1 # Updating LLVM
   2
   3 The Rust compiler uses LLVM as its primary codegen backend today, and naturally
   4 we want to at least occasionally update this dependency! Currently we do not
   5 have a strict policy about when to update LLVM or what it can be updated to, but
   6 a few guidelines are applied:
   7
   8 * We try to always support the latest released version of LLVM
   9 * We try to support the "last few" versions of LLVM (how many is changing over
  10   time)
  11 * We allow moving to arbitrary commits during development.
  12 * Strongly prefer to upstream all patches to LLVM before including them in
  13   rustc.
  14
  15 This policy may change over time (or may actually start to exist as a formal
  16 policy!), but for now these are rough guidelines!
  17
  18 ## Why update LLVM?
  19
  20 There are a few reasons nowadays that we want to update LLVM in one way or
  21 another:
  22
  23 * A bug could have been fixed! Often we find bugs in the compiler and fix
  24   them upstream in LLVM. We'll want to pull fixes back to the compiler itself as
  25   they're merged upstream.
  26
  27 * A new feature may be available in LLVM that we want to use in rustc,
  28   but we don't want to wait for a full LLVM release to test it out.
  29
  30 * LLVM itself may have a new release and we'd like to update to this LLVM
  31   release.
  32
  33 Each of these reasons has a different strategy for updating LLVM, and we'll go
  34 over them in detail here.
  35
  36 ## Bugfix Updates
  37
  38 For updates of LLVM that are to fix a small bug, we cherry-pick the bugfix to
  39 the branch we're already using. The steps for this are:
  40
  41 1. Make sure the bugfix is in upstream LLVM.
  42 2. Identify the branch that rustc is currently using. The `src/llvm-project`
  43    submodule is always pinned to a branch of the
  44    [rust-lang/llvm-project](https://github.com/rust-lang/llvm-project) repository.
  45 3. Fork the rust-lang/llvm-project repository
  46 4. Check out the appropriate branch (typically named `rustc/a.b-yyyy-mm-dd`)
  47 5. Cherry-pick the upstream commit onto the branch
  48 6. Push this branch to your fork
  49 7. Send a Pull Request to rust-lang/llvm-project to the same branch as before.
  50    Be sure to reference the Rust and/or LLVM issue that you're fixing in the PR
  51    description.
  52 8. Wait for the PR to be merged
  53 9. Send a PR to rust-lang/rust updating the `src/llvm-project` submodule with
  54    your bugfix. This can be done locally with `git submodule update --remote
  55    src/llvm-project` typically.
  56 10. Wait for PR to be merged
  57
  58 The tl;dr; is that we can cherry-pick bugfixes at any time and pull them back
  59 into the rust-lang/llvm-project branch that we're using, and getting it into the
  60 compiler is just updating the submodule via a PR!
  61
  62 Example PRs look like:
  63 [#59089](https://github.com/rust-lang/rust/pull/59089)
  64
  65 ## Feature updates
  66
  67 > Note that this is all information as applies to the current day in age. This
  68 > process for updating LLVM changes with practically all LLVM updates, so this
  69 > may be out of date!
  70
  71 Unlike bugfixes, updating to pick up a new feature of LLVM typically requires a
  72 lot more work. This is where we can't reasonably cherry-pick commits backwards
  73 so we need to do a full update. There's a lot of stuff to do here, so let's go
  74 through each in detail.
  75
  76 1. Create a new branch in the rust-lang/llvm-project repository. This branch
  77    should be named `rustc/a.b-yyyy-mm-dd` where `a.b` is the current version
  78    number of LLVM in-tree at the time of the branch and the remaining part is
  79    today's date. Move this branch to the commit in LLVM that you'd like, which
  80    for this is probably the current LLVM HEAD.
  81
  82 2. Apply Rust-specific patches to the llvm-project repository. All features and
  83    bugfixes are upstream, but there's often some weird build-related patches
  84    that don't make sense to upstream which we have on our repositories. These
  85    patches are around the latest patches in the rust-lang/llvm-project branch
  86    that rustc is currently using.
  87
  88 3. Build the new LLVM in the `rust` repository. To do this you'll want to update
  89    the `src/llvm-project` repository to your branch and the revision you've
  90    created. It's also typically a good idea to update `.gitmodules` with the new
  91    branch name of the LLVM submodule. Make sure you've committed changes to
  92    `src/llvm-project` to ensure submodule updates aren't reverted. Some commands
  93    you should execute are:
  94
  95    * `./x.py build src/llvm` - test that LLVM still builds
  96    * `./x.py build src/tools/lld` - same for LLD
  97    * `./x.py build` - build the rest of rustc
  98
  99    You'll likely need to update `src/rustllvm/*.cpp` to compile with updated
 100    LLVM bindings. Note that you should use `#ifdef` and such to ensure that the
 101    bindings still compile on older LLVM versions.
 102
 103 4. Test for regressions across other platforms. LLVM often has at least one bug
 104    for non-tier-1 architectures, so it's good to do some more testing before
 105    sending this to bors! If you're low on resources you can send the PR as-is
 106    now to bors, though, and it'll get tested anyway.
 107
 108    Ideally, build LLVM and test it on a few platforms:
 109
 110    * Linux
 111    * OSX
 112    * Windows
 113
 114    and afterwards run some docker containers that CI also does:
 115
 116    * `./src/ci/docker/run.sh wasm32-unknown`
 117    * `./src/ci/docker/run.sh arm-android`
 118    * `./src/ci/docker/run.sh dist-various-1`
 119    * `./src/ci/docker/run.sh dist-various-2`
 120    * `./src/ci/docker/run.sh armhf-gnu`
 121
 122 5. Prepare a PR to `rust-lang/rust`. Work with maintainers of
 123    `rust-lang/llvm-project` to get your commit in a branch of that repository,
 124    and then you can send a PR to `rust-lang/rust`. You'll change at least
 125    `src/llvm-project` and will likely also change `src/rustllvm/*` as well.
 126
 127 For prior art, previous LLVM updates look like
 128 [#55835](https://github.com/rust-lang/rust/pull/55835)
 129 [#47828](https://github.com/rust-lang/rust/pull/47828)
 130 [#62474](https://github.com/rust-lang/rust/pull/62474)
 131 [#62592](https://github.com/rust-lang/rust/pull/62592). Note that sometimes it's
 132 easiest to land `src/rustllvm/*` compatibility as a PR before actually updating
 133 `src/llvm-project`. This way while you're working through LLVM issues others
 134 interested in trying out the new LLVM can benefit from work you've done to
 135 update the C++ bindings.
 136
 137 ### Caveats and gotchas
 138
 139 Ideally the above instructions are pretty smooth, but here's some caveats to
 140 keep in mind while going through them:
 141
 142 * LLVM bugs are hard to find, don't hesitate to ask for help! Bisection is
 143   definitely your friend here (yes LLVM takes forever to build, yet bisection is
 144   still your friend)
 145 * If you've got general questions, @alexcrichton can help you out.
 146 * Creating branches is a privileged operation on GitHub, so you'll need someone
 147   with write access to create the branches for you most likely.
 148
 149 ## New LLVM Release Updates
 150
 151 Updating to a new release of LLVM is very similar to the "feature updates"
 152 section above. The release process for LLVM is often months-long though and we
 153 like to ensure compatibility ASAP. The main tweaks to the "feature updates"
 154 section above is generally around branch naming. The sequence of events
 155 typically looks like:
 156
 157 1. LLVM announces that its latest release version has branched. This will show
 158    up as a branch in https://github.com/llvm/llvm-project typically named
 159    `release/$N.x` where `$N` is the version of LLVM that's being released.
 160
 161 2. We then follow the "feature updates" section above to create a new branch of
 162    LLVM in our rust-lang/llvm-project repository. This follows the same naming
 163    convention of branches as usual, except that `a.b` is the new version. This
 164    update is eventually landed in the rust-lang/rust repository.
 165
 166 3. Over the next few months, LLVM will continually push commits to its
 167    `release/a.b` branch. Often those are bug fixes we'd like to have as well.
 168    The merge process for that is to use `git merge` itself to merge LLVM's
 169    `release/a.b` branch with the branch created in step 2. This is typically
 170    done multiple times when necessary while LLVM's release branch is baking.
 171
 172 4. LLVM then announces the release of version `a.b`.
 173
 174 5. After LLVM's official release, we follow the "feature update" section again
 175    to create a new branch in the rust-lang/llvm-project repository, this time
 176    with a new date. The commit history should look much cleaner as just a few
 177    Rust-specific commits stacked on top of stock LLVM's release branch.