| andybons | 3322f76 | 2015-08-24 21:37:09 | [diff] [blame] | 1 | # Updating clang |
| 2 | |
| Nico Weber | 4071eca | 2018-09-25 12:29:14 | [diff] [blame] | 3 | We distribute prebuilt packages of LLVM binaries, including clangand lld, that |
| 4 | all developersand bots pull at`gclient runhooks` time.These binaries are |
| 5 | just regular LLVM binaries built at afixed upstream revision.This document |
| 6 | describes how to build apackage at a newer revisionand updateChromium to it. |
| 7 | An archive of all packages built so faris at https://is.gd/chromeclang |
| 8 | |
| Nico Weber | f218afd | 2018-09-20 19:11:21 | [diff] [blame] | 9 | 1.Check that https://ci.chromium.org/p/chromium/g/chromium.clang/console |
| Arthur Eubanks | a8722a5 | 2020-08-25 17:15:12 | [diff] [blame] | 10 | looks reasonably green.Red botswith seemingly normal test failures are |
| 11 | usually ok, that likely means the testis brokenwith the stableClangas |
| 12 | well. |
| nodir | 06cbaa0 | 2015-08-25 17:15:24 | [diff] [blame] | 13 | 1.Sync yourChromium tree to the latest revision to pick up any plugin |
| Arthur Eubanks | f055cc5 | 2020-06-10 02:21:37 | [diff] [blame] | 14 | changes. |
| Hans Wennborg | fbda621 | 2025-04-04 14:30:44 | [diff] [blame] | 15 | 1.Run |
| 16 | [go/chrome-promote-clang](https://goto.google.com/chrome-promote-clang), |
| 17 | passingin the revision of theClangandRustpackage version you want to |
| 18 | push.For example: |
| 19 | |
| 20 | ``` |
| 21 | $ copy_staging_to_prod_and_goma.sh --clang-rev llvmorg-21-init-5118-g52cd27e6-1 --rust-rev f7b43542838f0a4a6cfdb17fbeadf45002042a77-1 |
| 22 | ``` |
| 23 | |
| 24 | Alternatively,use |
| 25 | [go/chrome-push-clang-to-goma](https://goto.google.com/chrome-push-clang-to-goma). |
| Nico Weber | 98926b02 | 2020-06-28 11:27:36 | [diff] [blame] | 26 | This takes a recent dry run CL to update clang,andif the trybots were |
| 27 | successful it will copy the binariesfrom the staging bucket to the |
| Hans Wennborg | fbda621 | 2025-04-04 14:30:44 | [diff] [blame] | 28 | production one. |
| 29 | |
| 30 | Writing to the production bucket requires special permissions. |
| 31 | Then it will push the packages to RBE.If you |
| Nico Weber | 98926b02 | 2020-06-28 11:27:36 | [diff] [blame] | 32 | donot have the necessary credentials todo the upload, ask |
| Arthur Eubanks | f055cc5 | 2020-06-10 02:21:37 | [diff] [blame] | 33 | clang@chromium.org to find someone who does. |
| Arthur Eubanks | a8722a5 | 2020-08-25 17:15:12 | [diff] [blame] | 34 | *Alternatively, to create your own roll CL, you can manually run |
| 35 | `tools/clang/scripts/upload_revision.py`with a recent upstream LLVM |
| Hans Wennborg | fbda621 | 2025-04-04 14:30:44 | [diff] [blame] | 36 | commit hashas the argument.After the`*_upload_clang`and`*_upload_rust` |
| 37 | trybots are successfully finished, run |
| Arthur Eubanks | a8722a5 | 2020-08-25 17:15:12 | [diff] [blame] | 38 | [go/chrome-promote-clang](https://goto.google.com/chrome-promote-clang) |
| Hans Wennborg | fbda621 | 2025-04-04 14:30:44 | [diff] [blame] | 39 | with thenewClangandRustpackage names. |
| Josip Sokcevic | a41c3c45 | 2024-05-30 15:25:38 | [diff] [blame] | 40 | 1.Run`tools/clang/scripts/sync_deps.py` to update the deps entriesin DEPS. |
| Hans Wennborg | fbda621 | 2025-04-04 14:30:44 | [diff] [blame] | 41 | 1.`gclient sync` to download those packages. |
| 42 | 1.Run`tools/rust/gnrt_stdlib.py` to update the GN filesfor theRust standard library. |
| Nico Weber | 91670290 | 2020-01-30 19:25:47 | [diff] [blame] | 43 | 1.Run an exhaustiveset oftry jobs to test thenew compiler.The CL |
| Arthur Eubanks | f055cc5 | 2020-06-10 02:21:37 | [diff] [blame] | 44 | description created previouslyby upload_revision.py includes |
| 45 | `Cq-Include-Trybots:` linesfor all needed bots, so it's sufficient to just |
| 46 | run `git cl try` (or hit "CQ DRY RUN" on gerrit). |
| 47 | 1. Commit the roll CL from the previous step. |
| Takuto Ikuta | 7478af72 | 2024-05-27 07:23:19 | [diff] [blame] | 48 | 1. The bots will now pull the prebuilt binary, and RBE will have a matching |
| nodir | 06cbaa0 | 2015-08-25 17:15:24 | [diff] [blame] | 49 | binary, too. |
| Nico Weber | 4071eca | 2018-09-25 12:29:14 | [diff] [blame] | 50 | |
| Arthur Eubanks | 637eab4 | 2020-06-15 18:03:07 | [diff] [blame] | 51 | ## Performance regressions |
| 52 | |
| 53 | After doing a clang roll, you may get a performance bug assigned to you |
| 54 | ([example](https://crbug.com/1094671)). Some performance noise is expected |
| 55 | while doing a clang roll. |
| 56 | |
| 57 | You can check all performance data for a clang roll via |
| 58 | `https://chromeperf.appspot.com/group_report?rev=XXXXXX`, where `XXXXXX` is the |
| Hans Wennborg | fbda621 | 2025-04-04 14:30:44 | [diff] [blame] | 59 | Chromium revision number, e.g. `778090` for the example bug (look in the first |
| 60 | message of the performance bug to find this). Click the checkboxes to display |
| 61 | graphs. Hover over points in the graph to see the value and error. |
| Arthur Eubanks | 637eab4 | 2020-06-15 18:03:07 | [diff] [blame] | 62 | |
| 63 | Serious regressions require bisecting upstream commits (TODO: how to repro?). |
| 64 | If the regressions look insignificant and there is green as well as red, you |
| 65 | can close the bug as "WontFix" with an explanation. |
| 66 | |
| Nico Weber | 4071eca | 2018-09-25 12:29:14 | [diff] [blame] | 67 | ## Adding files to the clang package |
| 68 | |
| 69 | The clang package is downloaded unconditionally by all bots and devs. It's |
| 70 | called"clang"for historical reasons, but nowadays also contains other |
| 71 | mission-critical toolchain pieces besides clang. |
| 72 | |
| 73 | Wetry to limit the contents of the clangpackage.They should meet these |
| 74 | criteria: |
| 75 | |
| 76 | - things that are usedby most developersuse most of the time(e.g. a |
| 77 | compiler, a linker, sanitizer runtimes) |
| 78 | - things neededfor doing official builds |
| 79 | |
| Andrew Grieve | 2989fb8 | 2024-04-22 14:42:36 | [diff] [blame] | 80 | # Adding a New Package |
| Nico Weber | 4071eca | 2018-09-25 12:29:14 | [diff] [blame] | 81 | |
| Andrew Grieve | 2989fb8 | 2024-04-22 14:42:36 | [diff] [blame] | 82 | If you want to make artifacts available thatdonot meet the criteriafor |
| 83 | being includedin the"clang"package, you can makepackage.py upload it to |
| 84 | a separate zip fileandthen download it on an opt-in basisbyusing |
| 85 | update.py's --package option. Here is [an example of adding a new package]. |
| 86 | |
| 87 | To test changes to `package.py`, change `CLANG_SUB_REVISION` in `update.py` to |
| 88 | a random number above 2000 and run the `*_upload_clang` trybots. |
| 89 | |
| 90 | Once the change to `package.py` is in, file a bug under `Tools > LLVM` |
| 91 | requesting that a new package be created ([example bug]). |
| 92 | |
| 93 | Once it's been uploadedand rolled, you can download it via: |
| 94 | |
| 95 | ``` |
| 96 | tools/clang/scripts/update.py --package your-package-name |
| 97 | ``` |
| 98 | |
| 99 | [an example of adding anewpackage](https://chromium-review.googlesource.com/c/chromium/src/+/5463029) |
| Takuto Ikuta | 7478af72 | 2024-05-27 07:23:19 | [diff] [blame] | 100 | [example bug]: https://crbug.com/335730441 |
| Hans Wennborg | 0499381b | 2025-04-07 11:13:59 | [diff] [blame] | 101 | |
| 102 | # On local patches and cherry-picks |
| 103 | |
| 104 | Exceptfor the addition ofClang plugins, whichdonot affect the compiler |
| 105 | output,Chromium's LLVM binaries are vanilla builds of the upstream source code |
| 106 | at a specific revision, with no alterations. |
| 107 | |
| 108 | We believe this helps when working with upstream: our compiler should behave |
| 109 | exactly the same as if someone else built LLVM at the same revision, making |
| 110 | collaboration easier. |
| 111 | |
| 112 | It also creates an incentive for stabilizing the HEAD revision of LLVM: since |
| 113 | we ship a vanilla build of an upstream revision, we have to ensure that a |
| 114 | revision can be found which is stable enough to build Chromium and pass all its |
| 115 | tests. While allowing local cherry-picks, reverts, or other patches, would |
| 116 | probably allow more regular toolchain releases, we believe we can perform |
| 117 | toolchain testing and fix issues fast enough that finding a stable revision is |
| 118 | possible, and that this is the right trade-off for us and for the LLVM |
| 119 | community. |
| 120 | |
| 121 | For Rust, since the interface between tip-of-tree rustc and LLVM is less |
| 122 | stable, and since landing fixes in Rust is much slower (even after approval, a |
| 123 | patch can take more than 24 hours to land), we allow cherry-picks of such fixes |
| 124 | in our Rust toolchain build. |
