Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork221
Commit1493ca6
ci: Custom GitHub Runners for Nix Builds (#1745)
* chores(ci): opted-out nix-fast-build* chores(ci): set up nix-github-actions* chores(ci): comment out vestigal testinfra-ami-build* chores(ci): use custom github runners* chores(ci): use nix-eval-jobs and sets AWS creds to /etc/nix/aws* feat(ci): do not build if already cached* chore: improve reproducibility of postgresql buildsDo not create postgresql package that depends on the runtime evaluation context asthat harms reproducibility.* chore: remove nix-github-actions dependency* feat(ci): split nix build workflow into separate extensions and checks jobsSplit the monolithic nix-build job into two workflows:one for building PostgreSQL extensions and another for checks.Building extensions can be resource-intensive and time-consuming, so isolating them allows for better resource allocation and parallelism.Once they are built, the checks job can run tests and validationson the already built extensions.* feat(ci): extract nix build setup into reusable action and split builds by architectureExtract AWS credential setup and nix build steps into a composite actionto reduce duplication. Split extension builds into separate jobs perarchitecture (aarch64-linux, aarch64-darwin, x86_64-linux) and updatematrix generation to group packages by system.* fix(ci): do not hide cached builds* Revert "fix(ci): do not hide cached builds"This reverts commit9d41b1e.* chore: Temporarily disable x86_64-linux buildsWe don't have a self hosted runner for x86_64-linux at the moment* fix: sort packages and filter out cached ones* fix: do not skip checks-matrix if dependencies are skipped* fix: do not return debug fields in GitHub Actions matrix output* debugging* fix(ci): use !cancelled() instead of always() for dependent job conditionsReplace always() with !cancelled() to prevent jobs from running when the workflowis cancelled. Add explicit success/skip conditions for dependent jobs to ensureproper workflow execution flow.* fix(ci): stop chaining aws rolesWe couldn't set duration to 18000 seconds when chaining roles.* ci: run nixos test on aarch64-linux* fix(ci): disable eval-cache and accept-flake-config* ci: split checks build jobs by system architectureRefactor GitHub Actions workflow to run build checks in parallel across differentarchitectures (aarch64-linux, aarch64-darwin) with separate job matrices.* fix(ci): use correct architecture name in aarch64-linux builds* fix(ci): do not try to build already cached checks* fix(ci): simplify GitHub Actions workflow for Nix buildsCreate a single nix-eval job to determine packages to build, removingredundant extension and check matrices.* Revert "chores(ci): comment out vestigal testinfra-ami-build"This reverts commite2db368.* feat(ci): eval on blacksmith-32vcpu-ubuntu-2404* feat: add ephemeral Nix install action for GitHub runners* refactor(ci): extract nix eval into reusable workflow* feat: enable x86_64-linux builds in CI* feat: add PostgreSQL version to GitHub Actions job namesWhen building a postgres extension, the build matrix may includemultiple time the same extension for different PostgreSQL versions.This change makes it easier to identify which job corresponds to which PostgreSQLversion in the workflow runs.* fix: disable treefmt flake checktreefmt is already included in the pre-commit hooks check.* feat: run actionlint on new GitHub Actions workflows* chore: improve github matrix script type annotations* feat: optimize CI runner selection based on package sizeDynamically assign larger runners (32vcpu) for Rust and PostGIS extensionswhile using smaller runners (8vcpu) for standard packages.* chore: fix package meta maintainers format* chore: create a nix package for generating GitHub Actions matrixAdd pytest tests for the packageAdd nix-eval-jobs in path for the package* fix: configure runner according to the matrix jobThe matrix job returns the type of runner, so we can configure the nixinstallation step accordingly.* Update nix-eval-jobsOur changes were merged upstream, so we can now track the originalrepository again.* refactor(ci): standardize nix installation and disable cache push by default- Replace DeterminateSystems/nix-installer-action with custom nix-install-ephemeral action across all workflows- Change default push-to-cache from 'true' to 'false' to prevent unnecessary nix/aws configurations- Explicitly enable push-to-cache only for nix-build and nix-eval workflows where caching is beneficial* feat: use big-parallel to identify large packages* fix(ci): ensure x86_64-linux build is considered in testinfra and test workflows* fix: nix devShell inclusion condition* fix(ci): eval should fail if github-matrix run fails* fix(ci): remove redundant build psql bundle step* fix: reduce ARM runner size from 8vcpu to 4vcpu for ephemeral buildsWe might not need the full 8vcpu for aarch64-linux builds, so thischange reduces the runner size to 4vcpu to wait less for availableblacksmith runners.* feat: do not return empty matrices if no package has to be built* feat: fail pipeline if nix evaluation fails* Update nix/ext/pgvector.nixCo-authored-by: samrose <samuel@supabase.io>* fix: add skip job only for systems that don't have any job* fix(github-matrix): handle evaluation errors without deadlockFix github-matrix that would hang when nix-eval-jobs encountered errors due to subprocess pipe deadlock - stderr buffer would fill while reading stdout.This change ensure that evaluation errors are visible and the workflow fails properly while still showing which packages succeeded.* feat(github-matrix): integrate github-action-utils for better error visibilityIntegrates github-action-utils library to improve error and warningvisibility in GitHub Actions UI through workflow command annotations.* feat(github-matrix): group evaluation errors by messageRefactor error handling to collect and group evaluation errors similar to warnings. Errors with the same message are now displayed together with a list of affected attributes.* fix(github-matrix): improve multiline error display in GitHub ActionsExtract core error messages and format them better for GitHub Actionsannotations.* fix(ci): skip run-testinfra and run-tests when nix-eval failsAdd nix-eval to needs dependencies and check its result in conditional expressions to prevent downstream test jobs from running when evaluation fails.* chore(github-matrix): update message when there are no build for a system* fix(github-matrix): backward compatibility for Result accessWe are running an older version of the 'result' library that uses'_value' instead of 'ok_value' to access the successful result of acomputation.* refactor: migrate from packages to legacyPackages for PostgreSQL extensionsTo be able to build extensions versions packages separately in CI, weneed to expose them in a nested structure. It is currently not possibleto do so with the flattened packages structure, as the individualextension packages are not directly accessible.In this change, we replace the flattened package structure with nestedlegacyPackages to improve discoverability of individual extensionpackages.* refactor(nix): remove "-all" suffix from extension package namesSimplify extension package naming by removing the redundant "-all" suffix that was appended to pname attributes.* refactor(ci): split nix build jobs into separate packages and checks workflowsTo make sure we only build what is necessary, we start building packages first, then run checks once all packages are built successfully.* feat: use 8 vCPU runner for aarch64 buildsUse the same 8 vCPU runner for aarch64 builds as used for x86_64 builds to improve build performance.* feat(ci): test blacksmith stickydisk for eval---------Co-authored-by: Jean-François Roche <jfroche@pyxel.be>Co-authored-by: samrose <samuel@supabase.io>1 parenta431c10 commit1493ca6
File tree
60 files changed
+1281
-184
lines changed- .github
- actions/nix-install-self-hosted
- workflows
- nix
- docs
- ext
- pg_cron
- pg_graphql
- pg_jsonschema
- pgmq
- pgroonga
- plv8
- tests
- wrappers
- packages
- github-matrix
- tests
- postgresql
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
60 files changed
+1281
-184
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | 17 | | |
22 | | - | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
23 | 139 | | |
24 | 140 | | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
| 141 | + | |
| 142 | + | |
35 | 143 | | |
36 | 144 | | |
37 | | - | |
38 | | - | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
39 | 150 | | |
40 | | - | |
| 151 | + | |
41 | 152 | | |
42 | 153 | | |
43 | 154 | | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
75 | 180 | | |
76 | | - | |
77 | | - | |
78 | | - | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
79 | 187 | | |
80 | 188 | | |
81 | | - | |
82 | | - | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
83 | 199 | | |
84 | 200 | | |
85 | 201 | | |
86 | 202 | | |
87 | 203 | | |
88 | | - | |
89 | | - | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
90 | 214 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
0 commit comments
Comments
(0)