Repository files navigation

These are my personal opinionated configurable GitHub Actions Workflows, aimed at my personal needs to be the singlecentral point of all my GitHub Actions Workflows. Over the years most of my workflows started from one or two anddiverged in all directions, this repository is to reel that all back in one easy to maintain them for all myrepositories.

Each workflow, well most of them, take two runs-on argumentsrunsOnChaos andrunsOnOrder. As the names suggestrunsOnChaos runs as many jobs at the same time as possible whererunsOnOrder runs them one by one. For mostjobsrunsOnChaos is fine, but for things likeTerraForm apply, andHelm install where state somewhere ischanged and only one can run at the same timerunsOnOrder is the way to go. Mainly came to this while working ongetting the workflows managing my home cluster to work as intended:

When not set they all default toubuntu-latest which means it will run on GitHub provided Runners.

All the quality control checks for the CI entry point are handled and controlled through Makefiles this gives eachproject the control to check and assert everything relevant to it. When creating the task matrix the repositories theMakefile is expected to have atask-list-ci command which will output a JSON array of tasks.

Sparse checkout is used as much as possible to keep job run times as long as possible. Several of my projects have1GB+ binary test data checked in, this takes some time to checkout, maybe even more so on Raspberry Pis. Byaggressively using thesparse-checkout checkout optionactions/checkoutprovides checkout times plummeted. For the biggest repo this went down to 4 seconds from 66 seconds without sparsecheckout. This resulted in a few inputs being such asociSparseCheckout,helmSparseCheckout, andterraformSparseCheckout. Each of those 3 is used withsparse-checkout-cone-mode: false meaning that you can usepatterns like the following in there:

sparseCheckout:|  !/*  /.terraform/*  /**/*.md  /composer.json  /composer.lock

Also since those are free form fields you're expected to put inworkingDirectory andterraformDirectory your self.For example in the case ofTerraForm the full checkout step is:

 -uses:actions/checkout@v4with:sparse-checkout-cone-mode:falsesparse-checkout:|       !${{ inputs.workingDirectory }}/*       /${{ inputs.workingDirectory }}${{ inputs.terraformDirectory }}*       ${{ inputs.terraformSparseCheckout }}

Everything going through the package endpoints assumesSemantic Versioning, and hasautomation to automatically decide what the appropriate version bump is. Projects use monotonically increasing releaseversion,r1,r2,r13,r66 etc etc.

All images are pushing to GitHub's container registry atghcr.io with the full repository name as image name takenfrom${{ github.repository }}. On each build image image tag will besha-COMMIT_SHA whereCOMMIT_SHA istaken from${{ github.sha }}.

Release images are tagged with the passed milestone title as the image tag.

There are to types of entry points supported.Package andproject, the former is meant forPHP packages/GitHub Actions/Anything-that-is-not-a-project while the latter is for websites/services.

Each type has two entry points. The first beingRelease Management, which takes care of everything related toreleasing new versions and all the information and bits required for that. This includes:

• Required labels
• Package Manager/Helm/TerraForm Diff
• OCI image retaging
• Helm deploy to Kubernetes
• Setting a milestone on PR's
• Creating a GitHub release
• Generating a changelog
• Push static content to a CDN

It should be triggered for the following events:

• Closing of a milestone
• PR's for the follow types:opened,labeled,unlabeled,synchronize,reopened,milestoned,demilestoned, andready_for_review.

The other entry point isCI it does everything related to validating the current push of the code meets expectationsthrough a series ofMakefile commands. When present CI will also build the OCI (Docker) image ifpresent. Any control over the different platforms the OCI image is released for is done here, the release managementdoing any retagging will detect which platforms are in the image it retags and uses those without the need toconfigure them.

And how to configure it.

The CI entry point only takes two configuration inputsenv andservices, both take a JSON string as argument.For example adding the Redis DSN to the environment is written as:

with:env:"{\"REDIS_DSN\":\"redis://redis:6379/6\"}"

While the service providing redis is written as:

with:services:"{\"redis\":{\"image\":\"redis\",\"options\":\"--health-cmd\\\"redis-cli ping\\\" --health-interval 1s --health-timeout 5s --health-retries 50\"}}"

The OCI image building and retagging responsibilities using Docker are split between the CI and Release Managemententry points. CI builds the images and decides what goes into them and for what platform, while Release Managementpurely retags the images build by CI.

CI will only build images if you provide theDockerfile you're using through thedockerfile input. ProvidedockerBuildTarget when using a multi-stage Docker file to build the desired target. Any additional arguments can bepassed to the Docker build command usingdockerBuildExtraArguments.

By default the OCI build workflow will detect all the platforms the first upstream image found inFROM is build forand build the image for the same platforms. If you want to override that pass anCSV of different platforms to theociPlatforms in put in this format:linux/amd64,linux/arm.

To speed up checking out during buildingociSparseCheckout can be used to exclude any files we don't care about andwill slowdown our checkout. For example in one of my projects thetests/data/ directory contains 1GB+ binary datafor testing, I have zero need to check that out during image build. So my sparse checkout allows everything but anyfile fromtests/data/:

ociSparseCheckout:|  /*  !/tests/data/*

To kick off retagging simple setociRetag totrue on the Release Management entry point.

When thehelmDirectory is passed to the release management entry point isn't an empty string both theHelm Diff and Helm Deploy workflows are executed against that directory.

As with any Helm chart being deployed we need to give it a name, thehelmReleaseName is used to specify that.

By default a sparse checkout is performed and only the passedhelmDirectory is checked out, if you need thanthat you can usehelmSparseCheckout with additional patterns to check out.

To set additional values or to overwrite existing values fromvalues.yamlhelmAdditionalArguments can be used topass any additional args to the diff and upgrade commands. For example to get dynamically generated values fromanother job:

name:Release Managementon:pull_request:types:      -opened      -labeled      -unlabeled      -synchronize      -reopenedmilestone:types:      -closedpermissions:contents:writeissues:writepull-requests:writepackages:writejobs:helm-json:name:Generate JSON for Helmruns-on:chaosoutputs:helm-json:${{ steps.helm-json.outputs.helm-json }}steps:      -run:printf "helm-json=%s" $(echo "{\"LOGLEVEL\":\"DEBUG\"}") >> $GITHUB_OUTPUTrelease-management:name:Release Managementneeds:      -helm-jsonsecrets:inherituses:WyriHaximus/github-workflows/.github/workflows/project-release-management.yaml@mainwith:helmAdditionalArguments:--set-json='application=${{ needs.helm-json.outputs.helm-json }}'

Because these workflows provide diff comments on PR's for Helm and others we need to specifyhelmReleaseValueName sowe can assign the previous tag on diff and closed milestone on deploy to that value. We can't usehelmAdditionalArguments for this as weautodetect the previous tag. SimilarlyhelmUpdateAppVersion is used to update theappVersion inChart.yaml,when set totrue, with the same value in those two scenarios.

A few, maybe somewhat random, linters are included in the work flows. One of them checks if links in markdownresources are responding with a ~200 status code and errors when they are no longer working.

When theterraformDirectory is passed to the release management entry point isn't an empty string both theTerraForm Diff and TerraForm Apply workflows are executed against that directory. The repository is expecting to havethe following secrets for TerraForm state storage on S3:

• TERRAFORM_STATE_KEY
• TERRAFORM_STATE_SECRET
• TERRAFORM_STATE_BUCKET
• TERRAFORM_STATE_REGION

By default a sparse checkout is performed and only the passedterraformDirectory is checked out, if you need thanthat you can useterraformSparseCheckout with additional patterns to check out. Further moreterraformParallelismandterraformLogLevel can be used to tune TerraForm's behavior.

In order to inject variables into TerraFormterraformVars is used to createterraform.tfvars intheterraformDirectory. Variables can be hard coded but secrets are available as environment variables:

terraformVars:|  kubernetes_config_path   = "~/.kube/config"  kubernetes_context       = "$HOME_KUBE_CONTEXT"  kubernetes_namespace     = "$HOME_KUBE_NAMESPACE"

name:Continuous Integrationon:push:branches:      -'main'      -'master'      -'refs/heads/v[0-9]+.[0-9]+.[0-9]+'      -'refs/heads/[0-9]+.[0-9]+.[0-9]+'pull_request:## This workflow needs the `pull-request` permissions to work for the package diffing## Refs: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissionspermissions:pull-requests:writecontents:readjobs:ci:name:Continuous Integrationuses:WyriHaximus/github-workflows/.github/workflows/package.yaml@main

name:Release Managementon:pull_request:types:      -opened      -labeled      -unlabeled      -synchronize      -reopened      -milestoned      -demilestoned      -ready_for_reviewmilestone:types:      -closedpermissions:contents:writeissues:writepull-requests:writejobs:release-management:name:Create Releaseuses:WyriHaximus/github-workflows/.github/workflows/package-release-management.yaml@mainwith:milestone:${{ github.event.milestone.title }}description:${{ github.event.milestone.title }}

name:Continuous Integrationon:push:branches:      -'main'      -'master'      -'refs/heads/r[0-9]+'pull_request:## This workflow needs the `pull-request` permissions to work for the package diffing## Refs: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissionspermissions:pull-requests:writecontents:readpackages:writejobs:ci:name:Continuous Integrationuses:WyriHaximus/github-workflows/.github/workflows/project.yaml@mainsecrets:inheritwith:runsOnChaos:chaosrunsOnOrder:queue

name:Release Managementon:pull_request:types:      -opened      -labeled      -unlabeled      -synchronize      -reopenedmilestone:types:      -closedpermissions:contents:writeissues:writepull-requests:writepackages:writejobs:release-management:name:Release Managementsecrets:inherituses:WyriHaximus/github-workflows/.github/workflows/project-release-management.yaml@mainwith:milestone:${{ github.event.milestone.title }}description:${{ github.event.milestone.description }}runsOnChaos:chaosrunsOnOrder:queue

• Tagv1(.0(.0)) - Needs to be done at some point as I want to version all of this in the same way as GitHub Actions are version with mutable major and minor tags and immutable patch tags.
• OCI Build
• OCI Retagging
• Make all runs-on for projects configurable
• Make all runs-on for packages configurable
• Get all CI QA checks to run on runsOn inputs instead of GitHub hosted Runners
• Helm Diff
• Helm Upgrading
• Helm Automatically detect all dependencies and load those in so we can remove hardcoding them in the workflows
• TerraForm Diff (Plan)
• TerraForm Apply
• Terraform vars from secrets
• Check links in Markdown files for non 200 status codes
• Make CI's test directly on OS runs-on array configurable
• Cronjob/Scheduled workflows for things like Docker image clean up
• Sparse checkout all the things
• Fix typo in release management entry point filenames, and have all users point at the currect one
• Add documentation once Utils entry point is more stable
• Add GHCR.io image clean up to project utils entry point

Copyright (c) 2025 Cees-Jan Kiewiet

Permission is hereby granted, free of charge, to any person obtaining a copyof this software and associated documentation files (the "Software"), to dealin the Software without restriction, including without limitation the rightsto use, copy, modify, merge, publish, distribute, sublicense, and/or sellcopies of the Software, and to permit persons to whom the Software isfurnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in allcopies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THEAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THESOFTWARE.