Testing Lua projects with GitHub Actions

Automate testing when pushing new code

GitHub Actions is a free tool provided by GitHub for software automation.It enables you to run a command on a virtual machine in response to changes toa repository, like a new commit. It’s an ideal candidate for automated testing,or continuous integration, for your Lua modules or projects.

By setting up automated testing you can ensure that any contributors to yourproject have their changes verified against your test suite as well. Allpull-request will be utilizes your testing workflow.

In this example I'm going to be usingbusted as my test suite runner, butyou can use pretty much anything: it’s easy to run any shell command.

We'll be using the following two GitHub Actions:

• gh-actions-lua
• gh-actions-luarocks

Feeling kind? Star the two projects above to help other people find out howto run Lua in their GitHub Actions workflows

Before getting started

Before setting up GitHub Actions you'll want to make sure you have a test suitethat runs successfully on your computer. Additionally, if you're developing aLua module for LuaRocks, then you'll want to have a.rockspec file that listsall your project’s dependencies and build process.

Even if you aren’t planning on publishing your code as a module, you canstill use a rockspec to declare your dependencies to make it easy to installthem during testing

Sicne I'm using busted, I use the following command to run my all of my tests:

$ busted

Optional: I have a rockspec file named something likemyproject-dev-1.rockspec that is checked into my repository. I recommendchecking in adev rockspec that can install your module by source from therepository. It might look something like this:

package="myproject"version="dev-1"source={url="git://github.com/leafo/myproject.git",}description={summary="Do something fancy",homepage="https://leafo.net",license="MIT"}dependencies={"lua >= 5.1","lpeg","luasocket","lua-cjson",}build={type="builtin",modules={["myproject"]="myproject/init.lua"}}

Configuring the Actions workflow

Aworkflow represents a collection of jobs you have created with GitHubActions. A job is a list of shell commands & actions executed on a virtualmachine. You can create a workflow with a specially named file in yourrepository.

For running a test suite on new commits we'll start by creating a new file inour repository:.github/workflows/test.yml

The nametest.yml can be anything you like. Additionally you can createmultiple workflows by creating multiple files in that directory

Insert the following into.github/workflows/test.yml to start out:

name:teston:[push]jobs:test:runs-on:ubuntu-lateststeps:-uses:actions/checkout@master-uses:leafo/gh-actions-lua@v5with:luaVersion:"5.1"-uses:leafo/gh-actions-luarocks@v2-name:setuprun:|luarocks install busted-name:testrun:|busted -o utfTerminal

Most of the lines in this file should be relatively self explanatory, buthere’s a quick summary:

Detailed documentation for the workflow syntax can be found on GitHub:Workflow syntax for GitHubActions

1. on instructs the workflow to run for everypush to the repository
2. We declare a job calledtest
3. steps lists everything that will happen to complete the job (each step is executed in order)
4. Theactions/checkout@master action is used to checkout the code in our repository (This action is authored by GitHub)
5. Theleafo/gh-actions-lua@v5 action will build and install Lua into the current environment
   • In this example we specify ask for Lua 5.1, examples with more Lua versions are shown below
6. Theleafo/gh-actions-luarocks@v2 action will build and install LuaRocks, this must come after the Lua installation
7. We issue a command calledsetup that will installbusted for use in the rest of our action
8. We issue a command calledtest that runsbusted in the working directory (with color enabled!)

In this example we included a version number (eg. @v5) with each action, itmay be necessary to update these versions number in the future. Generally youshould specify a version number so prevent unexpected failures happening whenan action’s source code is updated.

Head toRunning the action/workflow to see howto run the workflow (hint: you just commit and push), or scroll down to seesome ways to customize the workflow.

Installing dependencies with LuaRocks

The simplest way to install dependencies is to list them in the workflow filewithin thetest job in asetup step. Thegh-actions-luarocks action willautomatically configure LuaRocks to install for the current version of Lua yourtest is running for.

Here’s how you would edit.github/workflows/test.yml to add a few moredependencies:

 - name: setup   run: |     luarocks install busted+    luarocks install moonscript+    luarocks install luaossl     luarocks make

name:teston:[push]jobs:test:runs-on:ubuntu-lateststeps:-uses:actions/checkout@master-uses:leafo/gh-actions-lua@v5with:luaVersion:"5.1"-uses:leafo/gh-actions-luarocks@v2-name:setuprun:|luarocks install bustedluarocks install moonscriptluarocks install luaossl-name:testrun:|busted -o utfTerminal

If you have a rockspec in your repository’s root folder you can use LuaRocks toautomatically install the dependencies with themake command. (This has theside effect of also building your module)

If you have multiple rockspecs you will need to explicitly specify one likethis:luarocks make myrockspec-dev-1.rockspec

In our example rockspec we have the following dependencies declared:

-- myproject-dev-1.rockspecdependencies={"lua >= 5.1","lpeg","luasocket","lua-cjson",}

Modified thesetup step to look like this:

 - name: setup    run: |+    luarocks make     luarocks install busted

name:teston:[push]jobs:test:runs-on:ubuntu-lateststeps:-uses:actions/checkout@master-uses:leafo/gh-actions-lua@v5with:luaVersion:"5.1"-uses:leafo/gh-actions-luarocks@v2-name:setuprun:|luarocks makeluarocks install busted-name:testrun:|busted -o utfTerminal

luarocks make will use the rockspec located in your repository rootdirectory, and install the dependencies specified. If anything fails to installthen the job will fail.

busted is still installed manually because it’s not listed in the rockspecsince it’s a a test dependency, and not a runtime dependency.

Testing multiple versions of Lua with amatrix

Testing code that is designed to run on multiple versions of Lua is easilyaccomplished by listing out the versions you want to test in avariablematrix.

Make the following change to.github/workflows/test.yml:

 jobs:   test:+    strategy:+      fail-fast: false+      matrix:+        luaVersion: ["5.1", "5.2", "5.3", "luajit"]+     runs-on: ubuntu-latest      steps:     - uses: leafo/gh-actions-lua@v5       with:-        luaVersion: "5.1"+        luaVersion: ${{ matrix.luaVersion }}      - uses: leafo/gh-actions-luarocks@v2

name:teston:[push]jobs:test:strategy:fail-fast:falsematrix:luaVersion:["5.1","5.2","5.3","luajit"]runs-on:ubuntu-lateststeps:-uses:actions/checkout@master-uses:leafo/gh-actions-lua@v5with:luaVersion:${{ matrix.luaVersion }}-uses:leafo/gh-actions-luarocks@v2-name:buildrun:|luarocks install busted-name:testrun:|busted -o utfTerminal

With this change, on push the workflow will run 4 instances of thetest jobfor each version of Lua specified. You can see what versions are availableonthe gh-actions-luadocumentation.

Although not necessary, thefail-fast: false option will let all versionsrun to completion, even if one of them fails early. You can see multiplefailures and successes for every push, which can make it a lot easier todebug build issues.

Running the action/workflow

Commit.github/workflows/test.yml and push it to your repository and GitHubwill automatically start running your workflow on every push. If any of thesteps in your job fail, GitHub will report the commit as failed. Any mergerequests will also automatically be passed through the workflow to verifychanges.

If it doesn’t automatically run, check your repository Settings > Actions andenable it there.

Add a status badge to your repository

GitHub provides easy embeddable image to share the status of your build on your site or readme:

![build status](https://github.com/{USERNAME}/{REPO}/workflows/{WORKFLOW}/badge.svg)

You can generate the code for these directory by heading to your repository’sactions page and selecting the workflow you want.

Example from one of my projects:

I hope that helps you get started with GitHub Actions and Lua!