Anatomy of a Workflow File

Workflow files are declaratively written YAML files with a predefined structure that must be adhered to for a workflow to run successfully. Your YAML workflow files are stored and defined in a.github/workflows/ folder in your project’s root directory.

Your workflow folder can have multiple workflow files, each of which will perform a certain task. You can name these workflow files anything you’d like. However, for the sake of simplicity and readability, it’s common practice to name them after the tasks they achieve, such astest.yml.

Each file has a few elements that are required, but many, many more that are optional. The GitHub Actionsdocumentation is thorough and well-written, so be sure to check it out after you’ve finished reading this tutorial.

There are three main parts that make up the bulk of a workflow file:triggers,jobs, andsteps. You’ll cover these in the next sections.

Workflow Triggers

A trigger is an event that causes a workflow to run. There are many kinds of triggers. The most common ones are those that occur on a:

• Pull request
• Pushed commit to thedefault branch
• Tagged commit
• Manual trigger
• Request byanother workflow
• New issue being opened

You might also want to restrict triggers further by limiting it to a specific branch or set of files. Here’s a simple example of a trigger that runs a workflow onany push to the main branch:

on:push:branches:-main

For detailed information about the triggers not covered in this tutorial, you can check out the officialdocumentation.

Now that you know how events trigger workflows, it’s time to explore the next component of a workflow file: jobs.

Workflow Jobs

Each workflow has a singlejobs section, which is the container for the meat and potatoes of the workflow. A workflow can include one or more jobs that it will run, and each job can contain one or more steps.

Here’s an example of what this section would look likewithout any steps:

# ...jobs:my_first_job:name:My first jobmy_second_job:name:My second job

When you’re creating a job, the first thing to do is define the runner you want to use to run your job. Arunner is a GitHub-hosted virtual machine (VM) that executes your jobs for you. GitHub will provision and de-provision the VM so you don’t have to worry about maintaining any infrastructure for your CI/CD.

There are multiple supported operating systems available. You can find thefull list of GitHub-hosted runners in the documentation.

Defining a runner takes as little as a single line of YAML:

# ...jobs:my_first_job:name:My first jobruns-on:ubuntu-latest# ...my_second_job:name:My second jobruns-on:windows-latest# ...

In the above example,my_first_job will run inside an Ubuntu VM, andmy_second_job will run inside a Windows VM. Both use the-latest suffix in this case, but you could also specify the exact version of the operating system—for example,ubuntu-20.24, as long as it’s asupported version.

Workflow Steps

Steps are the main part of a job. As you’ve probably guessed, the steps declare the actions that need to be performed when executing the workflow. This can include tasks such asinstalling Python,running tests,linting your code, or using another GitHub action.

Just like your Python code, common and repeatable tasks can be abstracted away into separate workflows and reused. This means you can and should use other people’s GitHub Actions in your own workflows, similar to how you would when importing a Python library, to save you time reimplementing that functionality.

In the next section, you’ll see how you can use other GitHub Actions and how to find them.

Exploring the GitHub Marketplace

TheGitHub Marketplace is an online repository of all the actions people can use in their own workflows. GitHub, third-party vendors, and individuals build and maintain these GitHub Actions. Anyone can use the GitHub Action template to create their own action and host it in the marketplace.

This has led to a vast array of GitHub Actions available for nearly every type of task automation imaginable. All actions in the GitHub Marketplace are open source and free to use.

In the next section, you’ll look at two GitHub Actions that you’ll use for every Python project.

Including Actions in Workflows

Every Python-based workflow you create needs to not only check out your current repository into the workflow environment but also install and set up Python. Fortunately, GitHub has official GitHub Actions to help with both tasks:

# ...jobs:my_first_job:name:My first jobruns-on:ubuntu-lateststeps:-uses:actions/checkout@v4-uses:actions/setup-python@v5with:python-version:"3.13"-run:python -m pip install -r requirements.txt

In the example above, you can see that the first step insteps is to usethe officialcheckout action. This action checks out the code from your repository into the current GitHub workspace, allowing your workflow to access it. The@4 followingcheckout is a version specifier, indicating which version of the action to use. As of now, the latest version is v4.2.2, so you can refer to it using this syntax to specify the latest major version.

The second step of this example sets up Python in the environment. Again, this example uses the official GitHub Action to do this because of its ongoing support and development. Most actions, if not all, have extra configurations you can add to the step.

TheSetup Python action documentation contains the complete list of configurations. For now, the minimum you need to install Python into your workflow environment is to declare which version of Python you wish to install.

In the final step of the example, you use therun command. This command allows you to execute anybash orpowershell command, depending on which runner you’re using for the step. In this case, you’re installing the project’s dependencies from therequirements file.

Hopefully, you can see how powerful GitHub Actions can be. With very little code and effort, you have a reproducible way to set up an environment that’s ready for building, testing, and deploying your Python project.

You now have a basic understanding of the structure of a workflow file and how you can create your first workflow for a project. In the next section, you’ll do just that with a real-world example.

Testing on Multiple Versions of Python

In the linting workflow, you used thesetup-python action in yoursteps to set upPython 3.13 in the Ubuntu instance, which looked like this:

# ...jobs:lint:runs-on:ubuntu-lateststeps:-uses:actions/checkout@v4-uses:actions/setup-python@v5with:python-version:"3.13"cache:"pip"# ...# ...

Unfortunately, you can’t just add a list of Python versions topython-version and be done. What you need is astrategy matrix to test on multiple versions of Python.

To quote the official documentation:

A matrix strategy lets you use variables in a single job definition to automatically create multiple job runs that are based on the combinations of the variables. For example, you can use a matrix strategy to test your code in multiple versions of a language or on multiple operating systems. (Source)

In short, whatever variables you define in yourmatrix will run the same steps in the job, but using those variables. Here, you want to run on different versions of Python, but you could also use this to run or build your code on different operating systems.

Declaring a strategy is relatively straightforward. Before defining your steps but as part of your job, you can define your required strategy:

 1name:Run Tests 2 3on: 4push: 5branches: 6-master 7pull_request: 8branches: 9-master10workflow_call:11workflow_dispatch:1213jobs:14testing:15runs-on:ubuntu-latest16strategy:17matrix:18python-version:["3.9","3.10","3.11","3.12","3.13"]

As you can see, you’re declaring a variablepython-version, which is an array of version numbers. Great, this is the first part done! The second part is to tell thesetup-python action that you want to use these versions using a specialvariable syntax:

 1name:Run Tests 2 3on: 4push: 5branches: 6-master 7pull_request: 8branches: 9-master10workflow_call:11workflow_dispatch:1213jobs:14testing:15runs-on:ubuntu-latest16strategy:17matrix:18python-version:["3.9","3.10","3.11","3.12","3.13"]1920steps:21-uses:actions/checkout@v422-name:Set up Python ${{ matrix.python-version }}23uses:actions/setup-python@v524with:25python-version:${{ matrix.python-version }}26cache:"pip"

The Python setup step of the workflow now has two changes. The first is the addedname to the step. As you learned earlier, this isn’t required but it will help you identify which Python version failed by referencing the Python version in the step’s name. This is helpful, given that this step will run for five different versions of Python.

The second change is that instead of hard coding the version number into thewith: python-version part ofsetup-python, you can now refer to thepython-version defined in the matrix.

GitHub has a few specialcontexts that you can access as part of your workflows. Matrix is one of these. By defining the matrix as part of the strategy,python-version has now become a property of the matrix context. This means that you can access any variable defined as part of the matrix with the dot (.) syntax, for example,matrix.python-version.

Although this isn’t something that needs to be done for Real Python Reader, you could do the same with different OS versions. For example:

strategy:matrix:os:[ubuntu-latest,windows-latest]

You could then use the same dot notation to access theos variable you defined in the matrix withmatrix.os.

Now that you know how to use a matrix to run your steps declaratively using a different version of Python, it’s time to complete the testing workflow in full.

Finalizing the Testing Workflow

There are just a few more steps needed in order to finish the workflow. Now that Python is installed, the workflow will need to install the developer dependencies and then finally runpytest.

The Real Python Reader package uses apyproject.toml configuration file for declaring its dependencies. It also has optional developer dependencies, which includepytest. You can install them the same way you installed Ruff earlier, using therun command:

 1name:Run Tests 2 3on: 4push: 5branches: 6-master 7pull_request: 8branches: 9-master10workflow_call:11workflow_dispatch:1213jobs:14testing:15runs-on:ubuntu-latest16strategy:17matrix:18python-version:["3.9","3.10","3.11","3.12","3.13"]1920steps:21-uses:actions/checkout@v422-name:Set up Python ${{ matrix.python-version }}23uses:actions/setup-python@v524with:25python-version:${{ matrix.python-version }}26cache:"pip"2728-name:Install dependencies29run:|30python -m pip install --upgrade pip31python -m pip install .[dev]

This step is all you need to install the required dependencies. The only remaining step is to runpytest:

 1name:Run Tests 2 3on: 4push: 5branches: 6-master 7pull_request: 8branches: 9-master10workflow_call:11workflow_dispatch:1213jobs:14testing:15runs-on:ubuntu-latest16strategy:17matrix:18python-version:["3.9","3.10","3.11","3.12","3.13"]1920steps:21-uses:actions/checkout@v422-name:Set up Python ${{ matrix.python-version }}23uses:actions/setup-python@v524with:25python-version:${{ matrix.python-version }}26cache:"pip"2728-name:Install dependencies29run:|30python -m pip install --upgrade pip31python -m pip install .[dev]3233-name:Run Pytest34run:pytest

At this point, you have both a linting and testing workflow that are triggered whenever a PR or push event happens on master. Next, you’ll turn your attention to the CD part of CI/CD, and learn how you can automatically publish a package to PyPI.

Setting Up and Building the Package

As with the past two workflows, the first step is to define the triggers for the workflow. You’ve seen some common triggers that revolve around typical developer workflows, but automatically releasing with every new PR or push to the main branch isn’t ideal for Real Python Reader.

It makes more sense to bump the version of the package after several pull requests, bug fixes, or after adding new features. The modern way of triggering such a release after a version bump is to use the developer’s best friend,Git.

Git allows you to tag a commit to denote a notable point in time in the software’s development. This is often the tool of choice to define a new release. GitHub Actions have built-in support for usingGit tags as triggers through thetags keyword:

 1name:Publish to PyPI 2on: 3push: 4tags: 5-"*.*.*"

As you can see here, triggers also support glob patterns. So an asterisk (*) can match any character in a sequence. The pattern outlined above will match any character followed by a decimal point (.), another character, another decimal point, and finally, another character.

This means that 1.0.0 is a valid match, as is 2.5.60. This matches thesemantic versioning used by Real Python Reader. You could also usev*.*.* instead if you prefer. So, your Git tags should start with av, which stands forversion. For example, v1.0.0 would be a valid tag.

In order to trigger this workflow, you’d tag a commit with the version name:

$gittag-a"1.0.0"-m"1.0.0"$gitpush--tags

Pushing your new tag to GitHub will then trigger this workflow. Next, you’ll set up the environment and install the dependencies:

 1name:Publish to PyPI 2on: 3push: 4tags: 5-"*.*.*" 6 7jobs: 8publish: 9runs-on:ubuntu-latest10steps:11-uses:actions/checkout@v412-name:Set up Python13uses:actions/setup-python@v514with:15python-version:"3.13"1617-name:Install dependencies18run:|19python -m pip install --upgrade pip20python -m pip install .[build]2122-name:Build package23run:python -m build

First, you define thepublish job and install Python 3.13 into an Ubuntu VM. The next step installs the build dependencies of Real Python Reader. In the last step, you use the samerun command you’ve used before, but this time, instead of running Ruff orpytest, you’ll build the Real Python Reader package. By default,build will place the distribution files in a folder calleddist.

Excellent! You’ve implemented the first two main parts of the workflow plan. Before you can deploy to PyPI, you should know how to keep yourPyPI API token secure.

Keeping Your Secrets Secure

As you learned earlier, workflows get access to special contexts likematrix. Another context that all workflows have access to is thesecrets context. By storing sensitive data as arepository secret, you can ensure you never accidentally leak API keys, passwords, or other credentials. Your workflow can access those sensitive credentials using thesecrets context.

You can add secrets to your repository on the GitHub website. Once you’ve added them, you can’t view or edit them. You can only replace them with a new value. It’s a good idea to review theGitHub documentation to see how to add secrets on the GitHub website. The official docs are continually updated with any UI changes, making them the best source for learning how to use this GitHub feature.

Deploying Your Package

After securing your API key as a GitHub secret, you can access it in the workflow:

 1name:Publish to PyPI 2on: 3push: 4tags: 5-"*.*.*" 6 7jobs: 8publish: 9runs-on:ubuntu-latest10steps:11-uses:actions/checkout@v412-name:Set up Python13uses:actions/setup-python@v514with:15python-version:"3.13"1617-name:Install dependencies18run:|19python -m pip install --upgrade pip20python -m pip install .[build]2122-name:Build package23run:python -m build2425-name:Test publish package26uses:pypa/gh-action-pypi-publish@release/v127with:28user:__token__29password:${{ secrets.PYPI_API_TOKEN }}30repository-url:https://test.pypi.org/legacy/3132-name:Publish package33uses:pypa/gh-action-pypi-publish@release/v134with:35user:__token__36password:${{ secrets.PYPI_API_TOKEN }}

In this step, you get to use the official GitHub Action from thePython Packaging Authority (PyPA), which manages PyPI. This GitHub Action does most of the work and only needs a reference to your PyPI API token. Again, by default, it will look in yourdist folder for any new version of a package to upload.

Rather than using a traditional username and password to authenticate to PyPI, it’s best practice to use a scoped API token instead for automatic releases.

Since you’re using an API token and there’s no username, using__token__ as the username tells the GitHub Action that token authentication is being used. Just like with the previous matrix strategy, you can use dot notation to access the secret context, as insecrets.PYPI_API_TOKEN.

The name of the secret when stored in GitHub doesn’t matter, as long as it makes sense to you. The GitHub secret is namedPYPI_API_TOKEN, so you reference it inside the workflow using that name.

You may have noticed that the workflow includes a test step prior to publishing the package to PyPI. This step is almost identical to the publishing step, with one key difference: you’ll need to provide arepository-url to override the default URL and push the package totest.pypi.org.

Using TestPyPI is an excellent way to ensure that your package is built and versioned correctly. It allows you to identify and address any potential issues that might cause problems when publishing to the main PyPI repository.

If you’re following along with your own fork of the repository and intend to push your version to PyPI, then you’ll need to update the name of the project to a unique name. If you don’t update the project name, you’ll recieve anHTTP 403 error when trying to upload it. This is because you don’t have permission to publish therealpython-reader package to PyPI. Updating the project name will allow you to publish your own version.

As an example, you could add your username as a prefix to the project name:

[build-system]requires=["setuptools>=61.0.0","wheel"]build-backend="setuptools.build_meta"[project]name="username-realpython-reader"# ...

There’s just one more step of the workflow to complete—creating a GitHub release to promote and then sharing the release directly. Before you can do this, you’ll learn about GitHub environment variables.

Accessing GitHub Environment Variables

In order to publish a release to a GitHub repo, a GitHub token is required. You may have used these before if you’ve ever used the GitHub API. Given the security risk of using personal GitHub tokens in workflows, GitHub creates a read-only token in the secrets context by default. This means that you always have access to it if you need it.

In addition, every GitHub runner includes the handyGitHub CLI by default. This makes performing certain tasks, like creating a release, so much simpler. The GitHub CLI hasmany ways to authenticate the user, one of which is by setting an environment variable calledGITHUB_TOKEN.

You may see where this is going. The provided GitHub token can be used to access the CLI and ultimately create a seamless way to create the GitHub release. Here’s what that would look like in the workflow:

 1name:Publish to PyPI 2on: 3push: 4tags: 5-"*.*.*" 6 7jobs: 8publish: 9runs-on:ubuntu-latest10steps:11-uses:actions/checkout@v412-name:Set up Python13uses:actions/setup-python@v514with:15python-version:"3.13"1617-name:Install dependencies18run:|19python -m pip install --upgrade pip20python -m pip install .[build]2122-name:Build package23run:python -m build2425-name:Test publish package26uses:pypa/gh-action-pypi-publish@release/v127with:28user:__token__29password:${{ secrets.PYPI_API_TOKEN }}30repository-url:https://test.pypi.org/legacy/3132-name:Publish package33uses:pypa/gh-action-pypi-publish@release/v134with:35user:__token__36password:${{ secrets.PYPI_API_TOKEN }}3738-name:Create GitHub Release39env:40GITHUB_TOKEN:${{ secrets.GITHUB_TOKEN }}41run:|42gh release create ${{ github.ref_name }} ./dist/* --generate-notes

You’ll see that on lines 39 and 40, the workflow specifically assigns the GitHub token from the secrets context to an environment variable calledGITHUB_TOKEN. Any key values set inenv will be set as environment variables for the current step. This means that when you run the GitHub CLI (gh), it will have access to the token through the assigned environment variable. The GitHub CLI can’t directly access the secrets context itself.

GitHub also lets you access a special context calledgithub. The workflow references theref_name attribute in thegithub context. This is defined in the GitHub docs as follows:

The short ref name of the branch or tag that triggered the workflow run. (Source)

So,github.ref_name will be replaced with the attribute that triggered the workflow, which in this case is the Git tag’s name.

Thegh command above will create arelease with the same name as the tag used to trigger the release, upload all files from./dist, and auto-generate release notes. These release notes include any PRs that developers have merged since they created the last release, giving proper credit to the authors with links and usernames for their contributions.

You may want to add any missing details to the release notes. Remember that releases can be edited after creation if you need to include additional information, such as deprecation notices.

Congratulations! You now have automated linting, testing, and deployment in place. You can tag your latest commit, and the final deployment workflow should run successfully:

Now that the Real Python Reader has a CI/CD pipeline to ensure that any future codebase changes are robust and use readable and consistent code, you can add one more workflow to Real Python Reader. The cherry on the top of our CI/CD cake, so to speak.

In the next section, you’ll learn how to configure Dependabot to automate security and dependency updates.