Posted onApr 1, 2022 • Edited onJan 6, 2023

Automatic "Ready for Review" Github Action

#devops #github

TLDR: We wanted a GitHub Action to automatically assign reviewers and mark a draft pull request as "Ready for review" after our test suite passes. The final code can be found inthis gist here.

At Potloc, our continuous integration process involves, among other things, a GitHub workflow running on eachpush that tests the code against our full test suite. This check must pass for a pull request to be merged.

Our test suite has gotten to a size where it is difficult to run on a personal computer in a reasonable amount of time, hence our developers usually rely on this GitHub workflow to run the full test suite.

The process looks something like this:

Push code for a new feature
Create a new Pull Request in "Draft" mode
Wait for all the tests to pass
Mark the Pull Request as "Ready for review" and assign reviewers

Note that we consider it a good practice to wait until tests pass before assigning reviewers in order to prevent notifying them only to realize that some more changes are necessary.

In practice, we have an in-house tool to help us automate most of these tasks through theGitHub CLI, but for a long time we didn't have a way to automatically mark a pull request as "Ready for review" when the all tests passed, meaning we had to wait and periodically check the status of each of our PR.

Inspired by Artur Dryomov's excellent post onAutonomous GitHub Pull Requests, we set out to create a GitHub Action to help us automate this.

Solution:

At the moment of creating the draft pull request, we want to be able to specify what to do in the event that all tests pass.

To achieve this, we will use a tag namedautoready that we can put on our pull requests to signify that this PR should be automatically marked as "Ready for review" when all tests pass.

In addition, we want to be able to automatically assign reviewers when that happens. For that, we will be using a specific comment format that looks like this:

autoready-reviewers: reviewer1,reviewer2,organization/team1

Our workflow will automatically detect comments like this and assign each of the listed individual or team reviewers.

Implementation

GitHub Workflow configuration

Our workflow should run after each run of ourTest workflow and use its output status to determine whether or not to mark the pull request as "Ready for review".
.github/workflows/ready_for_review.yml:

name:Ready For Reviewon:workflow_run:workflows:["Test"]branches-ignore:[main]types:-completedjobs:mark_as_ready_for_review:runs-on:self-hostedif:${{ github.event.workflow_run.conclusion == 'success' }}steps:-name:Checkout Codeuses:actions/checkout@v3-name:Mark as Ready for Reviewrun:bash .github/workflows/mark_as_ready_for_review.sh "${{ secrets.ACCESS_TOKEN }}" "${{ join(github.event.workflow_run.pull_requests.*.number) }}"

This will run our custom scriptmark_as_ready_for_review.sh after eachsuccessful run of theTest workflow.

Some noteworthy points:

We need theCheckout Code action to get the latest version of thismark_as_ready_for_review.sh script.
Our script takes a couple of arguments as input:
1. AGitHub access token of the "user" on behalf of whom we will be performing these automatic actions. In our case, we have a dedicated bot account for this. We store this value in aGitHub secretsecrets.ACCESS_TOKEN.
2. A comma-separated list of all pull request IDs associated with this workflow run. Since a workflow run is attached to a particular commit hash, it is possible that multiple PRs have that same commit hash asHEAD.

Bash Script

Here is the script dissected and explained (scroll to the bottom for the full script):

#!/bin/bashset-eou pipefail# Make sure we get useful error messages on failure

Our inputs and constants:

TOKEN="${1}"PR_NUMBERS="${2}"LABEL="autoready"# the name of the 'label' on the PR used to detect whether or not this script should runREPO="your-repository"# the name of your repository on GitHubORGANIZATION="potloc"# the name of your GitHub organization or user to which the repository belongs

Then, we want to repeat the whole thing for as many pull requests as have been passed as input:

# Split the numbers string (comma-delimited)forpr_numberin$(echo$PR_NUMBERS |tr",""\n");do

Fetch the labels from the pull request. We will also need the Node ID of the PR to use GitHub's GraphQL API in a later step, so we also grab this at the same time.

# Get the node_id (and labels) from the PR number# - https://docs.github.com/en/graphql/guides/using-global-node-ids# - https://docs.github.com/en/rest/reference/pulls#get-a-pull-requestout=$(curl\--fail\--silent\--show-error\--header"Accept: application/vnd.github.v3+json"\--header"Authorization: token${TOKEN}"\--request"GET"\--url"https://api.github.com/repos/${ORGANIZATION}/${REPO}/pulls/${pr_number}")node_id=$(jq-r'.node_id'<<<$out)contains_label=$(jq"any(.labels[].name ==\"${LABEL}\"; .)"<<<$out)comments_url=$(jq-r".comments_url"<<<$out)# Check if the PR contains the label we wantif["$contains_label"=="true"];then# Continued below

Note that we usejq to simplify parsing of the JSON body returned by the GitHub API. This needs to be installed on the workers that will run this Workflow.

If the label exists on the PR, then we can mark is as "Ready for review". This API only exists in GitHub'sGraphQL API, hence the different request. This is where we make use of the previously-retrievednode_id:

# Mark the PR as ready for reviewcurl\--fail\--silent\--show-error\--header"Content-Type: application/json"\--header"Authorization: token${TOKEN}"\--request"POST"\--data"{\"query\":\"mutation { markPullRequestReadyForReview(input: { pullRequestId:\\\"${node_id}\\\" }) { pullRequest { id } } }\" }"\--url https://api.github.com/graphql

Delete the label to prevent running this script for this PR:

# Remove the labelcurl\--request"DELETE"\--header"Accept: application/vnd.github.v3+json"\--header"Authorization: token${TOKEN}"\--url"https://api.github.com/repos/${ORGANIZATION}/${REPO}/issues/${pr_number}/labels/${LABEL}"

Finally, we want to find which reviewers to assign to this PR. To do this, we fetch all comments on the PR and use a regex to find a comment matching ourautoready-reviewers: format we defined:

# Get the comments on the PRcomments_out=$(curl\--fail\--silent\--show-error\--header"Content-Type: application/vnd.github.v3+json"\--header"Authorization: token${TOKEN}"\--request"GET"\--url$comments_url)# Look for a comment matching the 'autoready-reviewers: ' pattern# If found, assign the mentionned reviewers to review this PRjq-r".[].body"<<<$comments_out |whileIFS=''readcomment;do  if[[$comment=~ autoready-reviewers:[[:space:]]([a-zA-Z0-9,\-\/]+)]];thenall_reviewers=${BASH_REMATCH[1]}# Get the first matching group of the regex (the comma-separated list of reviewers)

Using this list of reviewers, we differentiate between teams (e.g.potloc/devs) and individuals to assign by looking for the/ character:

# Split the reviewers between teams and individualsreviewers_array=()team_reviewers_array=()forreviewerin$(echo$all_reviewers |tr",""\n");do  if[[$reviewer=~[a-zA-Z0-9,\-]+\/[a-zA-Z0-9,\-]+]];then# In the case of a team reviewer, only take the part of the username after the '/':slug_array=(${reviewer//\//})team_slug=${slug_array[1]}    team_reviewers_array+=("\"$team_slug\"")elsereviewers_array+=("\"$reviewer\"")fidone# Join the array elements into a single comma-separated string:reviewers=$(IFS=,;echo"${reviewers_array[*]}")team_reviewers=$(IFS=,;echo"${team_reviewers_array[*]}")

The very last step is to make the API call to assign these individual and teams as reviewers to the PR:

# Assign reviewers
curl</span>
--fail</span>
--silent</span>
--show-error</span>
--output /dev/null</span>
--header"Accept: application/vnd.github.v3+json"</span>
--header"Authorization: token${TOKEN}"</span>
--request"POST"</span>
--url"https://api.github.com/repos/${ORGANIZATION}/${REPO}/pulls/${pr_number}/requested_reviewers"</span>
--data"{\"reviewers\":[${reviewers}],\"team_reviewers\":[${team_reviewers}]}"

Conclusion

And that is it! Now, to use this tool we can put theautoready label on a draft pull request and write a comment in the formautoready-reviewers: reviewer1,reviewer2,organization/team1.

In practice, at Potloc, we have a little helper in-house tool do these steps for us using theGitHub CLI andtty-prompt to
ease the selection of reviewers/teams and the formatting of this comment.

And this is what it looks like on GitHub's interface!