Automatic "Ready for Review" Github Action
TLDR: We wanted a GitHub Action to automatically assign reviewers and mark a draft pull request as "Ready for review" after our test suite passes. The final code can be found inthis gist here.
At Potloc, our continuous integration process involves, among other things, a GitHub workflow running on eachpush
that tests the code against our full test suite. This check must pass for a pull request to be merged.
Our test suite has gotten to a size where it is difficult to run on a personal computer in a reasonable amount of time, hence our developers usually rely on this GitHub workflow to run the full test suite.
The process looks something like this:
- Push code for a new feature
- Create a new Pull Request in "Draft" mode
- Wait for all the tests to pass
- Mark the Pull Request as "Ready for review" and assign reviewers
Note that we consider it a good practice to wait until tests pass before assigning reviewers in order to prevent notifying them only to realize that some more changes are necessary.
In practice, we have an in-house tool to help us automate most of these tasks through theGitHub CLI, but for a long time we didn't have a way to automatically mark a pull request as "Ready for review" when the all tests passed, meaning we had to wait and periodically check the status of each of our PR.
Inspired by Artur Dryomov's excellent post onAutonomous GitHub Pull Requests, we set out to create a GitHub Action to help us automate this.
Solution:
At the moment of creating the draft pull request, we want to be able to specify what to do in the event that all tests pass.
To achieve this, we will use a tag namedautoready
that we can put on our pull requests to signify that this PR should be automatically marked as "Ready for review" when all tests pass.
In addition, we want to be able to automatically assign reviewers when that happens. For that, we will be using a specific comment format that looks like this:
autoready-reviewers: reviewer1,reviewer2,organization/team1
Our workflow will automatically detect comments like this and assign each of the listed individual or team reviewers.
Implementation
GitHub Workflow configuration
Our workflow should run after each run of ourTest
workflow and use its output status to determine whether or not to mark the pull request as "Ready for review"..github/workflows/ready_for_review.yml
:
name:Ready For Reviewon:workflow_run:workflows:["Test"]branches-ignore:[main]types:-completedjobs:mark_as_ready_for_review:runs-on:self-hostedif:${{ github.event.workflow_run.conclusion == 'success' }}steps:-name:Checkout Codeuses:actions/checkout@v3-name:Mark as Ready for Reviewrun:bash .github/workflows/mark_as_ready_for_review.sh "${{ secrets.ACCESS_TOKEN }}" "${{ join(github.event.workflow_run.pull_requests.*.number) }}"
This will run our custom scriptmark_as_ready_for_review.sh
after eachsuccessful run of theTest
workflow.
Some noteworthy points:
- We need the
Checkout Code
action to get the latest version of thismark_as_ready_for_review.sh
script. - Our script takes a couple of arguments as input:
- AGitHub access token of the "user" on behalf of whom we will be performing these automatic actions. In our case, we have a dedicated bot account for this. We store this value in aGitHub secret
secrets.ACCESS_TOKEN
. - A comma-separated list of all pull request IDs associated with this workflow run. Since a workflow run is attached to a particular commit hash, it is possible that multiple PRs have that same commit hash as
HEAD
.
- AGitHub access token of the "user" on behalf of whom we will be performing these automatic actions. In our case, we have a dedicated bot account for this. We store this value in aGitHub secret
Bash Script
Here is the script dissected and explained (scroll to the bottom for the full script):
#!/bin/bashset-eou pipefail# Make sure we get useful error messages on failure
Our inputs and constants:
TOKEN="${1}"PR_NUMBERS="${2}"LABEL="autoready"# the name of the 'label' on the PR used to detect whether or not this script should runREPO="your-repository"# the name of your repository on GitHubORGANIZATION="potloc"# the name of your GitHub organization or user to which the repository belongs
Then, we want to repeat the whole thing for as many pull requests as have been passed as input:
# Split the numbers string (comma-delimited)forpr_numberin$(echo$PR_NUMBERS |tr",""\n");do
Fetch the labels from the pull request. We will also need the Node ID of the PR to use GitHub's GraphQL API in a later step, so we also grab this at the same time.
# Get the node_id (and labels) from the PR number# - https://docs.github.com/en/graphql/guides/using-global-node-ids# - https://docs.github.com/en/rest/reference/pulls#get-a-pull-requestout=$(curl\--fail\--silent\--show-error\--header"Accept: application/vnd.github.v3+json"\--header"Authorization: token${TOKEN}"\--request"GET"\--url"https://api.github.com/repos/${ORGANIZATION}/${REPO}/pulls/${pr_number}")node_id=$(jq-r'.node_id'<<<$out)contains_label=$(jq"any(.labels[].name ==\"${LABEL}\"; .)"<<<$out)comments_url=$(jq-r".comments_url"<<<$out)# Check if the PR contains the label we wantif["$contains_label"=="true"];then# Continued below
Note that we use
jq
to simplify parsing of the JSON body returned by the GitHub API. This needs to be installed on the workers that will run this Workflow.
If the label exists on the PR, then we can mark is as "Ready for review". This API only exists in GitHub'sGraphQL API, hence the different request. This is where we make use of the previously-retrievednode_id
:
# Mark the PR as ready for reviewcurl\--fail\--silent\--show-error\--header"Content-Type: application/json"\--header"Authorization: token${TOKEN}"\--request"POST"\--data"{\"query\":\"mutation { markPullRequestReadyForReview(input: { pullRequestId:\\\"${node_id}\\\" }) { pullRequest { id } } }\" }"\--url https://api.github.com/graphql
Delete the label to prevent running this script for this PR:
# Remove the labelcurl\--request"DELETE"\--header"Accept: application/vnd.github.v3+json"\--header"Authorization: token${TOKEN}"\--url"https://api.github.com/repos/${ORGANIZATION}/${REPO}/issues/${pr_number}/labels/${LABEL}"
Finally, we want to find which reviewers to assign to this PR. To do this, we fetch all comments on the PR and use a regex to find a comment matching ourautoready-reviewers:
format we defined:
# Get the comments on the PRcomments_out=$(curl\--fail\--silent\--show-error\--header"Content-Type: application/vnd.github.v3+json"\--header"Authorization: token${TOKEN}"\--request"GET"\--url$comments_url)# Look for a comment matching the 'autoready-reviewers: ' pattern# If found, assign the mentionned reviewers to review this PRjq-r".[].body"<<<$comments_out |whileIFS=''readcomment;do if[[$comment=~ autoready-reviewers:[[:space:]]([a-zA-Z0-9,\-\/]+)]];thenall_reviewers=${BASH_REMATCH[1]}# Get the first matching group of the regex (the comma-separated list of reviewers)
Using this list of reviewers, we differentiate between teams (e.g.potloc/devs
) and individuals to assign by looking for the/
character:
# Split the reviewers between teams and individualsreviewers_array=()team_reviewers_array=()forreviewerin$(echo$all_reviewers |tr",""\n");do if[[$reviewer=~[a-zA-Z0-9,\-]+\/[a-zA-Z0-9,\-]+]];then# In the case of a team reviewer, only take the part of the username after the '/':slug_array=(${reviewer//\//})team_slug=${slug_array[1]} team_reviewers_array+=("\"$team_slug\"")elsereviewers_array+=("\"$reviewer\"")fidone# Join the array elements into a single comma-separated string:reviewers=$(IFS=,;echo"${reviewers_array[*]}")team_reviewers=$(IFS=,;echo"${team_reviewers_array[*]}")
The very last step is to make the API call to assign these individual and teams as reviewers to the PR:
# Assign reviewers
curl</span>
--fail</span>
--silent</span>
--show-error</span>
--output /dev/null</span>
--header"Accept: application/vnd.github.v3+json"</span>
--header"Authorization: token${TOKEN}"</span>
--request"POST"</span>
--url"https://api.github.com/repos/${ORGANIZATION}/${REPO}/pulls/${pr_number}/requested_reviewers"</span>
--data"{\"reviewers\":[${reviewers}],\"team_reviewers\":[${team_reviewers}]}"
Conclusion
And that is it! Now, to use this tool we can put theautoready
label on a draft pull request and write a comment in the formautoready-reviewers: reviewer1,reviewer2,organization/team1
.
In practice, at Potloc, we have a little helper in-house tool do these steps for us using theGitHub CLI andtty-prompt
to
ease the selection of reviewers/teams and the formatting of this comment.
And this is what it looks like on GitHub's interface!
Interested in what we do at Potloc? Come join us!We are hiring 🚀
Full code:
https://gist.github.com/jeromepl/02e70f3ea4a4e8103da6f96f14eb213c
Top comments(0)
For further actions, you may consider blocking this person and/orreporting abuse