- Notifications
You must be signed in to change notification settings - Fork1.1k
feat: add AI documentation check workflow for pull requests#20328
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Closed
+457 −0
Closed
Changes fromall commits
Commits
Show all changes
13 commits Select commitHold shift + click to select a range
26204a6 feat: add AI documentation check workflow for pull requests
david-fraleyc20d661 fix: use heredoc syntax for multi-line strings in workflow
david-fraley2ff5fd7 refactor: align workflow structure with triage pattern
david-fraleyb0b3dec fix: use consistent multi-line string formatting throughout workflow
david-fraley5e4cc71 fix: correct YAML indentation in heredoc blocks
david-fraley61af44b refactor: simplify prompt to reference PR instead of embedding full diff
david-fraleybfbe7d8 feat: add execution time tracking to documentation check
david-fraley58be1b3 feat: extract only final analysis using delimiter markers
david-fraley89cfc5e feat: preserve original comment content when updating with results
david-fraley5beb53c feat: add intermediate comment update with task link
david-fraley06dd555 fix: use stable task name for documentation checks
david-fraley51df125 fix: remove task cleanup to preserve task logs
david-fraleya233591 refactor: use gh pr comment --edit-last for updates
david-fraleyFile filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,310 @@ | ||
| name: AI Documentation Updates Automation | ||
| on: | ||
| pull_request: | ||
| types: | ||
| - opened | ||
| - synchronize | ||
| - reopened | ||
| workflow_dispatch: | ||
| inputs: | ||
| pr_number: | ||
| description: "Pull Request number to process" | ||
| required: true | ||
| type: number | ||
| jobs: | ||
| documentation-check: | ||
| name: Check Documentation Updates with Claude Code | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| env: | ||
| CODER_URL: ${{ secrets.TRAIAGE_CODER_URL }} | ||
| CODER_SESSION_TOKEN: ${{ secrets.TRAIAGE_CODER_SESSION_TOKEN }} | ||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
| actions: write | ||
| steps: | ||
| - name: Determine Inputs | ||
| id: determine-inputs | ||
| if: always() | ||
| env: | ||
| GITHUB_ACTOR: ${{ github.actor }} | ||
| GITHUB_EVENT_NAME: ${{ github.event_name }} | ||
| GITHUB_EVENT_PR_NUMBER: ${{ github.event.pull_request.number }} | ||
| GITHUB_EVENT_USER_ID: ${{ github.event.sender.id }} | ||
| GITHUB_EVENT_USER_LOGIN: ${{ github.event.sender.login }} | ||
| INPUTS_PR_NUMBER: ${{ inputs.pr_number }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| # For workflow_dispatch, use the actor who triggered it | ||
| # For pull_request events, use the PR author | ||
| if [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then | ||
| if ! GITHUB_USER_ID=$(gh api "users/${GITHUB_ACTOR}" --jq '.id'); then | ||
| echo "::error::Failed to get GitHub user ID for actor ${GITHUB_ACTOR}" | ||
| exit 1 | ||
| fi | ||
| echo "Using workflow_dispatch actor: ${GITHUB_ACTOR} (ID: ${GITHUB_USER_ID})" | ||
| echo "github_user_id=${GITHUB_USER_ID}" >> "${GITHUB_OUTPUT}" | ||
| echo "github_username=${GITHUB_ACTOR}" >> "${GITHUB_OUTPUT}" | ||
| echo "Using PR number: ${INPUTS_PR_NUMBER}" | ||
| echo "pr_number=${INPUTS_PR_NUMBER}" >> "${GITHUB_OUTPUT}" | ||
| exit 0 | ||
| elif [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then | ||
| GITHUB_USER_ID=${GITHUB_EVENT_USER_ID} | ||
| echo "Using PR author: ${GITHUB_EVENT_USER_LOGIN} (ID: ${GITHUB_USER_ID})" | ||
| echo "github_user_id=${GITHUB_USER_ID}" >> "${GITHUB_OUTPUT}" | ||
| echo "github_username=${GITHUB_EVENT_USER_LOGIN}" >> "${GITHUB_OUTPUT}" | ||
| echo "Using PR number: ${GITHUB_EVENT_PR_NUMBER}" | ||
| echo "pr_number=${GITHUB_EVENT_PR_NUMBER}" >> "${GITHUB_OUTPUT}" | ||
| exit 0 | ||
| else | ||
| echo "::error::Unsupported event type: ${GITHUB_EVENT_NAME}" | ||
| exit 1 | ||
| fi | ||
| - name: Verify push access | ||
| env: | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| GITHUB_USERNAME: ${{ steps.determine-inputs.outputs.github_username }} | ||
| GITHUB_USER_ID: ${{ steps.determine-inputs.outputs.github_user_id }} | ||
| run: | | ||
| # Query the actor's permission on this repo | ||
| can_push="$(gh api "/repos/${GITHUB_REPOSITORY}/collaborators/${GITHUB_USERNAME}/permission" --jq '.user.permissions.push')" | ||
| if [[ "${can_push}" != "true" ]]; then | ||
| echo "::error title=Access Denied::${GITHUB_USERNAME} does not have push access to ${GITHUB_REPOSITORY}" | ||
| exit 1 | ||
| fi | ||
| - name: Post initial comment | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR_NUMBER: ${{ steps.determine-inputs.outputs.pr_number }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| RUN_ID: ${{ github.run_id }} | ||
| run: | | ||
| COMMENT_BODY=$(cat <<EOF | ||
| 🤖 **Documentation Check Started** | ||
| Analyzing PR changes to determine if documentation updates are needed... | ||
| [View workflow run](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${RUN_ID}) | ||
| EOF | ||
| ) | ||
| gh pr comment "${PR_NUMBER}" --body "${COMMENT_BODY}" --repo "${GITHUB_REPOSITORY}" --create-if-none --edit-last | ||
| - name: Download and install Coder binary | ||
| shell: bash | ||
| env: | ||
| CODER_URL: ${{ secrets.TRAIAGE_CODER_URL }} | ||
| run: | | ||
| if [ "${{ runner.arch }}" == "ARM64" ]; then | ||
| ARCH="arm64" | ||
| else | ||
| ARCH="amd64" | ||
| fi | ||
| mkdir -p "${HOME}/.local/bin" | ||
| curl -fsSL --compressed "$CODER_URL/bin/coder-linux-${ARCH}" -o "${HOME}/.local/bin/coder" | ||
| chmod +x "${HOME}/.local/bin/coder" | ||
| export PATH="$HOME/.local/bin:$PATH" | ||
| coder version | ||
| coder whoami | ||
| echo "$HOME/.local/bin" >> "${GITHUB_PATH}" | ||
| - name: Get Coder username from GitHub actor | ||
| id: get-coder-username | ||
| env: | ||
| CODER_SESSION_TOKEN: ${{ secrets.TRAIAGE_CODER_SESSION_TOKEN }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| GITHUB_USER_ID: ${{ steps.determine-inputs.outputs.github_user_id }} | ||
| run: | | ||
| user_json=$( | ||
| coder users list --github-user-id="${GITHUB_USER_ID}" --output=json | ||
| ) | ||
| coder_username=$(jq -r 'first | .username' <<< "$user_json") | ||
| [[ -z "${coder_username}" || "${coder_username}" == "null" ]] && echo "No Coder user with GitHub user ID ${GITHUB_USER_ID} found" && exit 1 | ||
| echo "coder_username=${coder_username}" >> "${GITHUB_OUTPUT}" | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
| fetch-depth: 0 | ||
| - name: Create Coder task for documentation analysis | ||
| id: create-task | ||
| env: | ||
| CODER_USERNAME: ${{ steps.get-coder-username.outputs.coder_username }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| PR_NUMBER: ${{ steps.determine-inputs.outputs.pr_number }} | ||
| RUN_ID: ${{ github.run_id }} | ||
| TEMPLATE_NAME: "traiage" | ||
| TEMPLATE_PRESET: "Default" | ||
| run: | | ||
| # Fetch PR details using `gh` CLI | ||
| pr_json=$(gh pr view "${PR_NUMBER}" --repo "${GITHUB_REPOSITORY}" --json 'url') | ||
| pr_url=$(echo "${pr_json}" | jq -r '.url') | ||
| # Build comprehensive prompt | ||
| PROMPT=$(cat <<'EOF' | ||
| Analyze PR ${pr_url} for documentation updates. | ||
| ## Task Description | ||
| You are tasked with analyzing code changes in a pull request and determining what documentation updates are needed. Please: | ||
| 1. Use the GitHub CLI or API to fetch the PR details, including: | ||
| - PR title and description | ||
| - List of files changed | ||
| - The actual code diff | ||
| 2. Review the PR description and code changes to understand what was changed | ||
| 3. Examine the existing documentation in the docs/ directory | ||
| 4. Identify any of the following needs: | ||
| - Updates required to existing documentation | ||
| - Documentation that needs to be deprecated | ||
| - New documentation that should be added | ||
| 5. Provide a clear, actionable list of documentation changes needed | ||
| ## PR to Analyze | ||
| ${pr_url} | ||
| ## IMPORTANT: Final Output Format | ||
| After completing your analysis, you MUST end your response with ONLY the following formatted output between the markers: | ||
| ---DOCUMENTATION-ANALYSIS-START--- | ||
| [Your concise bulleted list of recommendations OR "No documentation changes needed"] | ||
| ---DOCUMENTATION-ANALYSIS-END--- | ||
| The content between these markers should be: | ||
| - A simple bulleted list with links to docs that need updating | ||
| - OR a single sentence: "No documentation changes needed - [brief reason]" | ||
| - Maximum 10-15 lines | ||
| - Use markdown formatting | ||
| - Include file paths as links when referencing docs | ||
| Example format: | ||
| ---DOCUMENTATION-ANALYSIS-START--- | ||
| ### Documentation Updates Needed | ||
| - **Update** [docs/admin/templates.md](docs/admin/templates.md) - Add new parameter documentation | ||
| - **Create** docs/guides/new-feature.md - Document the new feature workflow | ||
| - **Deprecate** docs/old-feature.md - Feature has been removed | ||
| ---DOCUMENTATION-ANALYSIS-END--- | ||
| OR: | ||
| ---DOCUMENTATION-ANALYSIS-START--- | ||
| No documentation changes needed - This PR only contains internal refactoring with no user-facing changes. | ||
| ---DOCUMENTATION-ANALYSIS-END--- | ||
| EOF | ||
| ) | ||
| # Expand variables in the prompt | ||
| PROMPT=$(eval "cat <<EOF | ||
| ${PROMPT} | ||
| EOF | ||
| ") | ||
| export PROMPT | ||
| export TASK_NAME="doccheck-pr-${PR_NUMBER}" | ||
| export CONTEXT_KEY="gh-pr-${PR_NUMBER}" | ||
| echo "Creating task: ${CODER_USERNAME}/${TASK_NAME}" | ||
| ./scripts/documentation-check.sh create | ||
| echo "TASK_NAME=${CODER_USERNAME}/${TASK_NAME}" >> "${GITHUB_OUTPUT}" | ||
| echo "TASK_NAME=${CODER_USERNAME}/${TASK_NAME}" >> "${GITHUB_ENV}" | ||
| - name: Update comment with task link | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR_NUMBER: ${{ steps.determine-inputs.outputs.pr_number }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| TASK_NAME: ${{ steps.create-task.outputs.TASK_NAME }} | ||
| RUN_ID: ${{ github.run_id }} | ||
| run: | | ||
| COMMENT_BODY=$(cat <<EOF | ||
| 🤖 **Documentation Check Started** | ||
| Analyzing PR changes to determine if documentation updates are needed... | ||
| [View workflow run](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${RUN_ID}) | ||
| --- | ||
| 🔄 **Analysis in progress...** | ||
| 📋 Task: https://dev.coder.com/tasks/${TASK_NAME} | ||
| EOF | ||
| ) | ||
| gh pr comment "${PR_NUMBER}" --body "${COMMENT_BODY}" --repo "${GITHUB_REPOSITORY}" --edit-last | ||
| - name: Wait for task completion and get results | ||
| id: get-results | ||
| env: | ||
| TASK_NAME: ${{ steps.create-task.outputs.TASK_NAME }} | ||
| run: | | ||
| START_TIME=$(date +%s) | ||
| echo "Waiting for task to complete..." | ||
| ./scripts/documentation-check.sh wait | ||
| echo "Getting task results..." | ||
| TASK_OUTPUT=$(./scripts/documentation-check.sh summary) | ||
| END_TIME=$(date +%s) | ||
| DURATION=$((END_TIME - START_TIME)) | ||
| # Convert to minutes and seconds | ||
| MINUTES=$((DURATION / 60)) | ||
| SECONDS=$((DURATION % 60)) | ||
| if [ $MINUTES -gt 0 ]; then | ||
| DURATION_STR="${MINUTES}m ${SECONDS}s" | ||
| else | ||
| DURATION_STR="${SECONDS}s" | ||
| fi | ||
| # Save output to file for next step | ||
| echo "${TASK_OUTPUT}" > /tmp/task_output.txt | ||
| echo "${DURATION_STR}" > /tmp/task_duration.txt | ||
| echo "Task completed successfully in ${DURATION_STR}" | ||
| - name: Update PR comment with results | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR_NUMBER: ${{ steps.determine-inputs.outputs.pr_number }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} | ||
| TASK_NAME: ${{ steps.create-task.outputs.TASK_NAME }} | ||
| RUN_ID: ${{ github.run_id }} | ||
| run: | | ||
| TASK_OUTPUT=$(cat /tmp/task_output.txt) | ||
| DURATION=$(cat /tmp/task_duration.txt) | ||
| COMMENT_BODY=$(cat <<EOF | ||
| 🤖 **Documentation Check Started** | ||
| Analyzing PR changes to determine if documentation updates are needed... | ||
| [View workflow run](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${RUN_ID}) | ||
| --- | ||
| ${TASK_OUTPUT} | ||
| --- | ||
| ⏱️ Execution time: **${DURATION}** | ||
| 📋 Task: https://dev.coder.com/tasks/${TASK_NAME} | ||
| 🔗 [View workflow run](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${RUN_ID}) | ||
| EOF | ||
| ) | ||
| gh pr comment "${PR_NUMBER}" --body "${COMMENT_BODY}" --repo "${GITHUB_REPOSITORY}" --edit-last | ||
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.