In this PR you will find a conformance test to check this server against Anthropic'slist/tools
method. This is gated by aconformance
build tag because it's really a point in time test that we would expect to remove later. It intentionally ignores any fields nameddescription
because there were too many minor differences in text.
--- FAIL: TestListTools (0.60s) conformance_test.go:123: waiting for anthropic server to start... conformance_test.go:123: waiting for github server to start... conformance_test.go:36: sending anthropic: {"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}} conformance_test.go:36: received from anthropic: {"result":{"tools":[{"name":"create_or_update_file","description":"Create or update a single file in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"path":{"type":"string","description":"Path where to create/update the file"},"content":{"type":"string","description":"Content of the file"},"message":{"type":"string","description":"Commit message"},"branch":{"type":"string","description":"Branch to create/update the file in"},"sha":{"type":"string","description":"SHA of the file being replaced (required when updating existing files)"}},"required":["owner","repo","path","content","message","branch"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"search_repositories","description":"Search for GitHub repositories","inputSchema":{"type":"object","properties":{"query":{"type":"string","description":"Search query (see GitHub search syntax)"},"page":{"type":"number","description":"Page number for pagination (default: 1)"},"perPage":{"type":"number","description":"Number of results per page (default: 30, max: 100)"}},"required":["query"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"create_repository","description":"Create a new GitHub repository in your account","inputSchema":{"type":"object","properties":{"name":{"type":"string","description":"Repository name"},"description":{"type":"string","description":"Repository description"},"private":{"type":"boolean","description":"Whether the repository should be private"},"autoInit":{"type":"boolean","description":"Initialize with README.md"}},"required":["name"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"get_file_contents","description":"Get the contents of a file or directory from a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"path":{"type":"string","description":"Path to the file or directory"},"branch":{"type":"string","description":"Branch to get contents from"}},"required":["owner","repo","path"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"push_files","description":"Push multiple files to a GitHub repository in a single commit","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"branch":{"type":"string","description":"Branch to push to (e.g., 'main' or 'master')"},"files":{"type":"array","items":{"type":"object","properties":{"path":{"type":"string","description":"Path where to create the file"},"content":{"type":"string","description":"Content of the file"}},"required":["path","content"],"additionalProperties":false},"description":"Array of files to push"},"message":{"type":"string","description":"Commit message"}},"required":["owner","repo","branch","files","message"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"create_issue","description":"Create a new issue in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"title":{"type":"string","description":"Issue title"},"body":{"type":"string","description":"Issue body/description"},"assignees":{"type":"array","items":{"type":"string"},"description":"Array of usernames to assign"},"labels":{"type":"array","items":{"type":"string"},"description":"Array of label names"},"milestone":{"type":"number","description":"Milestone number to assign"}},"required":["owner","repo","title"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"create_pull_request","description":"Create a new pull request in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"title":{"type":"string","description":"Pull request title"},"body":{"type":"string","description":"Pull request body/description"},"head":{"type":"string","description":"The name of the branch where your changes are implemented"},"base":{"type":"string","description":"The name of the branch you want the changes pulled into"},"draft":{"type":"boolean","description":"Whether to create the pull request as a draft"},"maintainer_can_modify":{"type":"boolean","description":"Whether maintainers can modify the pull request"}},"required":["owner","repo","title","head","base"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"fork_repository","description":"Fork a GitHub repository to your account or specified organization","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"organization":{"type":"string","description":"Optional: organization to fork to (defaults to your personal account)"}},"required":["owner","repo"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"create_branch","description":"Create a new branch in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"branch":{"type":"string","description":"Name for the new branch"},"from_branch":{"type":"string","description":"Optional: source branch to create from (defaults to the repository's default branch)"}},"required":["owner","repo","branch"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"list_commits","description":"Get list of commits of a branch in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"page":{"type":"number","description":"Page number for pagination (default: 1)"},"perPage":{"type":"number","description":"Number of results per page (default: 30, max: 100)"},"sha":{"type":"string","description":"SHA of the file being replaced (required when updating existing files)"}},"required":["owner","repo"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"list_issues","description":"List issues in a GitHub repository with filtering options","inputSchema":{"type":"object","properties":{"owner":{"type":"string"},"repo":{"type":"string"},"state":{"type":"string","enum":["open","closed","all"]},"labels":{"type":"array","items":{"type":"string"}},"sort":{"type":"string","enum":["created","updated","comments"]},"direction":{"type":"string","enum":["asc","desc"]},"since":{"type":"string"},"page":{"type":"number"},"per_page":{"type":"number"}},"required":["owner","repo"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"update_issue","description":"Update an existing issue in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"type":"string"},"repo":{"type":"string"},"issue_number":{"type":"number"},"title":{"type":"string"},"body":{"type":"string"},"state":{"type":"string","enum":["open","closed"]},"labels":{"type":"array","items":{"type":"string"}},"assignees":{"type":"array","items":{"type":"string"}},"milestone":{"type":"number"}},"required":["owner","repo","issue_number"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"add_issue_comment","description":"Add a comment to an existing issue","inputSchema":{"type":"object","properties":{"owner":{"type":"string"},"repo":{"type":"string"},"issue_number":{"type":"number"},"body":{"type":"string"}},"required":["owner","repo","issue_number","body"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"search_code","description":"Search for code across GitHub repositories","inputSchema":{"type":"object","properties":{"q":{"type":"string","description":"Search query. See GitHub code search syntax: https://docs.github.com/en/search-github/searching-on-github/searching-code"},"order":{"type":"string","enum":["asc","desc"],"description":"Sort order (asc or desc)"},"per_page":{"type":"number","minimum":1,"maximum":100,"description":"Results per page (max 100)"},"page":{"type":"number","minimum":1,"description":"Page number"}},"required":["q"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"search_issues","description":"Search for issues and pull requests across GitHub repositories","inputSchema":{"type":"object","properties":{"q":{"type":"string","description":"Search query. See GitHub issues search syntax: https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests"},"sort":{"type":"string","enum":["comments","reactions","reactions-+1","reactions--1","reactions-smile","reactions-thinking_face","reactions-heart","reactions-tada","interactions","created","updated"],"description":"Sort field"},"order":{"type":"string","enum":["asc","desc"],"description":"Sort order (asc or desc)"},"per_page":{"type":"number","minimum":1,"maximum":100,"description":"Results per page (max 100)"},"page":{"type":"number","minimum":1,"description":"Page number"}},"required":["q"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"search_users","description":"Search for users on GitHub","inputSchema":{"type":"object","properties":{"q":{"type":"string","description":"Search query. See GitHub users search syntax: https://docs.github.com/en/search-github/searching-on-github/searching-users"},"sort":{"type":"string","enum":["followers","repositories","joined"],"description":"Sort field"},"order":{"type":"string","enum":["asc","desc"],"description":"Sort order (asc or desc)"},"per_page":{"type":"number","minimum":1,"maximum":100,"description":"Results per page (max 100)"},"page":{"type":"number","minimum":1,"description":"Page number"}},"required":["q"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}},{"name":"get_issue","description":"Get details of a specific issue in a GitHub repository.","inputSchema":{"type":"object","properties":{"owner":{"type":"string","description":"Repository owner (username or organization)"},"repo":{"type":"string","description":"Repository name"},"issue_number":{"type":"number","description":"Issue number"}},"required":["owner","repo","issue_number"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}}]},"jsonrpc":"2.0","id":1} conformance_test.go:36: sending github: {"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}} conformance_test.go:36: received from github: {"jsonrpc":"2.0","id":1,"result":{"tools":[{"description":"Add a comment to an existing issue","inputSchema":{"type":"object","properties":{"body":{"description":"Comment text","type":"string"},"issue_number":{"description":"Issue number to comment on","type":"number"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","issue_number","body"]},"name":"add_issue_comment"},{"description":"Create a new branch in a GitHub repository","inputSchema":{"type":"object","properties":{"branch":{"description":"Name for new branch","type":"string"},"from_branch":{"description":"Source branch (defaults to repo default)","type":"string"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","branch"]},"name":"create_branch"},{"description":"Create a new issue in a GitHub repository","inputSchema":{"type":"object","properties":{"assignees":{"description":"Usernames to assign to this issue","items":{"type":"string"},"type":"array"},"body":{"description":"Issue body content","type":"string"},"labels":{"description":"Labels to apply to this issue","items":{"type":"string"},"type":"array"},"milestone":{"description":"New milestone number","type":"number"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"},"title":{"description":"Issue title","type":"string"}},"required":["owner","repo","title"]},"name":"create_issue"},{"description":"Create or update a single file in a GitHub repository","inputSchema":{"type":"object","properties":{"branch":{"description":"Branch to create/update the file in","type":"string"},"content":{"description":"Content of the file","type":"string"},"message":{"description":"Commit message","type":"string"},"owner":{"description":"Repository owner (username or organization)","type":"string"},"path":{"description":"Path where to create/update the file","type":"string"},"repo":{"description":"Repository name","type":"string"},"sha":{"description":"SHA of file being replaced (for updates)","type":"string"}},"required":["owner","repo","path","content","message","branch"]},"name":"create_or_update_file"},{"description":"Create a new pull request in a GitHub repository","inputSchema":{"type":"object","properties":{"base":{"description":"Branch to merge into","type":"string"},"body":{"description":"PR description","type":"string"},"draft":{"description":"Create as draft PR","type":"boolean"},"head":{"description":"Branch containing changes","type":"string"},"maintainer_can_modify":{"description":"Allow maintainer edits","type":"boolean"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"},"title":{"description":"PR title","type":"string"}},"required":["owner","repo","title","head","base"]},"name":"create_pull_request"},{"description":"Create a review on a pull request","inputSchema":{"type":"object","properties":{"body":{"description":"Review comment text","type":"string"},"comments":{"description":"Line-specific comments array of objects, each object with path (string), position (number), and body (string)","items":{"additionalProperties":false,"properties":{"body":{"description":"comment body","type":"string"},"path":{"description":"path to the file","type":"string"},"position":{"description":"line number in the file","type":"number"}},"required":["path","position","body"],"type":"object"},"type":"array"},"commitId":{"description":"SHA of commit to review","type":"string"},"event":{"description":"Review action ('APPROVE', 'REQUEST_CHANGES', 'COMMENT')","type":"string"},"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber","event"]},"name":"create_pull_request_review"},{"description":"Create a new GitHub repository in your account","inputSchema":{"type":"object","properties":{"autoInit":{"description":"Initialize with README","type":"boolean"},"description":{"description":"Repository description","type":"string"},"name":{"description":"Repository name","type":"string"},"private":{"description":"Whether repo should be private","type":"boolean"}},"required":["name"]},"name":"create_repository"},{"description":"Fork a GitHub repository to your account or specified organization","inputSchema":{"type":"object","properties":{"organization":{"description":"Organization to fork to","type":"string"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo"]},"name":"fork_repository"},{"description":"Get details of a specific code scanning alert in a GitHub repository.","inputSchema":{"type":"object","properties":{"alertNumber":{"description":"The number of the alert.","type":"number"},"owner":{"description":"The owner of the repository.","type":"string"},"repo":{"description":"The name of the repository.","type":"string"}},"required":["owner","repo","alertNumber"]},"name":"get_code_scanning_alert"},{"description":"Get the contents of a file or directory from a GitHub repository","inputSchema":{"type":"object","properties":{"branch":{"description":"Branch to get contents from","type":"string"},"owner":{"description":"Repository owner (username or organization)","type":"string"},"path":{"description":"Path to file/directory","type":"string"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","path"]},"name":"get_file_contents"},{"description":"Get details of a specific issue in a GitHub repository.","inputSchema":{"type":"object","properties":{"issue_number":{"description":"The number of the issue.","type":"number"},"owner":{"description":"The owner of the repository.","type":"string"},"repo":{"description":"The name of the repository.","type":"string"}},"required":["owner","repo","issue_number"]},"name":"get_issue"},{"description":"Get details of the authenticated GitHub user. Use this when a request include \"me\", \"my\"...","inputSchema":{"type":"object","properties":{"reason":{"description":"Optional: reason the session was created","type":"string"}}},"name":"get_me"},{"description":"Get details of a specific pull request","inputSchema":{"type":"object","properties":{"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"get_pull_request"},{"description":"Get the review comments on a pull request","inputSchema":{"type":"object","properties":{"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"get_pull_request_comments"},{"description":"Get the list of files changed in a pull request","inputSchema":{"type":"object","properties":{"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"get_pull_request_files"},{"description":"Get the reviews on a pull request","inputSchema":{"type":"object","properties":{"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"get_pull_request_reviews"},{"description":"Get the combined status of all status checks for a pull request","inputSchema":{"type":"object","properties":{"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"get_pull_request_status"},{"description":"List code scanning alerts in a GitHub repository.","inputSchema":{"type":"object","properties":{"owner":{"description":"The owner of the repository.","type":"string"},"ref":{"description":"The Git reference for the results you want to list.","type":"string"},"repo":{"description":"The name of the repository.","type":"string"},"severity":{"description":"Only code scanning alerts with this severity will be returned. Possible values are: critical, high, medium, low, warning, note, error.","type":"string"},"state":{"default":"open","description":"State of the code scanning alerts to list. Set to closed to list only closed code scanning alerts. Default: open","type":"string"}},"required":["owner","repo"]},"name":"list_code_scanning_alerts"},{"description":"Get list of commits of a branch in a GitHub repository","inputSchema":{"type":"object","properties":{"owner":{"description":"Repository owner","type":"string"},"page":{"description":"Page number","type":"number"},"perPage":{"description":"Number of records per page","type":"number"},"repo":{"description":"Repository name","type":"string"},"sha":{"description":"Branch name","type":"string"}},"required":["owner","repo"]},"name":"list_commits"},{"description":"List issues in a GitHub repository with filtering options","inputSchema":{"type":"object","properties":{"direction":{"description":"Sort direction ('asc', 'desc')","enum":["asc","desc"],"type":"string"},"labels":{"description":"Filter by labels","items":{"type":"string"},"type":"array"},"owner":{"description":"Repository owner","type":"string"},"page":{"description":"Page number","type":"number"},"per_page":{"description":"Results per page","type":"number"},"repo":{"description":"Repository name","type":"string"},"since":{"description":"Filter by date (ISO 8601 timestamp)","type":"string"},"sort":{"description":"Sort by ('created', 'updated', 'comments')","enum":["created","updated","comments"],"type":"string"},"state":{"description":"Filter by state ('open', 'closed', 'all')","enum":["open","closed","all"],"type":"string"}},"required":["owner","repo"]},"name":"list_issues"},{"description":"List and filter repository pull requests","inputSchema":{"type":"object","properties":{"base":{"description":"Filter by base branch","type":"string"},"direction":{"description":"Sort direction ('asc', 'desc')","type":"string"},"head":{"description":"Filter by head user/org and branch","type":"string"},"owner":{"description":"Repository owner","type":"string"},"page":{"description":"Page number","type":"number"},"per_page":{"description":"Results per page (max 100)","type":"number"},"repo":{"description":"Repository name","type":"string"},"sort":{"description":"Sort by ('created', 'updated', 'popularity', 'long-running')","type":"string"},"state":{"description":"Filter by state ('open', 'closed', 'all')","type":"string"}},"required":["owner","repo"]},"name":"list_pull_requests"},{"description":"Merge a pull request","inputSchema":{"type":"object","properties":{"commit_message":{"description":"Extra detail for merge commit","type":"string"},"commit_title":{"description":"Title for merge commit","type":"string"},"merge_method":{"description":"Merge method ('merge', 'squash', 'rebase')","type":"string"},"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"merge_pull_request"},{"description":"Push multiple files to a GitHub repository in a single commit","inputSchema":{"type":"object","properties":{"branch":{"description":"Branch to push to","type":"string"},"files":{"description":"Array of file objects to push, each object with path (string) and content (string)","items":{"additionalProperties":false,"properties":{"content":{"description":"file content","type":"string"},"path":{"description":"path to the file","type":"string"}},"required":["path","content"],"type":"object"},"type":"array"},"message":{"description":"Commit message","type":"string"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","branch","files","message"]},"name":"push_files"},{"description":"Search for code across GitHub repositories","inputSchema":{"type":"object","properties":{"order":{"description":"Sort order ('asc' or 'desc')","enum":["asc","desc"],"type":"string"},"page":{"description":"Page number","minimum":1,"type":"number"},"per_page":{"description":"Results per page (max 100)","maximum":100,"minimum":1,"type":"number"},"q":{"description":"Search query using GitHub code search syntax","type":"string"},"sort":{"description":"Sort field ('indexed' only)","type":"string"}},"required":["q"]},"name":"search_code"},{"description":"Search for issues and pull requests across GitHub repositories","inputSchema":{"type":"object","properties":{"order":{"description":"Sort order ('asc' or 'desc')","enum":["asc","desc"],"type":"string"},"page":{"description":"Page number","minimum":1,"type":"number"},"per_page":{"description":"Results per page (max 100)","maximum":100,"minimum":1,"type":"number"},"q":{"description":"Search query using GitHub issues search syntax","type":"string"},"sort":{"description":"Sort field (comments, reactions, created, etc.)","enum":["comments","reactions","reactions-+1","reactions--1","reactions-smile","reactions-thinking_face","reactions-heart","reactions-tada","interactions","created","updated"],"type":"string"}},"required":["q"]},"name":"search_issues"},{"description":"Search for GitHub repositories","inputSchema":{"type":"object","properties":{"page":{"description":"Page number for pagination","type":"number"},"perPage":{"description":"Results per page (max 100)","type":"number"},"query":{"description":"Search query","type":"string"}},"required":["query"]},"name":"search_repositories"},{"description":"Search for GitHub users","inputSchema":{"type":"object","properties":{"order":{"description":"Sort order ('asc' or 'desc')","enum":["asc","desc"],"type":"string"},"page":{"description":"Page number","minimum":1,"type":"number"},"per_page":{"description":"Results per page (max 100)","maximum":100,"minimum":1,"type":"number"},"q":{"description":"Search query using GitHub users search syntax","type":"string"},"sort":{"description":"Sort field (followers, repositories, joined)","enum":["followers","repositories","joined"],"type":"string"}},"required":["q"]},"name":"search_users"},{"description":"Update an existing issue in a GitHub repository","inputSchema":{"type":"object","properties":{"assignees":{"description":"New assignees","items":{"type":"string"},"type":"array"},"body":{"description":"New description","type":"string"},"issue_number":{"description":"Issue number to update","type":"number"},"labels":{"description":"New labels","items":{"type":"string"},"type":"array"},"milestone":{"description":"New milestone number","type":"number"},"owner":{"description":"Repository owner","type":"string"},"repo":{"description":"Repository name","type":"string"},"state":{"description":"New state ('open' or 'closed')","enum":["open","closed"],"type":"string"},"title":{"description":"New title","type":"string"}},"required":["owner","repo","issue_number"]},"name":"update_issue"},{"description":"Update a pull request branch with the latest changes from the base branch","inputSchema":{"type":"object","properties":{"expectedHeadSha":{"description":"The expected SHA of the pull request's HEAD ref","type":"string"},"owner":{"description":"Repository owner","type":"string"},"pullNumber":{"description":"Pull request number","type":"number"},"repo":{"description":"Repository name","type":"string"}},"required":["owner","repo","pullNumber"]},"name":"update_pull_request_branch"}]}} conformance_test.go:254: Error Trace: /Users/williammartin/workspace/github-mcp-server/conformance/conformance_test.go:254 Error: Received unexpected error: inputSchema mismatch for tool "search_code": conformance_test.inputSchema{ Type: "object", Properties: map[string]any{ ... // 2 identical entries "per_page": map[string]any{"maximum": float64(100), "minimum": float64(1), "type": string("number"), ...}, "q": map[string]any{"type": string("string"), ...}, + "sort": map[string]any{"description": string("Sort field ('indexed' only)"), "type": string("string")}, }, Required: {"q"}, } Test: TestListTools Messages: expected the github list tools response to be a subset of the anthropic list tools responseFAILFAIL github.com/github/github-mcp-server/conformance 1.406s
You should be able to see the only difference in the schema after applying the stack of PRs this is on top is the addition of asort
property onsearch_code
.
The stack of PRs this is based on handle the more interesting missing cases, and this one, aside from introducing the test itself adjusts all the casing to conform. Unfortunately, the anthropic implementation is not always consistent see:per_page
vsperPage
, or the occasional snake case likeissue_number
.
Although this PR gives us a baseline conformance, I would suggest following up to make everything camel case since that is what the JSON RPC and Protocol examples from MPC spec lean towards.
Of all the PRs, this is probably the one I care least about getting merged, though I think the test itself has provided value.
Uh oh!
There was an error while loading.Please reload this page.
Description
In this PR you will find a conformance test to check this server against Anthropic's
list/tools
method. This is gated by aconformance
build tag because it's really a point in time test that we would expect to remove later. It intentionally ignores any fields nameddescription
because there were too many minor differences in text.Here is the test output:
You should be able to see the only difference in the schema after applying the stack of PRs this is on top is the addition of a
sort
property onsearch_code
.Reviewer Notes
The stack of PRs this is based on handle the more interesting missing cases, and this one, aside from introducing the test itself adjusts all the casing to conform. Unfortunately, the anthropic implementation is not always consistent see:
per_page
vsperPage
, or the occasional snake case likeissue_number
.Although this PR gives us a baseline conformance, I would suggest following up to make everything camel case since that is what the JSON RPC and Protocol examples from MPC spec lean towards.
Of all the PRs, this is probably the one I care least about getting merged, though I think the test itself has provided value.