Tester MCP
CLI-based tester for verifying that MCP servers work correctly when called directory and by agents
What is Tester MCP?
Tester MCP is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to cli-based tester for verifying that mcp servers work correctly when called directory and by agents
CLI-based tester for verifying that MCP servers work correctly when called directory and by agents
This server falls under the Developer Tools category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- CLI-based tester for verifying that MCP servers work correct
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx testerConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use Tester MCP
MCP Server Tester is an implementation-agnostic CLI tool for automated testing of any MCP server. It provides three distinct test modes: direct MCP tool tests that verify tools work correctly with API calls (no LLM needed), LLM eval tests that verify AI agents can discover and use your tools reliably, and a compliance mode that checks spec adherence as the MCP protocol evolves. It is designed to catch regressions and ensure real-world LLM usability, not just unit-level correctness.
Prerequisites
- Node.js 18+ to run the tester via npx
- An ANTHROPIC_API_KEY environment variable for eval tests that invoke LLMs (not needed for direct tool tests)
- A running MCP server to test, along with its server config JSON
- YAML test definition files for your tools and eval scenarios
Create a server config file for your MCP server under test
Create a JSON file describing how to launch the MCP server you want to test. This is the same format as Claude Desktop's mcpServers config.
{
"mcpServers": {
"my-server": {
"command": "node",
"args": ["./my-server.js"]
}
}
}Discover available tools in your server
Use the MCP inspector to list all tools exposed by your server so you know what to write tests for.
npx -y @modelcontextprotocol/inspector --cli \
--config my-server-config.json \
--server my-server \
--method tools/listCreate a tool-tests.yaml file
Write direct tool tests in YAML. Each test specifies a tool name, parameters, and expected outcome. These tests run without any LLM.
tools:
expected_tool_list: ['write_file', 'read_file']
tests:
- name: 'Write file successfully'
tool: 'write_file'
params: { path: '/tmp/test.txt', content: 'Hello world' }
expect: { success: true }
- name: 'Read file returns content'
tool: 'read_file'
params: { path: '/tmp/test.txt' }
expect: { success: true }Run direct tool tests
Execute the tools test suite against your server. No API key is needed. Results show per-test pass/fail with expected vs actual output.
npx mcp-server-tester tools tool-tests.yaml --server-config my-server-config.jsonCreate an eval-tests.yaml file and run LLM eval tests
Write eval tests that describe tasks in natural language and verify the LLM called the right tools. Set ANTHROPIC_API_KEY before running.
export ANTHROPIC_API_KEY="your-key"
npx mcp-server-tester evals eval-tests.yaml --server-config my-server-config.jsonTester MCP Examples
Eval test configuration example
A minimal eval-tests.yaml showing how to specify a natural language prompt, required tool calls, and an LLM-judge scoring criterion.
{
"evals": {
"models": ["claude-3-5-haiku-latest"],
"tests": [
{
"name": "LLM can write files",
"prompt": "Create a file at /tmp/greeting.txt with the content 'Hello'",
"expected_tool_calls": {
"required": ["write_file"]
},
"response_scorers": [
{
"type": "llm-judge",
"criteria": "Did the assistant successfully create the file?",
"threshold": 0.8
}
]
}
]
}
}Prompts to try
Tester is a CLI tool, not an interactive server. These are the commands you run to exercise it.
- npx mcp-server-tester tools tool-tests.yaml --server-config server-config.json
- npx mcp-server-tester evals eval-tests.yaml --server-config server-config.json
- npx mcp-server-tester compliance --server-config server-config.json
- npx -y mcp-server-tester --helpTroubleshooting Tester MCP
Eval tests fail with 'ANTHROPIC_API_KEY not set'
The evals command requires an Anthropic API key to call LLMs as the test subject. Export the key before running: export ANTHROPIC_API_KEY=your-key. Direct tool tests (the tools command) do not require any API key.
Tool tests report 'server not found' or connection refused
Check that the command and args in your server config JSON correctly launch your MCP server. Test by running the command manually in a terminal. The server-config format must use the same mcpServers structure as Claude Desktop config.
Eval tests pass locally but the LLM calls the wrong tools
Tool selection by LLMs is non-deterministic. Add required tool calls to expected_tool_calls and set a realistic threshold (0.7-0.8) in your llm-judge scorer. If a specific tool is never called, improve the tool's description in your MCP server to make its purpose clearer.
Frequently Asked Questions about Tester MCP
What is Tester MCP?
Tester MCP is a Model Context Protocol (MCP) server that cli-based tester for verifying that mcp servers work correctly when called directory and by agents It connects AI assistants to external tools and data sources through a standardized interface.
How do I install Tester MCP?
Follow the installation instructions on the Tester MCP GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with Tester MCP?
Tester MCP works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is Tester MCP free to use?
Yes, Tester MCP is open source and available under the MIT license. You can use it freely in both personal and commercial projects.
Tester MCP Alternatives — Similar Developer Tools Servers
Looking for alternatives to Tester MCP? Here are other popular developer tools servers you can use with Claude, Cursor, and VS Code.
Ecc
★ 188.2kThe agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
Javaguide
★ 155.8kJava 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发
Gemini CLI
★ 104.5kA secure MCP server that wraps the Google Gemini CLI, allowing clients to query Gemini models using local OAuth sessions without requiring an API key. It provides tools for model interaction and diagnostics with built-in protection against command in
Awesome MCP Servers
★ 87.3k⭐ Curated list of Model Context Protocol (MCP) servers - tools that extend Claude Desktop, Cursor, Windsurf, and other MCP clients with custom capabilities.
MCP Servers
★ 86.0kModel Context Protocol Servers
CC Switch
★ 77.5kA cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Gemini CLI & Hermes Agent. Only official website: ccswitch.io
Browse More Developer Tools MCP Servers
Explore all developer tools servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up Tester MCP in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use Tester MCP?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.