Apache Spark History
MCP Server and CLI for Apache Spark History Server. Debug Spark applications from AI agents, scripts, or the terminal.
What is Apache Spark History?
Apache Spark History is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to mcp server and cli for apache spark history server. debug spark applications from ai agents, scripts, or the terminal.
MCP Server and CLI for Apache Spark History Server. Debug Spark applications from AI agents, scripts, or the terminal.
This server falls under the Databases and Developer Tools categories on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- MCP Server and CLI for Apache Spark History Server. Debug Sp
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx mcp-apache-spark-historyConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use Apache Spark History
The Apache Spark History MCP Server connects AI agents and LLM clients to the Apache Spark History Server API, exposing 21 tools for diagnosing and analyzing Spark applications. It lets you list applications, inspect job and stage timelines, compare SQL execution plans, identify bottlenecks, and get executor resource usage — all from natural language queries. Engineers running Spark on Kubernetes or standalone clusters can use this server to answer performance questions like 'Why is my ETL slower than yesterday?' without leaving their AI assistant. A companion CLI (`shs`) provides the same capabilities for scripting and terminal workflows.
Prerequisites
- Python 3.9+ and `uv` or `uvx` installed (pip install uv)
- A running Apache Spark History Server with a reachable HTTP URL
- Network access from the MCP host to the Spark History Server endpoint
- An MCP-compatible client such as Claude Desktop, Claude Code, or Cursor
- Optional: Spark cluster credentials (username/password) if the History Server requires authentication
Install the MCP server via uvx
Use `uvx` to run the server directly from PyPI without a manual install step. This is the recommended approach for MCP clients that launch the server as a subprocess.
uvx --from mcp-apache-spark-history-server spark-mcpCreate a config.yaml file
The server reads its Spark History Server endpoints from a YAML config file. Create `~/.config/spark-mcp/config.yaml` (or any path you prefer) and define one or more named server entries.
servers:
prod:
url: http://spark-history.example.com:18080
auth:
username: admin
password: secret
include_plan_description: false
mcp:
transport: stdio
port: 18888Set environment variables (alternative to config file)
All config keys can be overridden with environment variables, which is useful for containerised deployments. Prefix server-specific keys with `SHS_SERVERS_<NAME>_`.
export SHS_SERVERS_PROD_URL=http://spark-history.example.com:18080
export SHS_SERVERS_PROD_AUTH_USERNAME=admin
export SHS_SERVERS_PROD_AUTH_PASSWORD=secret
export SHS_MCP_TRANSPORT=stdioAdd the server to your MCP client configuration
Register the server in your MCP client's config file so it is launched automatically. The example below uses Claude Desktop with `claude_desktop_config.json`.
Verify the connection
Run the CLI tool to confirm the server can reach your Spark History Server and list applications.
uvx --from mcp-apache-spark-history-server shs appsApache Spark History Examples
Client configuration
Claude Desktop config block that launches the Spark History MCP server via uvx with a config file path passed as an argument.
{
"mcpServers": {
"spark-history": {
"command": "uvx",
"args": ["--from", "mcp-apache-spark-history-server", "spark-mcp"],
"env": {
"SHS_SERVERS_PROD_URL": "http://spark-history.example.com:18080",
"SHS_MCP_TRANSPORT": "stdio"
}
}
}
}Prompts to try
Natural-language queries you can ask once the server is connected.
- "List my last 10 Spark applications and show their status"
- "Find the slowest SQL queries in application app-20240601-001 and explain why they are slow"
- "Compare the execution plan for SQL query 42 between yesterday's run and today's"
- "Show executor resource usage and identify any stragglers in the current ETL job"
- "Why is my Spark job running slower than it did last week?"Troubleshooting Apache Spark History
Server starts but returns 'connection refused' when calling tools
Verify the Spark History Server URL is correct and reachable from the machine running the MCP server. Check that the port (default 18080) is open and test with `curl http://spark-history.example.com:18080/api/v1/applications`.
Authentication errors when listing applications
Ensure `SHS_SERVERS_<NAME>_AUTH_USERNAME` and `SHS_SERVERS_<NAME>_AUTH_PASSWORD` are set correctly, or that the `auth.username` / `auth.password` fields in config.yaml match the Spark History Server credentials.
`uvx` command not found
Install `uv` first with `curl -LsSf https://astral.sh/uv/install.sh | sh`, then reload your shell. Alternatively install the package directly with `uv tool install mcp-apache-spark-history-server` and run `spark-mcp`.
Frequently Asked Questions about Apache Spark History
What is Apache Spark History?
Apache Spark History is a Model Context Protocol (MCP) server that mcp server and cli for apache spark history server. debug spark applications from ai agents, scripts, or the terminal. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install Apache Spark History?
Follow the installation instructions on the Apache Spark History GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with Apache Spark History?
Apache Spark History works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is Apache Spark History free to use?
Yes, Apache Spark History is open source and available under the Apache-2.0 license. You can use it freely in both personal and commercial projects.
Apache Spark History Alternatives — Similar Databases Servers
Looking for alternatives to Apache Spark History? Here are other popular databases servers you can use with Claude, Cursor, and VS Code.
Excelize
★ 20.6kGo language library for reading and writing Microsoft Excel™ (XLAM / XLSM / XLSX / XLTM / XLTX) spreadsheets
MCP Toolbox for Databases
★ 15.3kOpen source MCP server specializing in easy, fast, and secure tools for Databases.
DBHub
★ 2.8kA universal database gateway MCP server that enables AI assistants to connect to and query multiple databases (PostgreSQL, MySQL, MariaDB, SQL Server, SQLite) with support for schema exploration, SQL execution, and secure connections via SSH tunnels.
Tabularis
★ 2.1kA lightweight, cross-platform database client for developers. Supports MySQL, PostgreSQL and SQLite. Hackable with plugins. Built for speed, security, and aesthetics.
Postgres AI Guide
★ 1.7kMCP server and Claude plugin for Postgres skills and documentation. Helps AI coding tools generate better PostgreSQL code.
Anyquery
★ 1.7k🏎️ 🏠 ☁️ - Query more than 40 apps with one binary using SQL. It can also connect to your PostgreSQL, MySQL, or SQLite compatible database. Local-first and private by design.
Browse More Databases MCP Servers
Explore all databases servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up Apache Spark History in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use Apache Spark History?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.