PyAutoGUI
Provides automated GUI testing and control capabilities through an MCP server that enables mouse movements, keyboard input, screen captures, and image recognition across Windows, macOS, and Linux.
What is PyAutoGUI?
PyAutoGUI is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to provides automated gui testing and control capabilities through an mcp server that enables mouse movements, keyboard input, screen captures, and image recognition across windows, macos, and linux.
Provides automated GUI testing and control capabilities through an MCP server that enables mouse movements, keyboard input, screen captures, and image recognition across Windows, macOS, and Linux.
This server falls under the Browser Automation category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- Provides automated GUI testing and control capabilities thro
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx pyautogui-mcp-serverConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use PyAutoGUI
The PyAutoGUI MCP Server wraps the PyAutoGUI automation library as an MCP server, giving AI assistants programmatic control over the desktop GUI through mouse movements, clicks, drag-and-drop, keyboard input, screenshot capture, and on-screen image recognition. It runs on Windows, macOS, and Linux, making it suitable for cross-platform GUI test automation, repetitive desktop workflows, and scenarios where a web API does not exist. QA engineers and automation developers use it to let Claude drive applications as if operating a mouse and keyboard.
Prerequisites
- Python 3.12+ installed on your system
- uv package manager installed (recommended) or pip
- PyAutoGUI system dependencies: on Linux, install python3-tk python3-dev scrot via apt
- Display server access: on Linux headless servers, Xvfb is required
- An MCP-compatible client such as Claude Desktop
Install the server via pip
Install the mcp-pyautogui-server package from PyPI. Using uvx is also supported for isolated execution without a virtual environment.
pip install mcp-pyautogui-serverInstall Linux system dependencies if needed
On Linux, PyAutoGUI requires Tkinter, dev headers, and a screenshot utility. Install these via your package manager before running the server.
# Ubuntu / Debian
sudo apt-get install python3-tk python3-dev scrot
# For headless environments
sudo apt-get install xvfb
export DISPLAY=:99
Xvfb :99 -screen 0 1920x1080x24 &Add the server to your MCP client configuration
Open claude_desktop_config.json and add an entry for mcp-pyautogui-server. Use uvx for zero-dependency-management invocation, or python with the module flag if you installed via pip.
{
"mcpServers": {
"pyautogui": {
"command": "uvx",
"args": ["mcp-pyautogui-server"]
}
}
}Restart Claude Desktop
Quit and relaunch Claude Desktop. The PyAutoGUI tools for mouse, keyboard, and screen control will now be available in conversations.
Test interactively with MCP Inspector
Before using in Claude, validate tools using the MCP Inspector. This lets you call mouse_move, screenshot, or keyboard_type individually and see their responses.
npx @modelcontextprotocol/inspector uv --directory /path/to/mcp-pyautogui-server run mcp-pyautogui-serverPyAutoGUI Examples
Client configuration
claude_desktop_config.json entry using uvx to run the published mcp-pyautogui-server package without a manual virtual environment.
{
"mcpServers": {
"pyautogui": {
"command": "uvx",
"args": ["mcp-pyautogui-server"]
}
}
}Prompts to try
Example prompts using mouse, keyboard, and screenshot tools to automate desktop tasks.
- "Take a screenshot of the current screen and describe what you see."
- "Move the mouse to coordinates (500, 300) and click."
- "Type the text 'Hello World' at the current cursor position."
- "Press the keyboard shortcut Ctrl+C to copy the selected text."
- "Find the location of the 'Submit' button image on screen and click it."Troubleshooting PyAutoGUI
PyAutoGUI raises 'Xlib.error.DisplayNameError' or 'cannot connect to display' on Linux
The server requires a running display server. Set the DISPLAY environment variable (export DISPLAY=:0 for a local X session) or start Xvfb for headless environments: Xvfb :99 -screen 0 1920x1080x24 & then export DISPLAY=:99.
macOS blocks mouse/keyboard control with 'not trusted for accessibility'
Go to System Settings > Privacy & Security > Accessibility and add your terminal application (or the Python binary) to the allowed list. Without this permission, PyAutoGUI cannot send input events on macOS.
Screenshot tool returns an error or black image
On macOS 10.15+, screen recording permission is also required. Add your terminal to System Settings > Privacy & Security > Screen Recording. On Linux, ensure scrot or a compatible screenshot tool is installed.
Frequently Asked Questions about PyAutoGUI
What is PyAutoGUI?
PyAutoGUI is a Model Context Protocol (MCP) server that provides automated gui testing and control capabilities through an mcp server that enables mouse movements, keyboard input, screen captures, and image recognition across windows, macos, and linux. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install PyAutoGUI?
Follow the installation instructions on the PyAutoGUI GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with PyAutoGUI?
PyAutoGUI works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is PyAutoGUI free to use?
Yes, PyAutoGUI is open source and available under the MIT License license. You can use it freely in both personal and commercial projects.
PyAutoGUI Alternatives — Similar Browser Automation Servers
Looking for alternatives to PyAutoGUI? Here are other popular browser automation servers you can use with Claude, Cursor, and VS Code.
Chrome DevTools MCP
★ 40.6kAI-powered Chrome automation server with natural language element detection. Control Chrome browser through MCP protocol for testing, debugging, and performance analysis. Features 91% accuracy in element location, works with free AI models, and suppo
UI TARS Desktop
★ 34.9k📇 🏠 - Browser automation capabilities using Puppeteer, both support local and remote browser connection.
Playwright
★ 32.8kA production-ready browser automation server that enables AI assistants to interact with web pages using tools for navigation, element interaction, and data extraction. It features a built-in Inspector UI and robust crash recovery for reliable automa
Page Agent
★ 18.0kJavaScript in-page GUI agent. Control web interfaces with natural language.
Chrome
★ 11.7kAn extension-based MCP server that enables AI assistants to control your browser, leveraging existing sessions and login states for automation and content analysis. It provides over 20 tools for semantic tab search, interactive element manipulation,
LAMDA
★ 7.8kThe most powerful Android RPA agent framework, next generation mobile automation.
Browse More Browser Automation MCP Servers
Explore all browser automation servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up PyAutoGUI in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use PyAutoGUI?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.