AI Cursor Scraping Assistant
A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.
What is AI Cursor Scraping Assistant?
AI Cursor Scraping Assistant is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to powerful tool that leverages cursor ai and mcp (model context protocol) to easily generate web scrapers for various types of websites.
A powerful tool that leverages Cursor AI and MCP (Model Context Protocol) to easily generate web scrapers for various types of websites.
This server falls under the Search & Data Extraction category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- A powerful tool that leverages Cursor AI and MCP (Model Cont
Use Cases
Maintainer
Works with
Installation
NPM
npx -y ai-cursor-scraping-assistantManual Installation
npx -y ai-cursor-scraping-assistantConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use AI Cursor Scraping Assistant
The AI Cursor Scraping Assistant MCP server connects Cursor AI to a Python-based scraping toolkit that analyzes web pages and generates production-ready XPath selectors and Scrapy spiders from natural language prompts. It uses Camoufox, a stealth browser engine, to handle anti-bot protections such as Akamai, Datadome, and PerimeterX, and supports both product listing (PLP) and product detail (PDP) scraper templates. Developers use it to rapidly scaffold web scrapers for e-commerce and data extraction projects without writing XPath by hand.
Prerequisites
- Python 3.9 or higher installed
- Cursor AI IDE with MCP support enabled
- pip packages: mcp, camoufox, scrapy (installed via pip)
- Camoufox browser binary fetched via `python -m camoufox fetch`
- Repository cloned locally from GitHub
Clone the repository
Clone the AI-Cursor-Scraping-Assistant repository to your local machine and navigate into the project directory.
git clone https://github.com/TheWebScrapingClub/AI-Cursor-Scraping-Assistant.git
cd AI-Cursor-Scraping-AssistantInstall Python dependencies
Install the required Python packages: the MCP library, Camoufox stealth browser, and Scrapy for spider generation.
pip install mcp camoufox scrapyFetch the Camoufox browser binary
Download the Camoufox browser binary needed for stealth scraping. This step must complete before running the server.
python -m camoufox fetchConfigure the Camoufox file path
Open MCPfiles/xpath_server.py and update the CAMOUFOX_FILE_PATH variable to point to the local Camoufox template file downloaded in the previous step.
Start the MCP server
Launch the XPath MCP server from the MCPfiles directory. This starts the server in stdio mode, ready for Cursor AI to connect.
cd MCPfiles
python xpath_server.pyRegister the server in Cursor
Open Cursor IDE, go to Settings > MCP, and add the server. Point the command to your Python interpreter and the xpath_server.py script path.
AI Cursor Scraping Assistant Examples
Client configuration
Add this entry to your Cursor MCP settings to register the scraping assistant server.
{
"mcpServers": {
"ai-cursor-scraping-assistant": {
"command": "python",
"args": ["/path/to/AI-Cursor-Scraping-Assistant/MCPfiles/xpath_server.py"]
}
}
}Prompts to try
After connecting the server in Cursor, use these natural language prompts to generate scrapers.
- "Write an e-commerce PLP scraper for gucci.com that extracts product names, prices, and image URLs"
- "Write an e-commerce PDP scraper for nike.com product pages"
- "Analyze the anti-bot protection on target.com and generate XPath selectors for product listings"
- "Generate a Scrapy spider for the search results page on amazon.com"Troubleshooting AI Cursor Scraping Assistant
Camoufox fails to launch or the browser binary is not found
Re-run `python -m camoufox fetch` to re-download the binary, then verify that CAMOUFOX_FILE_PATH in xpath_server.py points to the correct downloaded file path.
Cursor IDE does not detect the MCP server
Ensure you are starting the server before opening Cursor, and that the Python path in the MCP configuration points to the same environment where mcp and camoufox are installed (use the full path to the venv Python if applicable).
Generated scraper fails on sites with heavy bot protection
Confirm Camoufox is being used (not plain requests) by checking xpath_server.py configuration. Sites using advanced fingerprinting may require additional Camoufox configuration options documented in the Camoufox README.
Frequently Asked Questions about AI Cursor Scraping Assistant
What is AI Cursor Scraping Assistant?
AI Cursor Scraping Assistant is a Model Context Protocol (MCP) server that powerful tool that leverages cursor ai and mcp (model context protocol) to easily generate web scrapers for various types of websites. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install AI Cursor Scraping Assistant?
Install via npm with the command: npx -y ai-cursor-scraping-assistant. Then add the server configuration to your AI client's JSON config file (e.g., claude_desktop_config.json or .cursor/mcp.json).
Which AI clients work with AI Cursor Scraping Assistant?
AI Cursor Scraping Assistant works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is AI Cursor Scraping Assistant free to use?
Yes, AI Cursor Scraping Assistant is open source and available under the MIT license. You can use it freely in both personal and commercial projects.
AI Cursor Scraping Assistant Alternatives — Similar Search & Data Extraction Servers
Looking for alternatives to AI Cursor Scraping Assistant? Here are other popular search & data extraction servers you can use with Claude, Cursor, and VS Code.
TrendRadar
★ 58.0kA real-time hotspot monitoring and news aggregation assistant that provides AI-powered analysis of trending topics across multiple platforms via the Model Context Protocol. It enables users to track news and receive automated notifications through va
Scrapling
★ 52.7k🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
PDF Math Translate
★ 33.9k[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero
GPT Researcher
★ 27.2kAn autonomous agent that conducts deep research on any data using any LLM providers
Agent Reach
★ 20.1kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Xiaohongshu
★ 13.7kMCP for xiaohongshu.com
Browse More Search & Data Extraction MCP Servers
Explore all search & data extraction servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up AI Cursor Scraping Assistant in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use AI Cursor Scraping Assistant?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.