Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
What is Scrapling?
Scrapling is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to 🕷️ an adaptive web scraping framework that handles everything from a single request to a full-scale crawl!
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
This server falls under the Search & Data Extraction category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- 🕷️ An adaptive Web Scraping framework that handles everythin
Use Cases
Maintainer
Works with
Installation
PIP
pip install scraplingManual Installation
pip install scraplingConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use Scrapling
Scrapling is a high-performance, adaptive Python web scraping framework that handles everything from single HTTP requests to full-scale concurrent crawls, with built-in anti-bot bypass capabilities including Cloudflare Turnstile solving. Its MCP server integration allows AI assistants like Claude to scrape targeted web content with intelligent pre-extraction, reducing token usage by pulling only the relevant data before it reaches the LLM. Developers use Scrapling when they need fast, resilient data extraction that adapts automatically when a website's layout changes, without manually updating CSS selectors.
Prerequisites
- Python 3.10+ installed on your system
- pip package manager
- For browser-based scraping (StealthyFetcher, DynamicFetcher): Playwright browsers installed via 'scrapling install'
- An MCP client such as Claude Desktop or Cursor
Install Scrapling with MCP support
Install the base package plus the AI extras that include the MCP server. For full browser automation and anti-bot bypass, also install the fetchers extra.
pip install "scrapling[ai,fetchers]"Download browser binaries
If you installed the fetchers extra for JavaScript rendering or Cloudflare bypass, run the Scrapling installer to download the required browser binaries.
scrapling installVerify the installation
Confirm Scrapling is correctly installed and the MCP server module is available by running a quick version check.
python -c "import scrapling; print(scrapling.__version__)"
scrapling shell # Optional: open the interactive scraping shellConfigure the MCP server in your AI client
Add Scrapling's MCP server to your Claude Desktop configuration. The server runs as a Python module directly, requiring no separate server process.
Test a basic scrape via CLI
Before connecting to an AI client, verify scraping works via the CLI extract command to ensure dependencies are correct.
scrapling extract get 'https://quotes.toscrape.com' output.md
scrapling extract fetch 'https://quotes.toscrape.com' output.md --css-selector '.quote .text'Scrapling Examples
Client configuration
Add Scrapling's MCP server to claude_desktop_config.json. It runs as a Python module, so ensure python points to the environment where scrapling is installed.
{
"mcpServers": {
"scrapling": {
"command": "python",
"args": ["-m", "scrapling.mcp"],
"env": {}
}
}
}Prompts to try
Once Scrapling is connected as an MCP server, use these prompts to extract web content through your AI client.
- "Scrape the top 10 quotes from https://quotes.toscrape.com and list them with their authors"
- "Fetch the product names and prices from https://books.toscrape.com"
- "Extract all the article headlines and links from the Hacker News front page"
- "Scrape this page and give me only the main article text, ignoring navigation and ads: https://example.com/article"Troubleshooting Scrapling
ImportError or ModuleNotFoundError when starting the MCP server
Ensure scrapling is installed in the same Python environment that Claude Desktop invokes. Use the full path to python if needed (e.g. /usr/local/bin/python3 or the path to your venv). Run 'which python && python -c "import scrapling"' to verify.
Fetching JavaScript-heavy pages returns empty or incomplete content
Use DynamicFetcher or StealthyFetcher which require the fetchers extra and browser binaries. Run 'pip install scrapling[fetchers]' and 'scrapling install' to download Playwright browsers, then retry.
Cloudflare or bot protection blocks scraping attempts
Switch to StealthyFetcher with solve_cloudflare=True. This uses a real browser with stealth patches. If it still fails, try 'scrapling install --force' to refresh browser binaries to the latest version.
Frequently Asked Questions about Scrapling
What is Scrapling?
Scrapling is a Model Context Protocol (MCP) server that 🕷️ an adaptive web scraping framework that handles everything from a single request to a full-scale crawl! It connects AI assistants to external tools and data sources through a standardized interface.
How do I install Scrapling?
Install via pip with: pip install scrapling. Then configure your AI client to connect to this MCP server.
Which AI clients work with Scrapling?
Scrapling works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is Scrapling free to use?
Yes, Scrapling is open source and available under the BSD-3-Clause license. You can use it freely in both personal and commercial projects.
Scrapling Alternatives — Similar Search & Data Extraction Servers
Looking for alternatives to Scrapling? Here are other popular search & data extraction servers you can use with Claude, Cursor, and VS Code.
TrendRadar
★ 58.0kA real-time hotspot monitoring and news aggregation assistant that provides AI-powered analysis of trending topics across multiple platforms via the Model Context Protocol. It enables users to track news and receive automated notifications through va
PDF Math Translate
★ 33.9k[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero
GPT Researcher
★ 27.2kAn autonomous agent that conducts deep research on any data using any LLM providers
Agent Reach
★ 20.1kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Xiaohongshu
★ 13.7kMCP for xiaohongshu.com
XHS Downloader
★ 11.2k小红书（XiaoHongShu、RedNote）链接提取/作品采集工具：提取账号发布、收藏、点赞、专辑作品链接；提取搜索结果作品、用户链接；采集小红书作品信息；提取小红书作品下载地址；下载小红书作品文件
Browse More Search & Data Extraction MCP Servers
Explore all search & data extraction servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up Scrapling in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use Scrapling?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.