ScrapeGraph
A production-ready Model Context Protocol server that enables language models to leverage AI-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structured data, and executing AI-powered web searches.
What is ScrapeGraph?
ScrapeGraph is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to production-ready model context protocol server that enables language models to leverage ai-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structure...
A production-ready Model Context Protocol server that enables language models to leverage AI-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structured data, and executing AI-powered web searches.
This server falls under the Search & Data Extraction category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- A production-ready Model Context Protocol server that enable
Use Cases
Maintainer
Works with
Installation
NPM
npx -y scrapegraph-mcpPIP
pip install scrapegraph-mcpManual Installation
npx -y scrapegraph-mcpConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use ScrapeGraph
ScrapeGraph MCP Server is a production-ready Model Context Protocol server that gives AI assistants access to AI-powered web scraping and data extraction capabilities via the ScrapeGraphAI cloud API. It can convert any webpage to clean markdown, extract structured JSON data matching a custom schema, run web searches, and orchestrate multi-page crawls. Developers and researchers use it to pull real-time web content into AI workflows, scrape product data, monitor page changes, and build data pipelines — all without managing headless browsers or parsing HTML manually.
Prerequisites
- Python 3.13+ or Node.js 18+ (depending on chosen install method)
- A ScrapeGraphAI API key from dashboard.scrapegraphai.com
- An MCP-compatible client such as Claude Desktop or Cursor
Get a ScrapeGraphAI API key
Register at dashboard.scrapegraphai.com and create an API key. Free tier credits are available for testing. Copy the key — you will set it as SGAI_API_KEY.
Install the package
Install scrapegraph-mcp via pip for Python environments, or use npx for a zero-install approach.
pip install scrapegraph-mcp
# or run directly without installing:
npx -y scrapegraph-mcpConfigure Claude Desktop
Add the server to your Claude Desktop config with your API key as an environment variable. On macOS the config file is at ~/Library/Application Support/Claude/claude_desktop_config.json.
{
"mcpServers": {
"scrapegraph": {
"command": "npx",
"args": ["-y", "scrapegraph-mcp"],
"env": {
"SGAI_API_KEY": "your_sgai_api_key"
}
}
}
}Restart Claude Desktop
Fully quit and relaunch Claude Desktop. The ScrapeGraph tools will appear when you click the hammer icon in the chat input.
Test with a webpage conversion
Ask Claude to convert a webpage to markdown. This verifies your API key is valid and the server is running correctly.
ScrapeGraph Examples
Client configuration
Claude Desktop configuration using npx with SGAI_API_KEY environment variable.
{
"mcpServers": {
"scrapegraph": {
"command": "npx",
"args": ["-y", "scrapegraph-mcp"],
"env": {
"SGAI_API_KEY": "your_sgai_api_key"
}
}
}
}Prompts to try
Once connected, try these prompts to use the web scraping and extraction capabilities.
- "Convert https://news.ycombinator.com to markdown and summarize the top 5 stories"
- "Extract the product name, price, and rating from https://example-shop.com/product/123 as JSON"
- "Search the web for recent news about Model Context Protocol and summarize the results"
- "Crawl https://docs.example.com and collect all the content from the documentation pages"
- "How many API credits do I have remaining on my ScrapeGraph account?"Troubleshooting ScrapeGraph
API returns 401 Unauthorized or 'invalid API key'
Verify SGAI_API_KEY is set correctly in the env block of your MCP config. The key should start with 'sgai-'. Log in to dashboard.scrapegraphai.com to confirm the key is active and has remaining credits.
Scraping fails on JavaScript-heavy single-page applications
ScrapeGraphAI uses headless browser rendering for SPAs. If results are incomplete, try using the 'extract' tool with a specific prompt describing the data you want instead of the generic 'scrape' tool.
Tool names from older documentation don't work (e.g., 'smartscraper' or 'markdownify')
The v3 API renamed tools. Use 'extract' instead of 'smartscraper', 'crawl_start' instead of 'smartcrawler_initiate', and 'scrape' with output_format='markdown' instead of 'markdownify'.
Frequently Asked Questions about ScrapeGraph
What is ScrapeGraph?
ScrapeGraph is a Model Context Protocol (MCP) server that production-ready model context protocol server that enables language models to leverage ai-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structured data, and executing ai-powered web searches. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install ScrapeGraph?
Install via npm with the command: npx -y scrapegraph-mcp. Then add the server configuration to your AI client's JSON config file (e.g., claude_desktop_config.json or .cursor/mcp.json).
Which AI clients work with ScrapeGraph?
ScrapeGraph works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is ScrapeGraph free to use?
Yes, ScrapeGraph is open source and available under the MIT License license. You can use it freely in both personal and commercial projects.
ScrapeGraph Alternatives — Similar Search & Data Extraction Servers
Looking for alternatives to ScrapeGraph? Here are other popular search & data extraction servers you can use with Claude, Cursor, and VS Code.
TrendRadar
★ 58.0kA real-time hotspot monitoring and news aggregation assistant that provides AI-powered analysis of trending topics across multiple platforms via the Model Context Protocol. It enables users to track news and receive automated notifications through va
Scrapling
★ 52.7k🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
PDF Math Translate
★ 33.9k[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero
GPT Researcher
★ 27.2kAn autonomous agent that conducts deep research on any data using any LLM providers
Agent Reach
★ 20.1kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Xiaohongshu
★ 13.7kMCP for xiaohongshu.com
Browse More Search & Data Extraction MCP Servers
Explore all search & data extraction servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up ScrapeGraph in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use ScrapeGraph?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.