CRW
Fast, lightweight Firecrawl alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/v1/scrape, /v1/crawl, /v1/search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benc
What is CRW?
CRW is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to fast, lightweight firecrawl alternative in rust. web scraper, crawler & search api with mcp server for ai agents. drop-in firecrawl-compatible api (/v1/scrape, /v1/crawl, /v1/search). 2.3x faster than...
Fast, lightweight Firecrawl alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/v1/scrape, /v1/crawl, /v1/search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benc
This server falls under the Search & Data Extraction category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- Fast, lightweight Firecrawl alternative in Rust. Web scraper
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx crwConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use CRW
CRW (fastCRW) is a fast, lightweight web scraper, crawler, and search server written in Rust that offers a drop-in Firecrawl-compatible REST API. It exposes endpoints for single-URL scraping, multi-page crawling, site mapping, structured extraction, and web search — all from a single ~50 MB static binary that requires no Redis or headless-browser sidecars. AI agents and MCP clients can use CRW to retrieve clean markdown, HTML, or structured JSON from any website at speeds benchmarked at 2.3x faster than Tavily and 1.5x faster than Firecrawl at 1,000-URL scale.
Prerequisites
- Node.js 18+ installed (for npx crw-mcp) or Docker for container deployment
- An MCP-compatible client such as Claude Desktop, Cursor, or Windsurf
- Optional: CRW_API_KEY and CRW_API_URL if using the managed fastCRW.com service instead of the embedded local mode
- Optional: Cargo (Rust toolchain) if installing via cargo install crw-mcp
Choose your installation method
CRW can run in embedded mode (zero external dependencies, no API key required) using npx, or connect to the managed fastCRW.com service for higher concurrency. For most local setups, npx crw-mcp starts the server with no configuration.
npx crw-mcpAdd CRW to Claude Code via CLI
Use the Claude Code CLI to register CRW as an MCP server. For embedded mode no credentials are needed. For the managed service pass your CRW_API_URL and CRW_API_KEY.
# Embedded mode (no auth)
claude mcp add crw -- npx crw-mcp
# Managed service
claude mcp add -e CRW_API_URL=https://api.fastcrw.com -e CRW_API_KEY=your_key_here crw -- npx crw-mcpConfigure Claude Desktop manually
Edit your claude_desktop_config.json to add the crw-mcp server. Use the env block only if targeting the managed service; omit it for local embedded mode.
{
"mcpServers": {
"crw": {
"command": "npx",
"args": ["crw-mcp"],
"env": {
"CRW_API_URL": "https://api.fastcrw.com",
"CRW_API_KEY": "your_key_here"
}
}
}
}Verify the server starts
Run the MCP server directly and check for startup output. In embedded mode you should see a confirmation that CRW is listening. No external services are required.
npx crw-mcpUse CRW capabilities in your AI client
CRW exposes /v1/scrape, /v1/crawl, /v1/map, /v1/extract, /v1/search, and /v1/change-tracking/diff endpoints. Ask Claude to scrape a URL, crawl a docs site, or search the web and CRW will handle the request.
CRW Examples
Client configuration
Minimal Claude Desktop config for CRW in local embedded mode.
{
"mcpServers": {
"crw": {
"command": "npx",
"args": ["crw-mcp"]
}
}
}Prompts to try
Example prompts that exercise CRW's scraping, crawling, and search capabilities.
- "Scrape https://example.com and return the content as clean markdown"
- "Crawl https://docs.mysite.com up to depth 2 and summarize each page"
- "Search the web for 'Rust async runtime comparison' and fetch the top 3 results"
- "Extract all links from https://news.ycombinator.com front page"
- "Check if https://example.com/pricing has changed since yesterday"Troubleshooting CRW
npx crw-mcp fails with 'command not found' or network error
Ensure Node.js 18+ is installed and npx is on your PATH. Alternatively install via Homebrew: brew install us/crw/crw-mcp, or via Cargo: cargo install crw-mcp.
API requests return 401 Unauthorized when using the managed service
Set the CRW_API_KEY environment variable to the token from your fastCRW.com account dashboard, and ensure CRW_API_URL points to https://api.fastcrw.com.
Crawl returns empty results for JavaScript-rendered pages
CRW's embedded mode does not run a headless browser by default. For JS-heavy sites, consider using the managed service which supports browser rendering, or pass the appropriate rendering option in the crawl request.
Frequently Asked Questions about CRW
What is CRW?
CRW is a Model Context Protocol (MCP) server that fast, lightweight firecrawl alternative in rust. web scraper, crawler & search api with mcp server for ai agents. drop-in firecrawl-compatible api (/v1/scrape, /v1/crawl, /v1/search). 2.3x faster than tavily, 1.5x faster than firecrawl in 1k-url benc It connects AI assistants to external tools and data sources through a standardized interface.
How do I install CRW?
Follow the installation instructions on the CRW GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with CRW?
CRW works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is CRW free to use?
Yes, CRW is open source and available under the AGPL-3.0 license. You can use it freely in both personal and commercial projects.
CRW Alternatives — Similar Search & Data Extraction Servers
Looking for alternatives to CRW? Here are other popular search & data extraction servers you can use with Claude, Cursor, and VS Code.
TrendRadar
★ 58.0kA real-time hotspot monitoring and news aggregation assistant that provides AI-powered analysis of trending topics across multiple platforms via the Model Context Protocol. It enables users to track news and receive automated notifications through va
Scrapling
★ 52.7k🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
PDF Math Translate
★ 33.9k[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero
GPT Researcher
★ 27.2kAn autonomous agent that conducts deep research on any data using any LLM providers
Agent Reach
★ 20.1kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Xiaohongshu
★ 13.7kMCP for xiaohongshu.com
Browse More Search & Data Extraction MCP Servers
Explore all search & data extraction servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up CRW in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use CRW?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.