Spectrawl
Unified web layer for AI agents. Search across 8 engines, stealth browse with Camoufox/Playwright, manage cookie auth, and act on 24 platforms.
What is Spectrawl?
Spectrawl is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to unified web layer for ai agents. search across 8 engines, stealth browse with camoufox/playwright, manage cookie auth, and act on 24 platforms.
Unified web layer for AI agents. Search across 8 engines, stealth browse with Camoufox/Playwright, manage cookie auth, and act on 24 platforms.
This server falls under the Search & Data Extraction category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- Unified web layer for AI agents. Search across 8 engines, st
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx spectrawlConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use Spectrawl
Spectrawl is a unified web layer MCP server that gives AI agents production-grade search and browsing capabilities through a single interface. It aggregates results across up to 8 search engines (including Gemini Grounded Search, Brave, and Tavily), performs stealth browsing with Camoufox anti-fingerprinting to avoid bot detection, and provides 24 platform action adapters for sites like GitHub, Reddit, LinkedIn, and Discord. Agents can search, scrape, extract structured data, solve CAPTCHAs, and automate web interactions all through Spectrawl's MCP tools.
Prerequisites
- Node.js 18+ installed
- GEMINI_API_KEY for primary search (Gemini Grounded Search) — free tier provides 5,000 requests/month
- At least one fallback search key: BRAVE_API_KEY or TAVILY_API_KEY
- An MCP-compatible client such as Claude Desktop, Cursor, or Claude Code
- Optional: Camoufox browser (install with npx spectrawl install-stealth) for stealth browsing
Install Spectrawl
Install Spectrawl globally or run it with npx. Then initialize the configuration file.
npm install -g spectrawl
npx spectrawl initInstall the stealth browser
Download Camoufox, the anti-fingerprinting browser used for stealth page visits that avoid bot detection.
npx spectrawl install-stealthSet required API keys
Export your search API keys as environment variables. GEMINI_API_KEY is used for primary Gemini Grounded Search; BRAVE_API_KEY and TAVILY_API_KEY serve as fallbacks.
export GEMINI_API_KEY=your_gemini_api_key
export BRAVE_API_KEY=your_brave_api_key
export TAVILY_API_KEY=your_tavily_api_key
# Optional platform keys:
export GITHUB_TOKEN=your_github_token
export DEVTO_API_KEY=your_devto_keyConfigure spectrawl.json
Edit the generated spectrawl.json to customize search cascade order, proxy settings, cookie auth refresh interval, and caching.
{
"search": { "cascade": ["gemini-grounded", "tavily", "brave"] },
"browse": { "proxy": { "host": "", "port": 10001 } },
"auth": { "refreshInterval": "4h" },
"cache": { "searchTtl": 3600 }
}Add to Claude Desktop config
Configure Spectrawl as an MCP server in your Claude Desktop or other MCP client configuration.
{
"mcpServers": {
"spectrawl": {
"command": "npx",
"args": ["spectrawl", "mcp"],
"env": {
"GEMINI_API_KEY": "your_gemini_api_key",
"BRAVE_API_KEY": "your_brave_api_key",
"TAVILY_API_KEY": "your_tavily_api_key"
}
}
}
}Start the MCP server
Launch Spectrawl in MCP server mode. Alternatively start the HTTP server if you need REST API access.
npx spectrawl mcp
# Or HTTP server mode:
npx spectrawl serve --port 3900Spectrawl Examples
Client configuration
Claude Desktop configuration for Spectrawl with Gemini as primary search engine and Brave as fallback.
{
"mcpServers": {
"spectrawl": {
"command": "npx",
"args": ["spectrawl", "mcp"],
"env": {
"GEMINI_API_KEY": "your_gemini_api_key",
"BRAVE_API_KEY": "your_brave_api_key",
"TAVILY_API_KEY": "your_tavily_api_key"
}
}
}
}Prompts to try
Example prompts that leverage Spectrawl's search, scraping, and platform automation capabilities.
- "Search for the latest news about MCP servers and scrape the top 3 articles for full content"
- "Browse the GitHub trending page and extract all repository names and star counts"
- "Search Reddit for discussions about Rust async performance and summarize the top 5 threads"
- "Extract all pricing information from this SaaS website: https://example.com/pricing"
- "Post a status update to my Dev.to account: [content]"Troubleshooting Spectrawl
Search returns no results or fails immediately
Verify that at least one search API key is set correctly. Run echo $GEMINI_API_KEY to confirm it is exported. The cascade falls back to TAVILY then BRAVE if the primary fails, so having multiple keys configured improves reliability.
Stealth browser fails to launch or pages are blocked
Run npx spectrawl install-stealth to ensure Camoufox is downloaded. If sites still block access, configure a residential proxy in spectrawl.json under browse.proxy with a host and port.
Platform actions fail with authentication errors
Platform adapters (GitHub, Reddit, etc.) require their own credentials. Set the appropriate env var (e.g., GITHUB_TOKEN for GitHub actions) and ensure the auth.refreshInterval in spectrawl.json is appropriate for your session duration.
Frequently Asked Questions about Spectrawl
What is Spectrawl?
Spectrawl is a Model Context Protocol (MCP) server that unified web layer for ai agents. search across 8 engines, stealth browse with camoufox/playwright, manage cookie auth, and act on 24 platforms. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install Spectrawl?
Follow the installation instructions on the Spectrawl GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with Spectrawl?
Spectrawl works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is Spectrawl free to use?
Yes, Spectrawl is open source and available under the MIT License license. You can use it freely in both personal and commercial projects.
Spectrawl Alternatives — Similar Search & Data Extraction Servers
Looking for alternatives to Spectrawl? Here are other popular search & data extraction servers you can use with Claude, Cursor, and VS Code.
TrendRadar
★ 58.0kA real-time hotspot monitoring and news aggregation assistant that provides AI-powered analysis of trending topics across multiple platforms via the Model Context Protocol. It enables users to track news and receive automated notifications through va
Scrapling
★ 52.7k🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
PDF Math Translate
★ 33.9k[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero
GPT Researcher
★ 27.2kAn autonomous agent that conducts deep research on any data using any LLM providers
Agent Reach
★ 20.1kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Xiaohongshu
★ 13.7kMCP for xiaohongshu.com
Browse More Search & Data Extraction MCP Servers
Explore all search & data extraction servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up Spectrawl in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use Spectrawl?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.