What is Web Scraper?
Web Scraper is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to generic url crawl + html extraction — fallback for sites without dedicated mcps.
Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.
This server falls under the Browser Automation category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- Generic URL crawl + HTML extraction — fallback for sites wit
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx com-thenextgennexus-web-scrapingConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
Frequently Asked Questions about Web Scraper
What is Web Scraper?
Web Scraper is a Model Context Protocol (MCP) server that generic url crawl + html extraction — fallback for sites without dedicated mcps. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install Web Scraper?
Follow the installation instructions on the Web Scraper GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with Web Scraper?
Web Scraper works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is Web Scraper free to use?
Yes, Web Scraper is open source and available under the MIT license. You can use it freely in both personal and commercial projects.
Web Scraper Alternatives — Similar Browser Automation Servers
Looking for alternatives to Web Scraper? Here are other popular browser automation servers you can use with Claude, Cursor, and VS Code.
Chrome DevTools MCP
★ 40.6kAI-powered Chrome automation server with natural language element detection. Control Chrome browser through MCP protocol for testing, debugging, and performance analysis. Features 91% accuracy in element location, works with free AI models, and suppo
UI TARS Desktop
★ 34.9k📇 🏠 - Browser automation capabilities using Puppeteer, both support local and remote browser connection.
Playwright
★ 32.8kA production-ready browser automation server that enables AI assistants to interact with web pages using tools for navigation, element interaction, and data extraction. It features a built-in Inspector UI and robust crash recovery for reliable automa
Page Agent
★ 18.0kJavaScript in-page GUI agent. Control web interfaces with natural language.
Chrome
★ 11.7kAn extension-based MCP server that enables AI assistants to control your browser, leveraging existing sessions and login states for automation and content analysis. It provides over 20 tools for semantic tab search, interactive element manipulation,
LAMDA
★ 7.8kThe most powerful Android RPA agent framework, next generation mobile automation.
Browse More Browser Automation MCP Servers
Explore all browser automation servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up Web Scraper in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use Web Scraper?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.