MediaCrawler MCP Server
Enables AI assistants to crawl and extract data from Chinese social media platforms like Bilibili, Xiaohongshu, and Douyin. Provides search, content detail retrieval, and creator information tools with persistent browser sessions and QR code login su
What is MediaCrawler MCP Server?
MediaCrawler MCP Server is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to enables ai assistants to crawl and extract data from chinese social media platforms like bilibili, xiaohongshu, and douyin. provides search, content detail retrieval, and creator information tools wit...
Enables AI assistants to crawl and extract data from Chinese social media platforms like Bilibili, Xiaohongshu, and Douyin. Provides search, content detail retrieval, and creator information tools with persistent browser sessions and QR code login su
This server falls under the Search & Data Extraction category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- Enables AI assistants to crawl and extract data from Chinese
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx mediacrawler-mcp-serverConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use MediaCrawler MCP Server
MediaCrawler MCP Server is a Python-based service that enables AI assistants to search and extract data from major Chinese social media platforms — including Bilibili, Xiaohongshu (Little Red Book), and Douyin — through a persistent browser session managed by Playwright. It provides platform-specific tools for keyword search, content detail retrieval, creator profile lookup, and comment thread extraction, with QR code login support for platforms that require authentication. Researchers, marketers, and developers who need to gather structured data from Chinese social platforms as part of AI workflows can use this server as an automated extraction backend.
Prerequisites
- Python 3.10 or later with Poetry package manager
- Redis server running locally (used for session and job management)
- Playwright with Chromium browser installed
- Platform accounts for Bilibili, Xiaohongshu, or Douyin (QR code login is supported)
- An MCP-compatible client or SSE-capable integration
Clone the repository and install Python dependencies
Clone the MediaCrawler MCP service repository and install all dependencies using Poetry.
git clone https://github.com/mcp-service/media-crawler-mcp-service.git
cd media-crawler-mcp-service
poetry installInstall the Playwright Chromium browser
MediaCrawler uses Playwright to drive a real browser for platform interactions. Install the Chromium browser binary.
poetry run playwright install chromiumConfigure the environment
Copy the example environment file and adjust settings such as the service port (default 9090), Redis connection parameters, and which platforms to enable.
cp .env.example .envStart Redis and launch the service
Start your Redis server, then launch the MediaCrawler MCP service. The service will be available on port 9090 by default.
redis-server &
poetry run python main.pyLog in to platforms via QR code
Visit the admin UI at http://localhost:9090/admin to initiate QR code login sessions for the platforms you want to crawl. Sessions are persisted across restarts.
Configure your MCP client to connect via SSE
Point your MCP client at the SSE endpoint. The server uses Server-Sent Events transport rather than stdio.
MediaCrawler MCP Server Examples
Client configuration
Configure your MCP client to connect to the MediaCrawler service via its SSE endpoint.
{
"mcpServers": {
"mediacrawler": {
"url": "http://localhost:9090/mcp",
"transport": "sse"
}
}
}Prompts to try
Once connected, use these prompts to search and extract data from Chinese social media platforms.
- "Search Bilibili for videos about Python machine learning and show me the top 5 results"
- "Get the details and stats for Bilibili video BV1234567890"
- "Search Xiaohongshu for posts about skincare routines with the keyword 保湿"
- "Get the creator profile and recent posts for this Xiaohongshu user"
- "Show me the comment thread on this Bilibili video"Troubleshooting MediaCrawler MCP Server
Service fails to start with Redis connection error
Ensure Redis is running before starting main.py. Run redis-server in a separate terminal or as a background service. Check the .env file for the correct Redis host and port settings.
Xiaohongshu detail requests fail or return empty results
The xhs_crawler_detail tool requires a valid xsec_token and xsec_source from a prior search result. Always use the token returned by xhs_search when fetching post details.
Platform login session expires or QR code scan fails
Visit http://localhost:9090/admin to reinitiate a fresh QR code login session for the affected platform. Persistent sessions are stored but can expire if the platform revokes them.
Frequently Asked Questions about MediaCrawler MCP Server
What is MediaCrawler MCP Server?
MediaCrawler MCP Server is a Model Context Protocol (MCP) server that enables ai assistants to crawl and extract data from chinese social media platforms like bilibili, xiaohongshu, and douyin. provides search, content detail retrieval, and creator information tools with persistent browser sessions and qr code login su It connects AI assistants to external tools and data sources through a standardized interface.
How do I install MediaCrawler MCP Server?
Follow the installation instructions on the MediaCrawler MCP Server GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with MediaCrawler MCP Server?
MediaCrawler MCP Server works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is MediaCrawler MCP Server free to use?
Yes, MediaCrawler MCP Server is open source and available under the MIT license. You can use it freely in both personal and commercial projects.
MediaCrawler MCP Server Alternatives — Similar Search & Data Extraction Servers
Looking for alternatives to MediaCrawler MCP Server? Here are other popular search & data extraction servers you can use with Claude, Cursor, and VS Code.
TrendRadar
★ 58.0kA real-time hotspot monitoring and news aggregation assistant that provides AI-powered analysis of trending topics across multiple platforms via the Model Context Protocol. It enables users to track news and receive automated notifications through va
Scrapling
★ 52.7k🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
PDF Math Translate
★ 33.9k[EMNLP 2025 Demo] PDF scientific paper translation with preserved formats - 基于 AI 完整保留排版的 PDF 文档全文双语翻译，支持 Google/DeepL/Ollama/OpenAI 等服务，提供 CLI/GUI/MCP/Docker/Zotero
GPT Researcher
★ 27.2kAn autonomous agent that conducts deep research on any data using any LLM providers
Agent Reach
★ 20.1kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Xiaohongshu
★ 13.7kMCP for xiaohongshu.com
Browse More Search & Data Extraction MCP Servers
Explore all search & data extraction servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up MediaCrawler MCP Server in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use MediaCrawler MCP Server?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.