Local RAG
A fully offline local RAG server that utilizes ChromaDB and Ollama to index and query PDF, text, and Markdown documents. It allows users to manage local knowledge bases and perform semantic searches with AI-generated responses.
What is Local RAG?
Local RAG is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to fully offline local rag server that utilizes chromadb and ollama to index and query pdf, text, and markdown documents. it allows users to manage local knowledge bases and perform semantic searches wit...
A fully offline local RAG server that utilizes ChromaDB and Ollama to index and query PDF, text, and Markdown documents. It allows users to manage local knowledge bases and perform semantic searches with AI-generated responses.
This server falls under the Knowledge & Memory category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- query_documents
- ingest_file
- delete_file
- list_files
- status
Use Cases
Maintainer
Works with
Installation
NPM
npx -y mcp-local-ragManual Installation
npx -y mcp-local-ragConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use Local RAG
MCP Local RAG is a fully offline retrieval-augmented generation server that lets AI assistants search and query your local PDF, text, and Markdown documents without any cloud services or API keys. It uses LanceDB for vector storage and Xenova/all-MiniLM-L6-v2 as a locally-downloaded embedding model, combining semantic similarity with keyword boosting to surface technically precise results. Developers and knowledge workers use it to query their private documentation, code specs, and research notes while keeping all data on their own machine.
Prerequisites
- Node.js 18 or higher (package runs via npx)
- An MCP-compatible client such as Claude Desktop or Claude Code
- Local documents to index (PDF, .txt, .md files)
- Sufficient disk space for the embedding model cache (~90 MB) and LanceDB vector store
Add the server to your MCP client
For Claude Code, use the claude mcp add command to register the server with the BASE_DIR pointing to your documents folder. The package runs entirely via npx with no prior install step.
claude mcp add local-rag --scope user --env BASE_DIR=/path/to/your/documents -- npx -y mcp-local-ragConfigure Claude Desktop (alternative)
If using Claude Desktop, add the server block to claude_desktop_config.json. Set BASE_DIR to your document root and optionally configure DB_PATH for the vector store location.
{
"mcpServers": {
"local-rag": {
"command": "npx",
"args": ["-y", "mcp-local-rag"],
"env": {
"BASE_DIR": "/path/to/your/documents",
"DB_PATH": "./lancedb/",
"CACHE_DIR": "./models/"
}
}
}
}Wait for the embedding model to download on first run
On first startup, the server downloads the Xenova/all-MiniLM-L6-v2 embedding model (~90 MB) into CACHE_DIR. Subsequent starts are immediate. No internet access is needed after this download.
Ingest your documents
Ask your AI client to ingest files, or use the CLI directly. The server chunks documents by semantic meaning (not fixed character counts) and stores embeddings in LanceDB.
npx mcp-local-rag ingest ./docs/Query your knowledge base
Once documents are ingested, ask natural language questions. The server uses vector search boosted by keyword matching to return high-relevance chunks. Check ingestion status with the status tool.
npx mcp-local-rag query "authentication API"Local RAG Examples
Client configuration
Claude Desktop config for mcp-local-rag with document root and optional vector DB path.
{
"mcpServers": {
"local-rag": {
"command": "npx",
"args": ["-y", "mcp-local-rag"],
"env": {
"BASE_DIR": "/Users/yourname/documents",
"DB_PATH": "/Users/yourname/.mcp-local-rag/lancedb/",
"CACHE_DIR": "/Users/yourname/.mcp-local-rag/models/",
"MODEL_NAME": "Xenova/all-MiniLM-L6-v2"
}
}
}
}Prompts to try
Example prompts for ingesting documents and running semantic searches.
- "Ingest the document at /Users/me/docs/api-spec.pdf"
- "What does the API documentation say about authentication and token refresh?"
- "List all files that have been ingested and their status"
- "Search my documents for anything related to useEffect cleanup patterns"
- "That result about rate limiting looks relevant — read the surrounding chunks for more context"Troubleshooting Local RAG
Embedding model download fails or server hangs on first start
Check that your CACHE_DIR path is writable and you have internet access for the initial model download. After the first download, the server works fully offline. You can also pre-download the model manually using the @xenova/transformers package.
Ingest fails for large PDF files
The default MAX_FILE_SIZE is 100 MB (104857600 bytes). For larger files, set the MAX_FILE_SIZE env var to a higher value in your MCP config. Also ensure the file is not password-protected.
Search returns irrelevant results for exact technical terms like class names or error codes
Adjust the RAG_HYBRID_WEIGHT env var (default 0.6) upward toward 1.0 to give more weight to keyword matching over semantic similarity. This improves recall for exact identifiers and error codes.
Frequently Asked Questions about Local RAG
What is Local RAG?
Local RAG is a Model Context Protocol (MCP) server that fully offline local rag server that utilizes chromadb and ollama to index and query pdf, text, and markdown documents. it allows users to manage local knowledge bases and perform semantic searches with ai-generated responses. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install Local RAG?
Install via npm with the command: npx -y mcp-local-rag. Then add the server configuration to your AI client's JSON config file (e.g., claude_desktop_config.json or .cursor/mcp.json).
Which AI clients work with Local RAG?
Local RAG works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is Local RAG free to use?
Yes, Local RAG is open source and available under the MIT License license. You can use it freely in both personal and commercial projects.
Local RAG Alternatives — Similar Knowledge & Memory Servers
Looking for alternatives to Local RAG? Here are other popular knowledge & memory servers you can use with Claude, Cursor, and VS Code.
MemPalace
★ 52.6kA local AI memory system that stores all conversations verbatim and organizes them into navigable structures. It provides 19 MCP tools for AI assistants to search and retrieve past decisions, debugging sessions, and architecture debates automatically
Kratos
★ 25.7k🏛️ Memory System for AI Coding Tools - Never explain your codebase again. MCP server with perfect project isolation, 95.8% context accuracy, and the Four Pillars Framework.
Context Mode
★ 15.4kAn MCP server that preserves LLM context by intercepting large data outputs and returning only concise summaries or relevant sections. It enables efficient sandboxed code execution, file processing, and documentation indexing across multiple programm
Memu
★ 13.7kMemory for 24/7 proactive agents like OpenClaw.
MemOS
★ 9.3kMemOS (Memory Operating System) is a memory management operating system designed for AI applications. Its goal is: to enable your AI system to have long-term memory like a human, not only remembering what users have said but also actively invoking, u
Everos
★ 5.4kBuild, evaluate, and integrate long-term memory for self-evolving agents.
Browse More Knowledge & Memory MCP Servers
Explore all knowledge & memory servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up Local RAG in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use Local RAG?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.