LLM Server Docs
End-to-end documentation to set up your own local & fully private LLM server on Debian. Equipped with chat, web search, RAG, model management, MCP servers, image generation, and TTS.
What is LLM Server Docs?
LLM Server Docs is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to end-to-end documentation to set up your own local & fully private llm server on debian. equipped with chat, web search, rag, model management, mcp servers, image generation, and tts.
End-to-end documentation to set up your own local & fully private LLM server on Debian. Equipped with chat, web search, RAG, model management, MCP servers, image generation, and TTS.
This server falls under the Developer Tools category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- End-to-end documentation to set up your own local & fully pr
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx llm-server-docsConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use LLM Server Docs
LLM Server Docs is a comprehensive documentation and configuration guide for running a fully private, local LLM server on Debian Linux, combining Ollama or llama.cpp for inference, Open WebUI for the chat interface, SearXNG for private web search, Kokoro FastAPI for text-to-speech, ComfyUI for image generation, and an MCP proxy for tool integration. It is not a standalone MCP server package but rather an end-to-end reference for developers who want to self-host a complete AI stack without sending data to external cloud providers. The guide covers GPU setup, Docker networking, RAG configuration, and MCP server integration for fetch and sequential-thinking.
Prerequisites
- Debian Linux system (or compatible) with an NVIDIA GPU for optimal performance
- Docker and Docker Compose installed (sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin)
- NVIDIA drivers and CUDA toolkit installed for GPU acceleration
- Basic familiarity with Docker networking and Linux system administration
Install Ollama for local model management
Ollama provides an OpenAI-compatible API and simplifies downloading and running local LLM models. Set OLLAMA_HOST to expose it on all interfaces within your Docker network.
curl -fsSL https://ollama.com/install.sh | sh
export OLLAMA_HOST=0.0.0.0Create a Docker network for the stack
All services communicate over a shared Docker bridge network so containers can reach each other by name.
docker network create -d bridge app-netDeploy Open WebUI with GPU support
Open WebUI provides the chat interface with model switching, voice calling, and web search integration. Mount the data volume so conversation history persists across restarts.
docker run -d -p 3000:8080 --network app-net --gpus all \
-v open-webui:/app/backend/data --name open-webui \
ghcr.io/open-webui/open-webui:cudaDeploy SearXNG for private web search
SearXNG aggregates results from 245+ search engines without tracking. Configure Open WebUI to use it by setting the web search engine to 'searxng' and pointing to the container URL.
docker run -d -p 5050:8080 --name searxng --network app-net \
-v "${PWD}/searxng:/etc/searxng" \
-e "BASE_URL=http://0.0.0.0:5050/" searxng/searxngConfigure MCP proxy for tool integration
Run an MCP proxy on port 3131 to expose MCP servers (fetch, sequential-thinking, searxng) to Open WebUI. Test connectivity from inside the Open WebUI container.
# Test MCP proxy connectivity from Open WebUI container
docker exec -i open-webui curl http://mcp-proxy:3131/servers/fetch/mcpDeploy Kokoro FastAPI for text-to-speech
Kokoro FastAPI provides an OpenAI-compatible TTS endpoint. Configure Open WebUI TTS settings to use OpenAI engine with the API base pointing to the Kokoro container.
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI && docker compose up --buildLLM Server Docs Examples
Client configuration
MCP proxy configuration for connecting fetch and sequential-thinking servers to Open WebUI.
{
"mcpServers": {
"fetch": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-fetch"]
},
"sequential-thinking": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-sequential-thinking"]
}
}
}Prompts to try
Example queries to use once your local LLM stack is running.
- "Search the web privately for the latest news about open-source LLMs"
- "Generate an image of a futuristic city at sunset using ComfyUI"
- "Read this article at https://example.com and summarize it"
- "What GPU is my server using right now? Run nvidia-smi"Troubleshooting LLM Server Docs
Containers cannot communicate with each other by hostname
Make sure all containers are started with '--network app-net'. Containers on different networks cannot reach each other by name. Run 'docker network inspect app-net' to see which containers are attached.
GPU is not detected inside Docker containers
Install the NVIDIA Container Toolkit ('nvidia-ctk') and restart the Docker daemon. Pass '--gpus all' to docker run commands that need GPU access. Verify with 'docker run --rm --gpus all nvidia/cuda:12-base nvidia-smi'.
Ollama models are not visible in Open WebUI
Set OLLAMA_HOST=0.0.0.0 before starting Ollama, and configure Open WebUI's Ollama API URL to point to your host machine's IP on port 11434 (not localhost, since Open WebUI runs in a container).
Frequently Asked Questions about LLM Server Docs
What is LLM Server Docs?
LLM Server Docs is a Model Context Protocol (MCP) server that end-to-end documentation to set up your own local & fully private llm server on debian. equipped with chat, web search, rag, model management, mcp servers, image generation, and tts. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install LLM Server Docs?
Follow the installation instructions on the LLM Server Docs GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with LLM Server Docs?
LLM Server Docs works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is LLM Server Docs free to use?
Yes, LLM Server Docs is open source and available under the MIT license. You can use it freely in both personal and commercial projects.
LLM Server Docs Alternatives — Similar Developer Tools Servers
Looking for alternatives to LLM Server Docs? Here are other popular developer tools servers you can use with Claude, Cursor, and VS Code.
Ecc
★ 188.2kThe agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
Javaguide
★ 155.8kJava 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发
Gemini CLI
★ 104.5kA secure MCP server that wraps the Google Gemini CLI, allowing clients to query Gemini models using local OAuth sessions without requiring an API key. It provides tools for model interaction and diagnostics with built-in protection against command in
Awesome MCP Servers
★ 87.3k⭐ Curated list of Model Context Protocol (MCP) servers - tools that extend Claude Desktop, Cursor, Windsurf, and other MCP clients with custom capabilities.
MCP Servers
★ 86.0kModel Context Protocol Servers
CC Switch
★ 77.5kA cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Gemini CLI & Hermes Agent. Only official website: ccswitch.io
Browse More Developer Tools MCP Servers
Explore all developer tools servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up LLM Server Docs in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use LLM Server Docs?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.