LLM Orchestration Toolkit
LLM orchestration toolkit for agent workflows: planner + workers + synthesis, optional router (LLM + learned fallback), supports OpenAI/Anthropic/Ollama/llama.cpp, real scraping with caching, MCP server integration, and a TUI chat UI.
What is LLM Orchestration Toolkit?
LLM Orchestration Toolkit is a Model Context Protocol (MCP) server that allows AI assistants like Claude, Cursor, and VS Code to llm orchestration toolkit for agent workflows: planner + workers + synthesis, optional router (llm + learned fallback), supports openai/anthropic/ollama/llama.cpp, real scraping with caching, mcp serv...
LLM orchestration toolkit for agent workflows: planner + workers + synthesis, optional router (LLM + learned fallback), supports OpenAI/Anthropic/Ollama/llama.cpp, real scraping with caching, MCP server integration, and a TUI chat UI.
This server falls under the Coding Agents category on MCPgee, the world's largest MCP server directory with 33,000+ servers.
Features
- LLM orchestration toolkit for agent workflows: planner + wor
Use Cases
Maintainer
Works with
Installation
Manual Installation
npx llm-useConfiguration
Configuration Details
claude_desktop_config.json
Performance
Response Metrics
Resource Usage
How to Set Up and Use LLM Orchestration Toolkit
llm-use is an LLM orchestration toolkit that implements a planner-plus-workers architecture, routing tasks across multiple AI providers — Anthropic, OpenAI, Ollama, and llama.cpp — in a single coordinated workflow. It includes a learned router with token-vector similarity for automatic model selection, real web scraping with caching, a terminal UI for interactive chat, session history persistence, and an MCP server integration via PolyMCP. Developers building multi-LLM agent pipelines who need provider-agnostic orchestration with cost tracking and offline support will find it a capable foundation.
Prerequisites
- Python 3.10+ with pip
- At least one LLM provider: ANTHROPIC_API_KEY for Anthropic, OPENAI_API_KEY for OpenAI, or Ollama installed locally
- Optional: Playwright for dynamic web scraping, uvicorn for the MCP server mode
- An MCP-compatible client such as Claude Desktop or Claude Code
Clone the repository and install dependencies
Clone the llm-use repository and install the core package along with desired provider libraries.
git clone https://github.com/llm-use/llm-use.git
cd llm-use
pip install -e .
pip install anthropic openai # cloud providers
pip install polymcp uvicorn # MCP server supportSet provider API keys
Export your API keys as environment variables. At minimum you need one provider configured.
export ANTHROPIC_API_KEY=your_anthropic_key
export OPENAI_API_KEY=your_openai_keyRun a basic orchestrated task from the CLI
Test the planner-worker pipeline by running a task with an orchestrator model and a worker model. Cost breakdown is printed at the end.
python3 cli.py exec \
--orchestrator anthropic:claude-3-7-sonnet-20250219 \
--worker ollama:llama3.1:8b \
--task "Summarize the key differences between REST and GraphQL"Launch the TUI chat interface
Start the interactive terminal UI to have a multi-turn conversation routed through the orchestrator-worker pipeline.
python3 cli.py chat \
--orchestrator anthropic:claude-3-7-sonnet-20250219 \
--worker ollama:llama3.1:8bStart the MCP server for integration with AI clients
Launch llm-use as an MCP server using PolyMCP and uvicorn so Claude Desktop or other clients can use it as a tool provider.
uvicorn llm_use.mcp_server:app --host 0.0.0.0 --port 8000Add to your MCP client configuration
Register the running llm-use MCP server endpoint in your Claude Desktop or Claude Code config file.
{
"mcpServers": {
"llm-use": {
"url": "http://localhost:8000",
"env": {
"ANTHROPIC_API_KEY": "your_anthropic_key",
"OPENAI_API_KEY": "your_openai_key"
}
}
}
}LLM Orchestration Toolkit Examples
Client configuration
Connect Claude Desktop to a locally running llm-use MCP server.
{
"mcpServers": {
"llm-use": {
"url": "http://localhost:8000",
"env": {
"ANTHROPIC_API_KEY": "your_anthropic_key"
}
}
}
}Prompts to try
Example tasks you can run via the CLI or after connecting through MCP.
- "Compare 10 products across these URLs and give me a ranked recommendation"
- "Plan and execute a research task on quantum computing advances in 2024 using web scraping"
- "Run a planner-worker pipeline to write a detailed blog post outline on AI safety"
- "Summarize 3 web pages on topic X with caching enabled to avoid re-fetching"Troubleshooting LLM Orchestration Toolkit
Provider not found or model name error
Use the exact provider prefix format: anthropic:model-name, openai:model-name, or ollama:model-name. Run ollama list to see available local models.
Web scraping fails with dynamic pages
Install Playwright and its browser binaries: pip install playwright && playwright install. Then add --enable-scrape --scrape-backend playwright to your exec command.
Session history not persisting between runs
Session logs are stored in ~/.llm-use/sessions. Ensure the directory is writable. If using Docker or a restricted environment, mount this path as a volume.
Frequently Asked Questions about LLM Orchestration Toolkit
What is LLM Orchestration Toolkit?
LLM Orchestration Toolkit is a Model Context Protocol (MCP) server that llm orchestration toolkit for agent workflows: planner + workers + synthesis, optional router (llm + learned fallback), supports openai/anthropic/ollama/llama.cpp, real scraping with caching, mcp server integration, and a tui chat ui. It connects AI assistants to external tools and data sources through a standardized interface.
How do I install LLM Orchestration Toolkit?
Follow the installation instructions on the LLM Orchestration Toolkit GitHub repository. Clone the repo, install dependencies, and add the server config to your AI client.
Which AI clients work with LLM Orchestration Toolkit?
LLM Orchestration Toolkit works with all major MCP-compatible AI clients including Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot), Windsurf, and Cline.
Is LLM Orchestration Toolkit free to use?
Yes, LLM Orchestration Toolkit is open source and available under the MIT license. You can use it freely in both personal and commercial projects.
LLM Orchestration Toolkit Alternatives — Similar Coding Agents Servers
Looking for alternatives to LLM Orchestration Toolkit? Here are other popular coding agents servers you can use with Claude, Cursor, and VS Code.
Dify
★ 142.2kProduction-ready platform for agentic workflow development.
Ruflo
★ 54.0k🌊 The leading agent orchestration platform for Claude. Deploy intelligent multi-agent swarms, coordinate autonomous workflows, and build conversational AI systems. Features enterprise-grade architecture, self-learning swarm intelligence, RAG integrat
Goose
★ 45.7kan open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM
Antigravity Awesome Skills
★ 38.3kInstallable GitHub library of 1,400+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.
AgentScope
★ 25.5kBuild and run agents you can see, understand and trust.
Serena
★ 24.5kA coding agent toolkit that provides IDE-like semantic code retrieval and editing tools, enabling LLMs to efficiently navigate and modify codebases using symbol-level operations instead of basic file reading and string replacements.
Browse More Coding Agents MCP Servers
Explore all coding agents servers available in the MCPgee directory. Each server includes setup guides for Claude, Cursor, and VS Code.
Set Up LLM Orchestration Toolkit in Your Editor
Choose your AI client for step-by-step setup instructions.
Quick Config Preview
Add this to your claude_desktop_config.json or .cursor/mcp.json
Ready to use LLM Orchestration Toolkit?
Browse our complete directory of 33,000+ MCP servers, read setup guides for your editor, and start building with the Model Context Protocol.