LLM Search

Install the package from PyPI using pip. It is recommended to use a virtual environment to avoid dependency conflicts.

python -m venv .venv
source .venv/bin/activate
pip install llmsearch

Create a YAML configuration file that tells pyLLMSearch where your documents are, which embedding model to use, and how to configure search. Use the sample templates from the repository as a starting point.

# docs_config.yaml
documents:
  - path: /path/to/your/documents
    extensions: [".pdf", ".md", ".docx"]

embeddings:
  model_name: sentence-transformers/all-MiniLM-L6-v2
  persist_dir: /path/to/embeddings-store

search:
  hyde_enabled: true
  reranking_enabled: true

Create a second YAML file that specifies which LLM to use for answering questions. You can use OpenAI, Azure OpenAI, or a local model via LiteLLM and Ollama.

# llm_config.yaml
model:
  type: openai
  model_name: gpt-4o
  api_key: sk-your-openai-api-key

Run the embedding generation step. This processes your documents and stores vector embeddings in the persist directory you configured. Re-run this step when you add new documents.

llmsearch index create --config docs_config.yaml

Launch pyLLMSearch as an SSE-based MCP server. Clients like Cursor, Windsurf, and VS Code can connect to it for RAG-powered document queries.

llmsearch app mcp --docs-config docs_config.yaml --llm-config llm_config.yaml

Add the MCP server endpoint to your editor's MCP configuration. The server listens on localhost:8080 by default over SSE.

{
  "mcpServers": {
    "llm-search": {
      "url": "http://localhost:8080/sse"
    }
  }
}