Harbor

https://github.com/user-attachments/assets/e4897391-c5a8-4391-93c3-9f8b76155f11

Setup your local LLM stack effortlessly.

# Starts fully configured Open WebUI and llama.cpp
harbor up

# Now, Open WebUI can do Web RAG and TTS/STT
harbor up searxng speaches

Harbor is a CLI and companion app that lets you spin up a complete local LLM stack—backends like Ollama, llama.cpp, or vLLM, frontends like Open WebUI, plus supporting services like SearXNG for web search, Speaches for voice chat, and ComfyUI for image generation—all pre-wired to work together with a single harbor up command. No manual setup: just pick the services you want and Harbor handles the Docker Compose orchestration, configuration, and cross-service connectivity so you can focus on actually using your models.

News

• v0.5.14 - Eight new services (Latitude, AutoGen Studio, LobeHub, R2R, Caddy, RAGFlow, ClaraVerse, Qwen Code), a Dockerized Harbor CLI runner, external Ollama support for Boost and Open WebUI, opt-in CLI command wrappers, and fixes for ROCm detection, Perplexica on LAN, AnythingLLM settings persistence and the Harbor App on dot-prefixed homes
• v0.5.13 - harbor models pull no longer reports success when a model download actually failed, and the PrismML backend documents its two Bonsai 2 quants with measured speed and accuracy
• v0.5.12 - New PrismML llama.cpp backend that runs Ternary Bonsai 2 GGUFs on CPU, NVIDIA, and ROCm with the same model workflow and cross-service integrations as llama.cpp
• v0.5.11 - Halogen stays up through long Open WebUI generations with image 0.6.3, a 65536-token chat budget and a 900-second engine watchdog
• v0.5.10 - n8n first-time setup includes the Assistant code sandbox so new instances can finish the GUI wizard
• v0.5.9 - Hugging Face downloads run as your host user and recover root-owned caches automatically after upgrading
• v0.5.8 - New Halogen Flash Server backend for AMD Strix Halo and a fix for Hugging Face downloads failing on root-owned cache directories

Documentation

• Installing Harbor Guides to install Harbor CLI and App, including the agent install prompt for Claude Code, Codex, and similar tools
• Harbor User Guide High-level overview of working with Harbor
• Harbor App Overview and manual for the Harbor companion application
• Harbor Services Catalog of services available in Harbor
• Harbor CLI Reference Read more about Harbor CLI commands and options. Read about supported services and the ways to configure them.
• Harbor Guides Practical guides for local LLM stack setup, local web RAG, coding agents, OpenAI-compatible backends, and self-hosted AI stack planning.
• Join our Discord Get help, share your experience, and contribute to the project.

What can Harbor do?

✦ Local LLMs

Run LLMs and related services locally, with no or minimal configuration, typically in a single command or click.

# All backends are pre-connected to Open WebUI
harbor up ollama
harbor up llamacpp
harbor up vllm