Skip to main content

What is ollmo

ollmo is a production-grade RAG (Retrieval-Augmented Generation) platform with hybrid retrieval, multi-tenant isolation, and streaming chat. Upload documents, ask questions, and get answers with citations:

Side-by-side chat and document with the cited passage highlighted

Backend: Go + Fiber. Frontend: Next.js App Router + shadcn/ui.

Core Features​

  • Hybrid Retrieval — Milvus dense vectors + MySQL FULLTEXT lexical matching, fused with RRF, with KB-level Rerank configuration; built-in retrieval testing with adjustable keyword ↔ semantic weights
  • Multi-Tenancy — Logical isolation across MySQL, Milvus, and MinIO; every table carries tenant_id
  • Document Pipeline — Upload (drag & drop, paste, multi-file) → parse (MinerU / local parser) → chunk → embed → index, fully asynchronous (Asynq + Redis)
  • Streaming Chat — Real-time SSE streaming with source citations; agent opening messages deliver a human-like greeting
  • Citation Tracing — Clicking a citation opens the source document with the cited passage highlighted in place: PDF rendering, paginated parsed view, and a side-by-side chat + document panel
  • Curated Q&A — Maintain human-curated standard Q&A for a knowledge base; exact matches return the standard answer directly
  • Auto-Memory — Long conversations are automatically summarized in the background (async LLM task); summaries are injected into later sessions for cross-conversation continuity
  • Configurable Models — Tenant-level management of LLM / Embedding / Rerank models, with per-KB overrides; a two-panel picker (providers on the left, models on the right)
  • Agent Canvas — Visually orchestrate agent pipelines (React Flow): intent classification → retrieval → condition branch → LLM → direct reply, with node-level debugging
  • Data Pipeline Canvas — Visually compose the ingestion DAG: source → parser → chunker → embedder → sink
  • GraphRAG — LLM-powered entity & relation extraction at ingestion; entity matching enriches retrieval context at query time
  • Usage & Cost — Team-level token / cost statistics: overview, per user, per model, and call details
  • Observability & Replay — Every Agent graph run is fully recorded: node paths, timings, and branch decisions are replayable, with one-click bad-case tracing (see Observability & Replay)
  • Team & Permissions — Invitation-based membership (admin / member), per-user daily message quota, private / team-shared knowledge bases
  • Admin Console — User management, team plans & quotas (Free / Pro / Enterprise), system settings, analytics dashboard, audit logs
  • External API — API-key authenticated programmatic access (retrieval, KB management, streaming chat)
  • Internationalization — Chinese / English UI, shared message contracts, no language code in URLs

Tech Stack​

LayerChoice
BackendGo 1.26+ / Fiber / GORM
FrontendNext.js 15 (App Router) / React / shadcn-ui
DatabaseMySQL 8.0 (utf8mb4) — metadata
Vector DBMilvus 2.5+ — dense vector retrieval
Object StorageMinIO (S3 compatible) — raw documents, parsing artifacts
Task QueueAsynq + Redis — parsing, embedding, index rebuild, auto-summary
Document ParserMinerU (containerized, optional GPU) + local parser
AuthLocal account (email + password + JWT)

Architecture​

                    ┌─────────────────────────────────────────────────────┐
                    │                   Next.js Web (3001)                 │
                    └────────────────────────┬────────────────────────────┘
                                             │ HTTP / SSE
                    ┌────────────────────────▼────────────────────────────┐
                    │                Go API Server (8080)                 │
                    │  Fiber · JWT auth · Tenant isolation · RESTful API  │
                    └───────┬────────────┬──────────────┬─────────────────┘
                            │            │              │
                   ┌────────▼──┐  ┌──────▼──────┐  ┌───▼────────────┐
                   │  MySQL    │  │   Redis     │  │    MinIO       │
                   │ metadata  │  │ task queue  │  │  file storage  │
                   └───────────┘  └──────┬──────┘  └────────────────┘
                                         │
                                ┌────────▼────────┐
                                │  Asynq Worker   │
                                │  parse · embed  │
                                │   auto-summary  │
                                └────────┬────────┘
                                         │
                                ┌────────▼────────┐
                                │     Milvus      │
                                │  dense vector   │
                                │    retrieval    │
                                └─────────────────┘

Next Steps​