Robin/openwiki-wiki
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f55af056a7d7b4d44f62a7434cdecd3ab3d85efa
openwiki-wiki/themes/04-local-llm.md
T

219 lines
18 KiB
Markdown
Raw Blame History

---
type: "Reference"
title: "Theme 4 · Local LLM deployment (Ollama / bge-reranker / RAGFlow)"
description: "Synthesis page for Robin's local-LLM fleet — Ollama serve + qwen2.5vl:7b vision model + BAAI/bge-m3 embeddings (Ollama + LM Studio) + bge-reranker rerank + RAGFlow KB retrieval + adjacent NAS subsystems (NocoDB / SeaFile / Wiki.js / Feishu mail)."
tags: [theme, local-llm, ollama, bge-reranker, ragflow, workflow-tooling]
---
# Theme 4 · Local LLM deployment (Ollama / bge-reranker / RAGFlow)
This is the synthesis page for **本地大模型部署** — Robin's local inference and retrieval layer that serves the entire agent stack documented in [`/themes/03-ai-agents.md`](/themes/03-ai-agents.md). The fleet combines an Ollama serve with multimodal + embedding models, an LM Studio instance running `BAAI/bge-m3`, a bge-reranker rerank service, and a RAGFlow KB layer; four adjacent NAS subsystems (NocoDB / SeaFile / Wiki.js / Feishu mail) sit alongside RAGFlow and share the same daily cron health-check pipeline.
## What this theme covers
- **Local inference runtime:** Ollama serve on the desktop (host `192.168.192.67`, port `11434`) running `qwen2.5vl:7b` for vision and `nomic-embed-text` / `BAAI/bge-m3` for embeddings.
- **Local reranker:** bge-reranker served on desktop port `8090`.
- **Local embedder (LM Studio):** `BAAI/bge-m3` on port `7997` (LM Studio OpenAI-compatible API).
- **KB retrieval layer:** RAGFlow (`http://10.10.10.254:9980`) — 18 KB / 32,709 docs / 116,254 chunks as of last sync 2026-08-03.
- **Adjacent NAS subsystems:** NocoDB (`http://10.10.10.254:8081`), SeaFile (`http://10.10.10.254:9898`), Wiki.js (`http://10.10.10.254:3000`), Feishu mail (IMAP polling, awaiting Robin's IT password).
## Local-LLM fleet architecture
<!-- openwiki: mermaid parse failed and this diagram was converted to a text fence so it does not break rendering. Fix the diagram source and restore the mermaid fence. Parser error: Heuristic: an unescaped angle bracket inside a label breaks rendering; rephrase the label. -->
```text
flowchart TD
subgraph Desktop["Desktop · 192.168.192.67"]
OllamaServe["Ollama serve<br/>:11434"]
QwenV["qwen2.5vl:7b<br/>(vision)"]
NomicE["nomic-embed-text<br/>(embed)"]
LMStudio["LM Studio<br/>:7997"]
BgeM3LM["BAAI/bge-m3<br/>(embed via LM Studio)"]
Reranker["bge-reranker<br/>:8090"]
OllamaServe --> QwenV
OllamaServe --> NomicE
LMStudio --> BgeM3LM
end
subgraph NAS["NAS · 10.10.10.254"]
RAGFlow["RAGFlow KB<br/>:9980<br/>18 KB / 32,709 docs"]
NocoDB["NocoDB<br/>:8081<br/>13 lit tables"]
SeaFile["SeaFile<br/>:9898<br/>4 repos / 77 GB"]
WikiJS["Wiki.js<br/>:3000<br/>~1500 pages"]
Feishu["Feishu mail<br/>IMAP poll (awaiting creds)"]
end
subgraph Agents["Agents (see theme 03)"]
Hermes[Hermes Agent]
MiniMaxCode[MiniMax code]
CronDaily["cron 02:30<br/>update-sources.sh"]
end
RAGFlow -.embedding.-> OllamaServe
RAGFlow -.embedding.-> LMStudio
RAGFlow -.rerank.-> Reranker
Hermes -->|query| RAGFlow
MiniMaxCode -->|cross-query| RAGFlow
Hermes -->|metadata| NocoDB
Hermes -->|PDF retrieval| SeaFile
Hermes -->|published deep reads| WikiJS
MiniMaxCode -.future.-> Feishu
CronDaily -->|health probe| RAGFlow
CronDaily -->|count refresh| NocoDB
CronDaily -->|count refresh| SeaFile
CronDaily -->|count refresh| WikiJS
QwenV -.vision OCR.-> LDR["LDR SOP<br/>(theme 05)"]
```
The diagram shows the three local inference services (Ollama serve, LM Studio, bge-reranker) feeding the RAGFlow KB layer, with the four NAS subsystems (NocoDB / SeaFile / Wiki.js / Feishu mail) attached at the metadata / file / published-deep-read / IMAP layer. All three inference services and all four subsystems are probed nightly by `update-sources.sh` at 02:30.
## Service-down incident (2026-07-17 → ongoing)
This is the **dominant operational fact** for the theme right now.
- **Incident start:** Desktop AI service on `192.168.192.67` stopped responding at 2026-07-17. Affected ports: `7997` (LM Studio embed), `8090` (bge-reranker), `11434` (Ollama serve). All three local inference endpoints are simultaneously offline. Documented in [`/sources/2026-07-27-4-27-sop-vs-implementation.md`](/sources/2026-07-27-4-27-sop-vs-implementation.md) §4 (Ollama visual-review degradation).
- **Operational impact:** LDR SOP Step 4 (Ollama visual review of extracted PDF figures) cannot run — Robin's literature-deep-read pipeline falls back to human-in-the-loop visual review of SeaFile-hosted figures. Cross-agent RAGFlow queries fall back to metadata-only (NocoDB tables, Wiki.js GraphQL, SeaFile file listing) since vector retrieval depends on the embedder and reranker.
- **Recovery procedure:** documented at `/sources/2026-07-27-4-27-sop-vs-implementation.md` §4 — Robin to run `ollama serve &` and `ollama pull qwen2.5vl:7b` on the desktop, then verify with the `ollama_vision` smoke test in the same file.
- **Status note:** This is a watchlist item, not a `themes.md` row, because the underlying root cause (desktop service restart) is operational not recurring. Tracked implicitly through the LDR SOP and the daily cron health probe — when RAGFlow's `/api/v1/health` returns 200 and `/api/v1/datasets` returns 401 (auth-only), the KB inventory is still readable; the dependency is on the embed/rerank services for retrieval quality.
## RAGFlow KB inventory (18 KB / 32,709 docs / 116,254 chunks)
Snapshot from [`/sources/ragflow.md`](/sources/ragflow.md) (last sync 2026-08-03T18:30:02Z, landing 200, `/api/v1/health` 404, `/api/v1/datasets` 401 — auth confirmed, retrieval path not exercised since the incident).
| KB name | Dataset ID (prefix) | Docs | Chunks | Embed model |
| --- | --- | ---: | ---: | --- |
| 废弃库 | `daa9ee226765` | 15,922 | 1,604 | nomic-embed-text@Ollama |
| Organoid1 | `8dfa438274e5` | 7,815 | 8,714 | BAAI/bge-m3@LM-Studio |
| Organoid2 | `044cdb547620` | 4,974 | 5,888 | BAAI/bge-m3@LM-Studio |
| Organoid | `3c063ce2559c` | 2,514 | 30,978 | nomic-embed-text@Ollama |
| 3Brian | `06af5550559d` | 454 | 19,374 | nomic-embed-text@Ollama |
| Cellink | `cf973e0e559f` | 340 | 26,200 | nomic-embed-text@Ollama |
| Celvivo | `1555e5b27832` | 154 | 9,027 | BAAI/bge-m3@LM-Studio |
| RAGFlow | `e3203cec58ff` | 128 | 1,268 | nomic-embed-text@Ollama |
| FemtoBiomed_electroporation | `c97eb5606aa8` | 123 | 142 | BAAI/bge-m3@LM-Studio |
| Celvivo1 | `a83244a855c2` | 90 | 4,699 | nomic-embed-text@Ollama |
| seer | `79bfa1386a26` | 81 | 3,194 | text-embedding-v4@Tongyi-Qianwen |
| cellshot | `35d3e1bc680c` | 73 | 3,556 | BAAI/bge-m3@LM-Studio |
| LaminarWash | `1ccc414278d2` | 21 | 1,257 | BAAI/bge-m3@LM-Studio |
| Celvivo commercial | `922cde0880aa` | 10 | 40 | BAAI/bge-m3@LM-Studio |
| Cellshot厂家 | `437d31de7a7b` | 5 | 49 | BAAI/bge-m3@LM-Studio |
| huibaibio | `92604bda7aab` | 3 | 107 | BAAI/bge-m3@LM-Studio |
| Organoid3 | `29d5de0a7ff5` | 1 | 57 | BAAI/bge-m3@LM-Studio |
| nanotemper | `496748f66a26` | 1 | 100 | nomic-embed-text@Ollama |
| **TOTAL** | — | **32,709** | **116,254** | — |
Embed-model distribution shows the migration story: earlier KBs (Organoid, 3Brain, Cellink, RAGFlow, Celvivo1, nanotemper, 废弃库) were embedded with `nomic-embed-text@Ollama`; later KBs (Organoid1, Organoid2, Celvivo, FemtoBiomed_electroporation, cellshot, LaminarWash, Celvivo commercial, Cellshot厂家, huibaibio, Organoid3) were embedded with `BAAI/bge-m3@LM-Studio`. The `seer` KB is the only one using a non-local embedder (`text-embedding-v4@Tongyi-Qianwen`). This is a real production fleet, not a single notebook.
### RAGFlow 0.17 server-side cap (operational note)
`/sources/ragflow-documents.md` documents that RAGFlow 0.17 server-side caps `/api/v1/documents` to 30 docs per request and silently ignores the `offset` parameter. Coverage in the documents sample:
- 6 small KBs (≤30 docs): 100% coverage
- 12 large KBs (>30 docs): only first 30 sampled, no pagination possible from the API alone
- Total sample: 401 doc-name rows across 18 KBs
Workaround for full enumeration: read the KB directly through Wiki.js, NocoDB metadata, or the `RAGFlow` doc PDF dump under SeaFile. Documented at [`/sources/ragflow-documents.md`](/sources/ragflow-documents.md) header.
## Adjacent NAS subsystems (one-line summaries)
Each subsystem has its own source page; the bullets below are the cross-agent query affordances.
### NocoDB — metadata store
- **Endpoint:** `http://10.10.10.254:8081` · 13 lit tables · 6,604 records (2026-07-27 snapshot; **`/sources/nocodb.md` last-sync timestamp `$(date -u +%Y-%m-%dT%H:%M:%SZ)` is unrendered** — the cron that wrote the source page left the template unfilled).
- **Per-table scale** (last full census): `brain_organoid_literature` 103, `intestinal_organoid_literature` 9, `Celvivo_literature` 109, `cardiac_organoid_literature` 11, `cancer_organoid_literature` 129, `liver_organoid_literature` 28, `other_organoid_literature` 724, `kidney_organoid_literature` 23, `retinal_organoid_literature` 4, `Cellink_literature` 2,499 (largest), `organoid_literature` (old) 2,777, `lung_organoid_literature` 48, `3Brain_literature` 147.
- **API quirk:** Cellink library `已精读` SingleSelect `options=[]` (NocoDB bug) — cannot PATCH "是". Documented at [`/sources/2026-07-27-4-27-sop-vs-implementation.md`](/sources/2026-07-27-4-27-sop-vs-implementation.md) §3. Workaround: write 精读摘要 + 精读日期 to PATCH-able fields; mark `已精读 = 是` via the wiki synthesis layer instead.
- **Cross-link:** [`/sources/nocodb.md`](/sources/nocodb.md)
### SeaFile — large-file storage
- **Endpoint:** `http://10.10.10.254:9898/api2/` · public domain `https://fb.biokingdom.top` for share-links · 4 repos / 77 GB total.
- **API quirk:** port 5000 is the Web UI reverse proxy, **not** the API. Token auth via `Authorization: Token ***` header. Share-link `?dl=1` suffix is required for direct download.
- **Daily use:** LDR SOP Step 5 uploads PDF + 3 extracted figures → SeaFile → embeds `![]()` markdown in Obsidian / Wiki.js / OpenWiki. Documented at [`/sources/ldr-report-template.md`](/sources/ldr-report-template.md) §4 and §5.
- **Cross-link:** [`/sources/seafile.md`](/sources/seafile.md)
### Wiki.js — published deep-read layer
- **Endpoint:** `http://10.10.10.254:3000/graphql` · estimated ≥ 1,648 pages (based on max page id 1633) · zh/literature subpath is the primary publishing target.
- **API quirk:** `pages.list(limit:N)` server-side caps to ~50 regardless of N; real count requires 5-orderBy union (TITLE / UPDATED / CREATED / ID / PATH) — documented at [`/sources/wikijs.md`](/sources/wikijs.md) header. The "47" number from earlier 7-27 evidence is a server-side cap artifact, not the true count.
- **Daily use:** LDR SOP Step 7 publishes deep-read reports to Wiki.js for cross-device / customer-facing reading. Maintenance is owned by `wikijs-obsidian-sync` skill (Robin + LDR auto-publish).
- **Cross-link:** [`/sources/wikijs.md`](/sources/wikijs.md)
### Feishu mail — IMAP polling (awaiting credentials)
- **Endpoint:** `imap.feishu.cn:993` (CN) or `imap.larksuite.com:993` (INTL) · `LOGIN` auth · 16-char app password required.
- **Status:** script `~/.local/bin/openwiki-update-feishu-mail.sh` is drafted (Robin 2026-07-29 spec); cron entry `*/30 * * * *` is not yet in `crontab`. Page at [`/sources/feishu-mail.md`](/sources/feishu-mail.md) is intentionally blank pending credentials.
- **Operational rule (Robin 7-28):** Feishu mailbox **is not** the same as Gmail — `openwiki auth google` does not reach it. Feishu needs a separate IMAP credentials path through `~/.openwiki_env.sh`.
- **Cross-link:** [`/sources/feishu-mail.md`](/sources/feishu-mail.md)
## Daily operations
Two cron pipelines drive the local-LLM fleet:
<!-- openwiki: mermaid parse failed and this diagram was converted to a text fence so it does not break rendering. Fix the diagram source and restore the mermaid fence. Parser error: Heuristic: an unescaped angle bracket inside a label breaks rendering; rephrase the label. -->
```text
flowchart LR
subgraph Cron1["02:00 daily · openwiki-update.sh"]
HN[Hacker News ingest]
Tavily[Tavily web search ingest]
GitRepo[git-repo ingest]
M3["M3 agent synthesis<br/>writes /themes + /commitments"]
end
subgraph Cron2["02:30 daily · openwiki-update-sources.sh"]
NocoDBP[NocoDB count refresh]
SeaFileP[SeaFile count refresh]
WikiJSP[Wiki.js count refresh]
RAGFlowP[RAGFlow health probe]
end
subgraph Cron3["*:30 daily · openwiki-update-feishu-mail.sh (waiting)"]
FeishuP[Feishu INBOX poll]
end
subgraph Runtime["Local inference runtime"]
OllamaEmbed["Ollama :11434<br/>nomic-embed-text"]
LMStudioEmbed["LM Studio :7997<br/>BAAI/bge-m3"]
RerankerSvc["bge-reranker :8090"]
end
subgraph Sink["Wiki sink"]
OpenWiki[OpenWiki<br/>commit and push Gitea]
end
Cron1 --> OpenWiki
Cron2 --> OpenWiki
Cron3 -.future.-> OpenWiki
RAGFlowP -.depends on.-> OllamaEmbed
RAGFlowP -.depends on.-> LMStudioEmbed
RAGFlowP -.depends on.-> RerankerSvc
```
The 02:00 cron pulls 3 connectors and runs M3 agent synthesis (writes to `/themes/` and `/commitments/`). The 02:30 cron pulls counts from the 4 NAS subsystems. The 30-min Feishu cron is queued but inactive until IMAP creds land. After each cron, OpenWiki commits and pushes to the Gitea public mirror.
- **Health check:** `~/.local/bin/openwiki-update-sources.sh` probes each NAS subsystem's API; on success the source page is rewritten; on failure the previous good state remains.
- **Cross-agent query:** Hermes / MiniMax code reads `sources/{ragflow,nocodb,seafile,wikijs,feishu-mail}.md` to discover "what's in this system" without an SQL/GraphQL round-trip; the 4 OpenWiki bridge skills route the cross-agent queries (see [`/themes/03-ai-agents.md`](/themes/03-ai-agents.md) §"MiniMax code").
## Cross-cutting relationship
- **Consumed by [`/themes/03-ai-agents.md`](/themes/03-ai-agents.md)** — every Hermes / MiniMax code / Codex call depends on the local LLM fleet (Ollama / bge-reranker / RAGFlow). The 2026-07-17 desktop service-down incident is the live dependency that breaks the cross-agent workflow.
- **Hosts the evidence layer for [`/themes/01-organoid-equipment.md`](/themes/01-organoid-equipment.md), [`/themes/02-bioprinting.md`](/themes/02-bioprinting.md), and [`/themes/05-robin-research.md`](/themes/05-robin-research.md)** — RAGFlow KBs (Celvivo / 3Brian / Cellink / Organoid1-3 / etc.) are the deep retrieval surface behind every vendor deep read; NocoDB is the metadata catalog; Wiki.js is the public publishing target.
- **Privacy rule (Robin 7-28):** Feishu mailbox is **not** Gmail. The `/sources/feishu-mail.md` page exists precisely to prevent future agents from trying `openwiki auth google` against it. Permanent memory record.
## Backlog for this theme
- **Bring desktop AI service back up** — restart `ollama serve` + `ollama pull qwen2.5vl:7b` + LM Studio + bge-reranker on `192.168.192.67`. Without this, the LDR SOP visual review step is degraded and RAGFlow vector retrieval is metadata-only.
- **Fix NocoDB `/sources/nocodb.md` unrendered timestamp** — the page contains literal `$(date -u +%Y-%m-%dT%H:%M:%SZ)` in the header; needs the cron template to actually substitute the variable. Documented but not yet fixed at the source page level.
- **SeaFile source-page library-name template** — `/sources/seafile.md` has 6 library rows with `?` placeholders for ID / size / file count / last update; the upstream `update-sources.sh` is not filling them. Either fix the script or remove the table.
- **RAGFlow `废弃库` (15,922 docs / 1,604 chunks) cleanup question** — Robin 7-27 said "永久不动". Currently the largest KB by doc count but only the 4th-largest by chunk count. Surface again only if Robin changes stance; do not auto-clean.
- **Wiki.js ≥ 1,648 page real count** — server-side cap of ~50 in `pages.list(limit:N)` is a known RAGFlow 0.17-style artifact. Full enumeration requires either a Wiki.js DB-direct query or a frontend crawler. Documented as a known limitation in [`/sources/wikijs.md`](/sources/wikijs.md).
- **Feishu IMAP integration** — script drafted; awaiting Robin IT to enable IMAP/SMTP and provide a 16-char app password. Once landed, `crontab` adds `*/30 * * * *` entry and `/sources/feishu-mail.md` starts receiving real evidence rows.
## What to read next
| Question | Go to |
| --- | --- |
| What is the RAGFlow KB inventory? | [`/sources/ragflow.md`](/sources/ragflow.md) |
| What does each RAGFlow KB actually contain (sample docs)? | [`/sources/ragflow-documents.md`](/sources/ragflow-documents.md) |
| What are the 13 NocoDB lit tables? | [`/sources/nocodb.md`](/sources/nocodb.md) |
| What is in SeaFile? | [`/sources/seafile.md`](/sources/seafile.md) |
| What is in Wiki.js? | [`/sources/wikijs.md`](/sources/wikijs.md) |
| What is the Feishu mail integration status? | [`/sources/feishu-mail.md`](/sources/feishu-mail.md) |
| How are the LDR PDFs and figures uploaded to SeaFile? | [`/sources/ldr-report-template.md`](/sources/ldr-report-template.md) §4-5 |
| What did the 7-27 SOP-vs-implementation pass reveal about the service-down incident? | [`/sources/2026-07-27-4-27-sop-vs-implementation.md`](/sources/2026-07-27-4-27-sop-vs-implementation.md) §4 |
<!-- openwiki: broken internal link [/sources/README-ACTIVATION.md] file "/sources/README-ACTIVATION.md" does not exist. Fix the href or restore the target, then delete this comment. -->
| What is the daily cron pipeline? | [`/sources/README-ACTIVATION.md`](/sources/README-ACTIVATION.md) §3 |
| How does MiniMax code read this stack? | [`/sources/OpenWiki速查手册.md`](/sources/OpenWiki速查手册.md) §5.8 |
| How does the agent layer depend on this fleet? | [`/themes/03-ai-agents.md`](/themes/03-ai-agents.md) |
Reference in New Issue View Git Blame Copy Permalink