Your Agents Forget. Neural Junkie Remembers — Without Sending the Whole Transcript.

You ask a follow-up in a long DM or collab channel: "What did we decide about the auth middleware?"

The agent answers like the earlier thread never happened.

That isn't a model quality problem. It's a context window budget problem — and every multi-agent desktop hub hits it.

Neural Junkie is my personal open-source hub for running AI specialists on your hardware. Agents only received the last few channel messages plus a short rolling summary. Everything else lived in SQLite but never came back at query time.

Conversation memory fixes that — without sending the whole transcript to the LLM.

Three layers of memory

Every turn still uses working memory:

Tail history — the last 2–10 messages for conversational continuity
Session summary — a ~2KB rolling compression of recent turns (on DMs and public channels)

The new layer is long-term memory:

Conversation memory — embed the user's latest question, search indexed past messages and collab markdown, inject the top matches as === RELEVANT PAST CONTEXT ===

You don't widen the window. You retrieve on demand.

What gets indexed

On write, the hub chunks and embeds:

Persisted chat messages — the same content that would appear in LLM history (noise filtered out)
Collab artifacts — plan.md, planning-summary.md, session-summary.md, findings.md, and other collabs/<id>/*.md deliverables

Collab channels are the biggest win: they never had session summaries. A three-hour /collaborate run could lose institutional decisions by message 50. Now those decisions stay searchable.

Local-first — no vector DB service

I didn't add Chroma or Qdrant. The stack reuses what Neural Junkie already runs:

Ollama nomic-embed-text for vectors
memory.db — SQLite chunk store beside messages.db
Brute-force cosine + keyword prefilter — fine for desktop-scale history

Default on when embed is available. Toggle in Settings → AI & providers → Conversation memory.

Clear channel history clears the memory index for that channel too.

How retrieval stays safe

Retrieval is scoped:

Message chunks: same channel only
Collab artifacts: same collaboration id
Chunks already in tail history: excluded (no duplication)

Budget cap ~1.5KB injected — enough for a few relevant excerpts, not a transcript dump.

Debug endpoints: GET /api/memory/stats and GET /api/memory/query for Pack dev and troubleshooting.

Try it

make pull-models   # includes nomic-embed-text
make start-all

Download: https://github.com/camronwood/neural-junkie/releases/latest

Enable Retrieve relevant past messages under Settings if you turned it off.

Long thread smoke test:

./scripts/test-conversation-memory.sh

If you run long collabs or DMs and hit a retrieval edge case — wrong channel scope, missing collab artifact — GitHub issues welcome.

Camron Wood — Neural Junkie (personal project)