SuperMemory

Local, attention-weighted long-term memory for Claude Code and Codex.

English · 简体中文

SuperMemory gives coding agents a memory that survives session boundaries and context compaction. Everything lives in one SQLite file. Recall is a hybrid search — BM25, embeddings, recency, importance, reinforcement and pinning — combined through a softmax "attention" layer, diversified with MMR, and cut to an explicit token budget. Claude Code and Codex connect over MCP and read and write the same database.

conversation ──► memory_remember ──► SQLite (chunks · FTS5 · vectors · version chain)
                                            │
new session ◄── memory_briefing ◄── attention-weighted recall ◄── memory_recall

Why

Coding agents start every session cold. CLAUDE.md and AGENTS.md carry static project conventions, but not the decisions, constraints and half-finished task state that accumulate while working. Context compaction then drops even that. The result is the same questions, the same rediscovered gotchas, and the same design debates every few hours.

SuperMemory closes that loop:

• write a durable fact the moment you learn it with memory_remember;
• get a budgeted hand-off document back at the next SessionStart;
• retrieve specifics on demand with memory_recall, without loading the whole store into the context window.

Features

• Local-first. One SQLite file under ~/.supermemory/. No server, account, API key or network call in the default configuration.
• Two agents, one brain. The MCP stdio server is shared by Claude Code and Codex; any other MCP client can use it too.
• Retrieval that behaves. Exact dedupe, near-duplicate hints, immutable version chains, soft delete, namespace isolation and a relevance gate.
• Context-aware ranking. Six features, a softmax attention distribution and MMR diversity selection. Every result is explainable and tunable.
• Measurable retrieval. A labelled benchmark scores Recall@k, MRR and nDCG and gates CI; --explain shows exactly why a result won.
• Chinese-first tokenization. SQLite's unicode61 tokenizer cannot search CJK text. SuperMemory indexes CJK unigrams and bigrams itself and splits camelCase / snake_case identifiers.
• Fail-open integration. Hooks never block a session: any error goes to stderr and the agent keeps working.

Requirements

Check everything at once:

python -m supermemory doctor

Quick start

git clone https://github.com/Yuziki1227/supermemory.git
cd supermemory
python install.py            # install, register MCP servers and hooks, self-check

Then restart Claude Code / Codex. The first session will not show a briefing because the database is empty; memories appear in the briefing once the agent has written some.

Other installer modes:

python install.py --dry-run   # show every file that would change
python install.py --dev       # use this checkout in place (no copy)
python install.py --uninstall # remove config entries, keep the database
python install.py --purge     # uninstall and delete ~/.supermemory

What the installer changes

Every modified file is backed up to ~/.supermemory/backups/. All merges are idempotent: running the installer twice does not duplicate entries.

Manual MCP registration

If you prefer not to run the installer, register the server yourself.

Claude Code (~/.claude.json, or a project-scoped .mcp.json):

{
  "mcpServers": {
    "supermemory": {
      "type": "stdio",
      "command": "python",
      "args": ["C:/Users/you/.supermemory/app/run.py", "mcp"],
      "env": { "PYTHONUTF8": "1" }
    }
  }
}

Codex (~/.codex/config.toml):

[mcp_servers.supermemory]
command = "python"
args = ["C:/Users/you/.supermemory/app/run.py", "mcp"]
env = { PYTHONUTF8 = "1" }
startup_timeout_sec = 30

Windows: point command at python.exe and keep the .cmd launcher only for human CLI use. The installer writes the paths for you.

pip install . also works for the CLI (supermemory ...); MCP registration is still manual. In that case use python -m supermemory mcp as the server command.

How it works

1. Storage: chunks, FTS index, version chain

Long input is chunked before it reaches SQL, in this order of preference: blank line → line → sentence → hard split. Defaults are 260 tokens per chunk with 48 tokens of overlap, so every row is a self-contained, retrievable atom.

Each row keeps title, body, summary, tags, source, importance, confidence, timestamps, embedding, content hash and version pointers.

Updates never overwrite. memory_update inserts a new row and marks the old one superseded_by with valid_to set. Normal recall only sees current rows; include_superseded=true shows the full history. "This changed" and "nobody maintains this" never get mixed up.

Deduplication has two levels:

• exact repeats (same namespace + kind + content hash) return the existing id instead of writing a new row;
• near-duplicates (cosine ≥ 0.90) are not merged automatically — the result carries a hint instead. "Use X" and "do not use X" can be vector neighbours; silently merging them is the fastest way to corrupt a memory store.

2. Retrieval: an attention distribution

Every candidate is described by six features:

The weighted sum is score. Then:

alpha_i = softmax(score_i / τ)      τ = 0.35

alpha is an attention distribution: a fixed budget of probability mass is split across candidates according to relevance, with τ controlling how sharp the split is. This is not a neural attention head — local inference is not the point — but the same mathematical shape, fully explainable and free to compute.

Selection uses MMR (maximal marginal relevance): each pick discounts candidates that are too similar to what was already selected, so the top-k is not five rephrasings of one memory. The result is then cut to a token budget.

A hard gate sits in front of all of this: a candidate must either match the lexical index or have cosine ≥ 0.15. Without it, "recent + important" alone drags unrelated memories into the top-k — the simplest source of recall poisoning.

3. Why CJK needs its own tokenizer

SQLite's unicode61 treats a whole Chinese sentence as one token, so MATCH "检索" never hits. SuperMemory uses one tokenizer for both indexing and querying: CJK runs are indexed as unigrams plus adjacent bigrams (检索层 → 检 索 层 检索 索层), and Latin identifiers are split on camelCase and snake_case (MemoryStore → memorystore memory store). Bigrams preserve phrases; BM25's IDF automatically down-weights single characters.

4. Embeddings

The default is an offline hashed embedding: words, identifiers and CJK n-grams are feature-hashed with blake2b into 1024 dimensions, sublinearly weighted, and L2-normalised. Nothing to download, no API key, deterministic across processes — it works immediately after install.

It correlates with BM25 but adds character-level similarity (typos, morphology, mixed Chinese/English). For true semantic vectors, switch to any OpenAI-compatible /v1/embeddings endpoint:

export SUPERMEMORY_EMBED_API_KEY=sk-...
python -m supermemory config --set embedder.provider=openai
python -m supermemory config --set embedder.model=text-embedding-3-small
python -m supermemory reembed

If the remote endpoint is unreachable, the process falls back to local embeddings with a warning on stderr. A memory service should not refuse to start because a key expired.

5. Namespaces

By default the namespace is the current project's directory name (SuperMemory/ → supermemory). Recall searches that namespace plus global; cross-project knowledge should be written with namespace="global". namespace="*" searches everything.

Resolution order for the project directory:

1. SUPERMEMORY_NAMESPACE (explicit namespace, * = all)
2. SUPERMEMORY_PROJECT_CWD (set by the hooks from the hook payload)
3. CLAUDE_PROJECT_DIR (injected by Claude Code)
4. the process working directory

6. Session hand-off

The SessionStart hook injects a briefing: pinned rules, open tasks, key decisions, preferences and gotchas, plus recent updates. It fires for every session source, including source=compact — the only supported moment to put text back in front of the model after a context window collapses.

The PreCompact hook cannot inject context (Claude Code rejects its hookSpecificOutput), so it only prints a plain-text reminder to stdout. Persisting state is still the agent's job; the SessionStart hook is the guaranteed recovery path.

The SessionEnd hook runs housekeeping: purge expired soft-deletes, trim the event log.

Retrieval alone is not enough — memory has to be pushed back at the right moment.

MCP tools

Tool descriptions encode when to use them, so the model does not need extra prompting. initialize also returns server instructions covering when to write, when to read and how to revise.

CLI

python -m supermemory ... and supermemory ... are equivalent; the latter requires ~/.supermemory/bin on PATH (the installer creates it).

# setup and health
supermemory doctor
supermemory init
supermemory stats

# write and read
supermemory remember "Content" -t "Title" -k decision --tag architecture -i 0.9
supermemory recall "how does retrieval work"
supermemory recall "how does retrieval work" --explain
supermemory get M12
supermemory list --kind decision
supermemory briefing --query "implement MCP"
supermemory bench bench/cases.jsonl       # retrieval regression suite

# revise and maintain
supermemory update M12 --content "Corrected content"
supermemory forget M12                     # soft delete
supermemory link M12 M7 --relation refines
supermemory maintenance

# portability and configuration
supermemory export backup.json
supermemory import backup.json
supermemory reembed --provider openai
supermemory config
supermemory config --set ranking.temperature=0.5

# integration
supermemory mcp                            # run the stdio MCP server
supermemory hook session-start             # hook entry point (reads JSON on stdin)

Retrieval quality

Two tools keep ranking changes honest.

Why did this result win?

supermemory recall "query" --explain appends the feature breakdown to each hit:

M3 [decision 0.63] retrieval layer
    Retrieval uses BM25 and vector scores.
    ↳ lex 1.00 · vec 0.42 · imp 0.46 · rec 1.00 · rein 0.00 · pin 0 · score 0.626 · attn 1.0000

The same switch exists over MCP: memory_recall with "explain": true.

Benchmark

bench/cases.jsonl is a self-contained corpus plus labelled queries. Each run seeds a temporary database and reports Recall@k, MRR, nDCG@k, negative-case behaviour and token cost:

python -m supermemory bench bench/cases.jsonl
python -m supermemory bench bench/cases.jsonl --json
python -m supermemory bench bench/cases.jsonl --fail-under-recall 0.85 --fail-under-mrr 0.75

CI runs the suite, so a ranking change that breaks retrieval fails the build. Add a case for every retrieval bug you fix; see bench/README.md for the file format. Benchmarks use the built-in defaults — pass --config ~/.supermemory/config.json to measure a tuned configuration.

Configuration

~/.supermemory/config.json is optional; unset values use defaults. The most useful knobs:

{
  "ranking": {
    "weights": { "lexical": 0.36, "vector": 0.28, "importance": 0.13,
                 "recency": 0.09, "reinforcement": 0.06, "pinned": 0.08 },
    "temperature": 0.35,
    "half_life_days": 45,
    "mmr_lambda": 0.22
  },
  "recall": { "default_limit": 8, "max_tokens": 1400, "vector_floor": 0.15 },
  "briefing": { "max_tokens": 900 },
  "store": { "chunk_max_tokens": 260, "namespace_mode": "cwd" }
}

• temperature lower → attention concentrates on the top hit; higher → flatter distribution.
• mmr_lambda higher → stronger diversity penalty; 0 disables MMR.
• half_life_days lower → recent memories dominate.
• namespace_mode: cwd (default), fixed (use fixed_namespace), none (everything goes to global).

Environment variables:

Repository layout

supermemory/
  config.py        defaults, paths, namespace resolution
  text.py          tokenizer, token estimate, summaries, chunking
  embed.py         local hashed embeddings / OpenAI-compatible embeddings
  db.py            schema, FTS5, connection setup
  ranking.py       attention scoring + MMR
  core.py          MemoryStore: writes, recall, versions, briefing, export
  render.py        token-aware output rendering
  mcp_server.py    dependency-free MCP stdio server
  cli.py           command line interface
  hooks.py         Claude Code hook entry points
  bench.py         labelled retrieval benchmark and metrics
install.py         installer
run.py             single launcher for MCP / hooks / CLI
bench/             benchmark cases and format documentation
tests/             32 tests, including a real MCP subprocess handshake

Design decisions

• No automatic summaries. A summary is the first sentence, not an LLM rewrite. That saves an inference call and prevents summary drift — memory saying one thing and its summary another. When semantic compression is needed, the agent writes its own kind=decision memory.
• No automatic merging. See above: negation semantics get eaten.
• No automatic decay deletion. Recency only affects ranking; it is never written back. Memories do not disappear because you have not used them for a while.
• Hooks fail open. Every exception is printed to stderr and returns 0. A broken memory system must not break normal coding.
• Hand-rolled MCP instead of an SDK. The server depends only on the CPython standard library plus numpy, so it runs on any machine with Python.
• Bilingual docs, one codebase. User-facing CLI text is Chinese today; every feature is documented in both README languages.

FAQ

No briefing after restart? The database is empty (nothing was written yet), or the hooks did not take effect. Check python -m supermemory stats; test the hook manually:

echo '{"source":"startup","cwd":"/path/to/project"}' | python -m supermemory hook session-start

The namespace looks wrong. Namespaces are derived from the project directory name. A stdio MCP process does not necessarily start in the project root, so SuperMemory prefers SUPERMEMORY_PROJECT_CWD → CLAUDE_PROJECT_DIR → the process cwd. If it is still wrong, set SUPERMEMORY_NAMESPACE explicitly or add it to the MCP server's env.

Chinese recall is weak. Run doctor and confirm FTS5 ✓. Then check for one-character queries — the query tokenizer drops single CJK characters when bigrams are available.

Can two agents write at the same time? Yes. The database uses WAL, a 30-second busy timeout, and BEGIN IMMEDIATE for writes. The MCP server, hooks and CLI all share one file safely.

How do I move to another machine? Copy ~/.supermemory/memory.db together with its -wal file, or use supermemory export on the old machine and supermemory import on the new one.

The database is getting large. supermemory maintenance purges soft-deleted rows past their retention window and compacts the FTS index.

Is any data sent over the network? No, not in the default configuration. If you enable the OpenAI-compatible embedder, memory text is sent to the endpoint you configure.

Development

python -m unittest discover -s tests -v
python -m supermemory bench bench/cases.jsonl
python install.py --dev       # run the checkout in place

No third-party test framework, no build step. Tests isolate themselves with a temporary SUPERMEMORY_HOME and never touch your real database. CI runs the unit suite and the retrieval benchmark on Linux (Python 3.10–3.14) and Windows.

See AGENTS.md for the architecture, invariants and contribution workflow, and CONTRIBUTING.md for the short version.

License

MIT © 2026 Yuziki1227