Knowledge Graph Memory for Agents: Document Registry and Hybrid Search

🌐 한국어

The most common way to give an agent long-term memory is write it to a file and read it back later. It works fine at first—right up until you have thirty files. After that, "which file did I write that in?" becomes the new problem.

Adding embedding search solves about half of it. But embeddings understand meaning, not relationships. Ask "what event led to this decision" and you get semantically similar sentences back, not the chain of causation. This article is about turning a pile of files into a graph.

1. Document Registry — Store Locations, Not Content

The first decision is whether the body text goes into the database. I'd argue it shouldn't. Keep files on the filesystem and put only metadata in the database.

CREATE TABLE documents (

    id            TEXT PRIMARY KEY,

    tenant_id     TEXT NOT NULL,     -- multi-tenant isolation

    agent_id      TEXT NOT NULL,     -- per-agent namespace

    scope         TEXT NOT NULL,     -- personal / team / shared

    path          TEXT NOT NULL,     -- workspace-relative path

    title         TEXT,

    doc_type      TEXT,              -- note / memory / context / skill

    content_hash  TEXT,              -- SHA-256, for change detection

    embedding     BLOB,              -- vector for semantic search

    updated_at    TIMESTAMP,

    UNIQUE (agent_id, scope, path)

);

You gain a lot from this. A human can edit files in an editor, git handles versioning, and if the database is lost the originals survive. The database is only an index—rebuild it.

What you need in exchange is synchronization, and content_hash does that job. A file watcher detects changes, recomputes the hash, and re-embeds only when it differs. Embedding calls cost money, so this one comparison saves a great deal.

func (w *SyncWorker) onFileChanged(path string) error {

    content, err := os.ReadFile(path)

    if err != nil {

        return err

    }

    sum := sha256.Sum256(content)

    hash := hex.EncodeToString(sum[:])

    doc, _ := w.store.GetByPath(path)

    if doc != nil && doc.ContentHash == hash {

        return nil    // identical content → skip re-embedding

    }

    return w.reindex(path, content, hash)

}

2. Wikilinks — Write Relationships Into the Body

Make people manage relationships through a separate UI and nobody uses it. That holds for agents too. So make it syntax you write inline.

See [[incidents/2026-03-payment]] for the March payment outage.

That's when we changed the [[decisions/retry-policy|retry policy]].

The form is [[target]] or [[target|display text]]. A parser scrapes the pattern and stores it in a links table.

CREATE TABLE links (

    from_doc_id TEXT NOT NULL,

    to_doc_id   TEXT NOT NULL,

    link_type   TEXT NOT NULL,   -- wikilink / reference

    context     TEXT,            -- ~50 chars of surrounding text

    UNIQUE (from_doc_id, to_doc_id, link_type)

);

CREATE INDEX idx_links_from ON links(from_doc_id);

CREATE INDEX idx_links_to   ON links(to_doc_id);   -- for backlinks

Two things matter here.

• The context column — storing ~50 characters around the link lets you later see why two documents are connected without opening either. When traversing the graph, having that one line changes how usable the result feels
• The to_doc_id index — this gives you backlinks for free. "Documents referencing this one" becomes a single index lookup. No need to store bidirectional links separately

Parsing details worth handling: skip empty [[]], trim surrounding whitespace, append .md when the extension is missing, and treat a link to a not-yet-existing document as normal rather than an error. Pointing at a document you intend to write is a natural usage pattern.

3. Hybrid Search — Mixing Keywords and Meaning

Neither alone is enough.

• Keyword search (FTS) — strong on exact nouns, error codes, people's names. Embeddings are useless for finding ORA-01555
• Embedding search — strong on differently-worded queries like "that time payments got slow"

You combine them by normalizing scores. The two scoring systems are completely different (BM25 is unbounded, cosine similarity is 0–1), so scale each to 0–1, then multiply by a weight and sum.

func hybridSearch(q string, kw, vecWeight float64) []Hit {

    ftsHits := ftsSearch(q)           // BM25 scores

    vecHits := vectorSearch(embed(q)) // cosine similarity

    scores := map[string]float64{}

    for _, h := range normalize(ftsHits) {   // divide by max → 0..1

        scores[h.ID] += h.Score * kw

    }

    for _, h := range normalize(vecHits) {

        scores[h.ID] += h.Score * vecWeight

    }

    return topN(scores, 10)

}

Memory doesn't live in one place. When documents, session summaries, and knowledge graph entities sit in different stores, fan out in parallel and merge.

query ─┬─→ document vault search   [weight 0.4]

       ├─→ session summary search  [weight 0.3]

       └─→ knowledge graph search  [weight 0.3]

              ↓

       normalize each source to 0..1 → multiply by weight

              ↓

       merge and dedupe by ID → sort by score DESC → top N

The weights aren't absolute. If session summaries are unusually useful in your domain, raise that one. What you must preserve is the order: normalize first, weight second. Flip it and whichever source has the largest raw scale always wins.

4. Multi-Hop Traversal — Following the Relationships

This is the real reason you built a graph. Ask "what's related to this decision" and search finds only directly-mentioned documents. Context two or three hops away never arrives. A recursive query walks it.

WITH RECURSIVE walk(doc_id, depth) AS (

    SELECT :start_id, 0

    UNION

    SELECT l.to_doc_id, w.depth + 1

    FROM links l

    JOIN walk w ON l.from_doc_id = w.doc_id

    WHERE w.depth < 2              -- 2 hops maximum

)

SELECT d.*, w.depth

FROM walk w JOIN documents d ON d.id = w.doc_id

WHERE w.depth > 0

ORDER BY w.depth;

Two practical cautions.

• Cut depth off at 2 or 3. In a well-connected graph, four hops reaches nearly every document. The moment "related" becomes "everything," search stops meaning anything
• Decay the score by distance. Weight hop 1 at 1.0 and hop 2 at 0.5 and directly-related documents naturally float to the top

Using UNION rather than UNION ALL is deliberate too: already-visited nodes don't duplicate, so cyclic links can't spin you into an infinite loop.

5. Injecting Into Context — Don't Dump It All

Good search doesn't mean you paste all the results into the prompt; that puts you back where you started. Hand it over in stages.

• L0 — title and a one-line abstract. Always injected. Tens of tokens
• L1 — paragraph-level excerpts. Only the few highest-relevance hits
• L2 — full text. Only when the agent explicitly asks via read_file

The agent always knows what exists while expanding only what it needs. Same as a person skimming a table of contents and opening only the chapter that matters.

Summary

• Body text on the filesystem, metadata and index only in the database — surviving originals make recovery easy
• Compare content_hash to re-embed only changed documents — that's where most of the cost savings live
• Express relationships as [[wikilinks]] inline — a separate UI goes unused
• One to_doc_id index gives backlinks for free; a context column preserves why the link exists
• Search combines keyword + embeddings, each normalized to 0–1 before weighting — normalization comes first
• Multi-hop means recursive query + depth cap of 2–3 + per-hop score decay
• Inject context in L0/L1/L2 stages — dumping the full result set undoes the whole exercise

The hard part of agent memory isn't storage—it's retrieval. Design how you'll find something again three weeks from now before you design what to write down. Seen that way, the graph isn't a fancy feature; it's the cheapest retrieval path available.