Most attempts to give an AI a memory end up in one of two places. Either you embed everything into a vector database and hope cosine similarity hands back the right chunk, or you keep a growing file of notes that you paste into the context window at the start of every session. Both work for a while. Both rot.
The vector database rots quietly. You cannot read it, you cannot audit it, and when it returns the wrong passage you have no way to see why. It is a black box that occasionally lies with confidence. The notes file rots loudly. It grows until it no longer fits, so you trim it, and the trimming is guesswork because you cannot tell which line was load-bearing. Worse, the agent reading it has no idea which parts are still true. It treats a stale preference from three months ago with the same weight as the decision you made this morning.
I have been building an alternative and running it as my own daily system for a couple of months. The idea is simple to state. Treat the knowledge base like a wiki that an agent curates, give it the same discipline a good wiki has, and make recall something you can read and check rather than something you have to trust. I call it the agentic knowledge engine, and the rest of this piece is what it is, how it works and what it actually solves.
The shape of the thing
At the centre is a plain folder of markdown files. Raw material goes into one zone. The agent digests it into a second zone, a wiki of small cited articles grouped into bundles and topics. Nothing is embedded into an opaque index as the primary store. The articles are the store, and you can open any of them and read it.
That choice sounds almost too modest to matter, but it is the whole game. Because the knowledge lives as readable articles with citations, every other property I care about follows from it. Retrieval can be a walk over indexes instead of a similarity search. An audit can count broken links and uncited claims instead of measuring a loss you cannot interpret. And when the agent answers a question, it can point at the exact article each fact came from, the same way a good analyst shows their working.
There are two files at the root that govern everything, and the split between them is deliberate. One is a frozen harness. It holds the contracts that never change in the middle of a run: what an article must contain, how a citation is written, what counts as a structural defect, the handful of hard rules the agent may never break. The agent reads it once and treats it as fixed. The other is an iterable program. It holds the behaviour: the verbs the agent runs, the budgets it works inside, the routing heuristics it uses to decide where a source belongs. That file is meant to be edited over time to make the agent work better.
The reason to separate them is that if the rules and the behaviour live in the same place, every failure can be fixed by quietly redefining success. Freeze the contracts and that move becomes impossible. An improvement to the behaviour is now measurable against a yardstick that means the same thing run after run. This is borrowed from the autoresearch pattern, where a stable evaluation harness sits underneath an experimental loop, and it is the single design decision I would keep if I had to throw away all the others.
Storing, injecting and recalling
It helps to see the engine as three flows rather than one architecture. How knowledge gets stored, what gets injected into context at the start of a session, and how old knowledge is recalled when a task needs it.
Storing is the job of a verb called consolidate. It scans the input zone for material it has not seen, works out which bundle each source belongs in by comparing its content against the scope of each bundle, picks or creates a topic inside that bundle, and writes a cited article. Every factual claim in the article carries a citation back to its source. If a source matches no bundle, it is not forced into the nearest one. It goes to a quarantine folder and waits for a human to decide, because a misfiled article corrupts retrieval silently and a quarantined one is loud and gets fixed.
Injecting is what happens at the start of every session, and the trick is to keep it cheap and flat. The frozen harness and the program are read once. Two thin routers load, one listing the bundles and one listing the kinds of experience the agent has recorded. And a small snapshot loads, no more than fifteen hundred tokens, holding the agent’s identity, its distilled lessons and a handful of recent one-line episode summaries. That is the entire fixed cost of starting up, and it does not grow as the knowledge base grows. A wiki ten times the size injects the same amount.
Recalling is the part most systems get wrong by making it uniform. Here it is tiered, and the cheap layer always answers first.
Why recall is layered
When a question arrives, the first thing the agent checks is the snapshot already sitting in its context. For a large class of questions, the ones about recent decisions or stable preferences, that snapshot is enough and the agent answers without reading a single file. This is tier zero and it costs nothing.
If the snapshot does not settle it, the agent walks the indexes. It starts at the top-level bundle list, picks the relevant bundle, reads that bundle’s index to pick a topic, reads the topic’s index to pick the specific articles, and only then opens the article bodies. The indexes are deliberately free of body text, so the routing is light and the expensive reads happen last and only where they earn their place. This is tier one, the primary path, and it carries the citation discipline. Every fact in the answer names the article it came from.
Only when the walk misses, or when the phrasing of a question is unlikely to match the one-line descriptions in the indexes, does the agent reach for tier two. That is a local hybrid search, keyword matching and embeddings merged together, run over the same articles. And here is the part that keeps the whole thing honest. The search returns pointers, never answers. It tells the agent which articles look relevant, and the agent then opens those articles, reads them and cites them exactly as it would have on the walk. The embedding index is a router of last resort, not a source. No claim ever enters an answer through the side door of a similarity score.
If all three tiers come up empty, the agent says so. It does not invent. An honest gap is a feature, because it gives you a precise list of what the knowledge base is missing rather than a confident sentence you have to fact-check.
The point of layering it this way is economy with a clear conscience. The free layer handles the common case. The curated walk handles most of the rest and stays fully auditable. The fuzzy fallback catches what word-matching would miss, without ever being trusted enough to bypass the citation rule.
The verbs, and the loop they run in
The agent does everything through a small set of verbs. Query reads. Consolidate writes from raw sources. Refine audits without changing anything. Evaluate measures retrieval against a fixed set of questions. Reflect maintains the agent’s record of its own experience. Five verbs, no more, and that constraint is itself a design choice. Every action the agent can take is one of these, which makes the system’s behaviour small enough to reason about.
What turns this from a filing system into something that improves is that the write path is a loop, not a single step.
Before the agent consolidates anything it recalls. It pulls a few of its most relevant past episodes and its distilled lessons into context, so that a routing decision it agonised over last week is not relitigated from scratch this week. Then it consolidates. Then, automatically and in the same chain, it audits its own work. The audit reduces the structural health of the wiki to a single number, drift, counting orphaned articles, broken links, index mismatches, uncited claims and a few other defects. The number is compared against the previous run, so any change that degrades the wiki is caught immediately and labelled a regression rather than slipping through.
After the audit the agent reflects. It reads the experience it just had, merges any recurring lesson into its distilled notes, and regenerates the snapshot that the next session will load. Then it rebuilds the search index so the fuzzy fallback never goes stale, and commits everything to version control, which doubles as a second audit trail you can read with ordinary tools.
The thing I want to draw attention to is where the learning goes. It lands in the agent’s episodes and its distilled reflections. It never lands in the frozen contracts. The engine sharpens how it works without ever being able to rewrite the definition of what correct means. That boundary is what lets the system get better without drifting, because the measuring stick is nailed down while everything around it is free to move.
Two memories, kept apart
A detail that matters more than it looks: the engine keeps two kinds of memory and refuses to mix them.
One is semantic. It is the wiki of facts. A statement like a central bank raised rates to a particular level is a fact, and it lives in an article with a citation. The other is episodic. It is the record of what the agent did and what happened. A lesson like multi-item newsletters should become one summary article rather than ten stubs is not a fact about the world, it is a fact about how to work, and it lives as experience.
Mixing them would poison retrieval in both directions. The wiki would fill with operational asides and the experience log would fill with trivia. Keeping them apart means each can be retrieved on its own terms. When a task is generative, the agent leans on its experience for tone and judgement. When a task needs facts, it walks the wiki.
Episodic memory, in detail
The episodic side is the part I find most interesting, because it is where the system stops being a clever filing cabinet and starts behaving like something that learns. It deserves a closer look.
An episode is a small time-stamped record of a single experience. It has a shape: the situation the agent was in, the actions it took, the outcome, and the insight worth keeping. There are three kinds. Operational episodes are the agent’s own verb runs, where it records how it routed a tricky source, what it quarantined, whether the audit improved or regressed. Life episodes are time-stamped daily entries, a log of what actually happened on a given day, which makes the past recallable as context rather than lost. Signal episodes capture decisions, preferences and reactions the moment they are expressed, so a stated opinion is not forgotten by the next session.
The lifecycle is the same four beats every run. The agent recalls before it acts, pulling at most a few of the most relevant past episodes plus its distilled lessons into context. It acts, now guided by what happened last time a similar input came through. It captures a new episode after acting. And periodically it reflects, merging recurring insights into a single distilled set of lessons and regenerating the snapshot that the next session loads.
The mechanism that keeps this from collapsing under its own weight is distillation. An ever-growing pile of episodes would make recall useless, too many candidates and too much stale detail. So once a lesson has been captured, is supported by two or more episodes and is old enough to have proven itself, the episodes behind it are stamped as distilled and leave the active recall surface. They stay on disk for audit, but they no longer compete for the agent’s attention. The newest episode confirming each lesson is kept live as a worked example. The effect is a memory that compresses as it ages: fresh experience is recalled in full, repeated experience becomes a one-line rule, and the cost of recalling never grows with the size of the store. The compression chain runs experience into distilled lessons into a snapshot, so what the agent carries into each new session is small, current and the product of everything it has learned so far.
This is what closes the loop the discourse around AI memory usually leaves open. An agent with a large model and no episodic memory is brilliant and amnesiac. It solves the same routing problem from scratch every time, makes the same misjudgement twice, and has no way to notice that it is repeating itself. Give it a place to record what happened and a discipline for distilling it, and the second time it meets a problem it has seen before, it remembers the answer.
How this sits next to the self-curated memory file
It is worth comparing this to the approach that has become popular lately, the one Andrej Karpathy has talked about: treat an LLM’s memory as a plain text file that the model itself maintains, adding, replacing and removing facts as it goes, and read back into context when needed. It is a good idea and the engine agrees with its core instinct. Memory should be readable text the model curates, not an opaque embedding store. A markdown file you can open and edit beats a vector blob you have to trust, for exactly the auditability reasons this whole system is built around.
Where the engine goes further is in two places. The first is the split between semantic and episodic. A single self-curated memory file mixes facts about the world with lessons about how to work, and over time that mixture gets harder to keep clean, because the two have different lifespans and different retrieval needs. The engine separates them on purpose: the wiki for facts, the episodes for experience, each retrieved on its own terms. The second is the time-stamped, event-based nature of episodic recall. A memory file tends to flatten everything into a present-tense list of what is true now. Episodes keep the arrow of time. They record that something happened on a particular day, in a particular situation, with a particular outcome, which is what lets the agent reason about what worked rather than only about what is. The snapshot then plays the role Karpathy’s memory file plays, a small always-loaded digest, but it is generated from the episodes and lessons beneath it rather than being the primary store, so it stays bounded and never has to be hand-trimmed.
Put simply, the self-curated file is the right instinct applied to one layer. The engine applies the same instinct across two layers and adds the missing dimension of time, so the agent can hold not just what it knows but what it has been through.
Letting other tools read it
Everything so far makes the knowledge readable to me and to my own agent. It does nothing to make it readable to anyone else. The articles follow my conventions, the cross-references use my link syntax, and a different tool or another team’s agent would have to be taught all of it before it could make sense of a single bundle. The usual fix is to write an exporter, a step that walks the wiki and emits a clean copy in some portable shape. I tried that and then threw it away, because an export step is a copy you have to remember to run and it drifts from the source the moment you forget.
The better answer turned out to be an open format. There is an emerging vendor-neutral specification for exactly this, a way of describing a folder of markdown articles plus frontmatter as a portable knowledge bundle, with a couple of reserved filenames and a machine-readable graph of cross-references. The important thing about aligning to it was that it was an alignment, not a rebuild. Each article gained a small frontmatter block naming its type and a list of the other articles it relates to. Each bundle gained a derived changelog. Nothing about the shape of the wiki changed. The standard sits on top of what was already there.
The payoff is that the export step disappears. Because each bundle now carries everything the format asks for, a bundle is already a conformant bundle. Its mirror on a public repository is a valid portable bundle with no translation in between, which means the unit I expose to the outside world is a thing that already exists and stays fresh on its own, rather than a copy I have to regenerate and keep in sync.
It also resolves a tension I had been living with. Inside the wiki the links stay strictly local, a same-bundle-only rule that keeps each bundle self-contained and keeps the agent from wandering across the whole store while answering one question. But a portable graph wants to express relationships that genuinely cross those boundaries. The format lets both be true at once. The local wikilinks stay inside a bundle for fast navigation, and a separate machine-readable list of relations is allowed to cross bundles, so an external reader gets the full graph without the internal discipline being weakened.
And because this is the engine, conformance went into the loop rather than into a checklist. The same audit that counts structural defects now also checks that every article carries its type and that every cross-reference resolves, and reports a single conformant-or-not verdict. Portability cannot quietly rot any more than the structure can, because the thing that would catch the rot runs on every write. The principle underneath is one I keep relearning: an external standard is a portability layer, not a reason to start over, and when a portability layer adds manual friction the right move is to push the conformance into the live artifact instead of into a copy beside it.
What it actually solves
Strip away the structure and the engine is an answer to five specific failures.
It solves the black box. Retrieval is a walk you can read, and every answer cites the article behind each claim, so when it is wrong you can see exactly where and fix the article rather than retraining an index.
It solves context bloat. The startup cost is flat by construction. A capped snapshot, body-free routers and a recall budget that does not grow with the store mean a session opens with the same modest cost whether the wiki holds fifty articles or five thousand.
It solves silent rot. The audit runs on every write and reduces health to one comparable number, so a regression is caught the next time the agent touches the wiki rather than discovered months later when an answer comes back wrong.
It solves the agent that forgets. Because experience is stored, recalled and distilled rather than thrown away at the end of each run, the system curates and retrieves a little better each time instead of starting from zero. That is the autoresearch idea turned on the agent’s own behaviour, and it is the part I find genuinely worth the effort.
And it solves lock-in. Because the articles conform to an open format and each bundle is already a portable bundle, the knowledge is not trapped inside my conventions or my tools. Any agent that understands the format can read a slice of it, and it stays that way on its own rather than through an export I have to remember to run.
None of this is exotic. It is a folder of markdown, a frozen rulebook, an editable playbook and a handful of verbs that run in a disciplined loop. The wager is that discipline beats cleverness here, that a knowledge base you can read, check and hand to another tool will outlast one you have to trust, and that an agent which remembers what it learned is worth more than one with a larger model and no memory of yesterday. So far the wager is holding.