A Journey into Agentic AI — Part 3: Memory

Every conversation you have with an AI agent begins in perfect silence. There are no half-remembered names, no accumulated resentments, no shorthand that builds up between people who have worked together for years. The model you spoke with yesterday has no idea you existed. This is not a bug so much as a design constraint baked into how large language models work: the context window is finite, the weights are frozen between training runs, and each new session is, in a very literal sense, the first time you have ever met.

For casual use, this is fine. For an agent you expect to manage a software project across weeks and months, it is a serious problem. How do you give a stateless system a sense of continuity?

The answer, it turns out, is older than computers.

The Bible as Compression

Consider a strange fact: the Bible, as a plain text file, is roughly five megabytes. That is smaller than many smartphone app icons. Five megabytes to contain thousands of years of genealogies, legal codes, poetry, prophecy, letters, apocalypse — the lives and deaths of an entire civilization, compressed into something a dial-up modem could transfer in under a minute.

When I first encountered this figure, my instinct was to treat it as a compression problem. Surely we could do better. Summarize more aggressively. Collapse redundancy. Fit it into a single page of bullet points.

But that instinct is exactly wrong. The compression is already the miracle. What the text preserves is not raw events but interpretable structure — narrative, genealogy, law, and prophecy interlocked in ways that let readers expand any fragment back into meaning. A homily takes two verses and unpacks them into an hour of lived relevance. A daily devotional distills a chapter into a single sentence that a person can carry through their day. A Bible story for children takes the Exodus and makes a five-year-old feel the weight of freedom and obligation. The source text is dense not because it is trying to be efficient but because it is trying to be generative: a small, stable thing that produces an enormous amount of meaning when read carefully in context.

This is a profound model for how agents can handle memory. You do not need the agent to remember everything. You need a source text small enough to fit in a context window, structured well enough that the agent can expand the relevant parts on demand, and stable enough that it does not need to be rewritten every session. The agent’s job is to do what the homilist does: read the source, understand the current moment, and draw the connection.

Using a source text and then asking the AI to expand on it and relate it to the current task turns out to be a surprisingly general solution to the finite-context problem. The transfer is not automatic — you have to write the source text well, and the agent has to be prompted to treat it as authoritative — but the pattern is sound. Small, well-structured files that anchor the agent’s understanding of what matters, why, and what it should do next.

Kanerva’s Sparse Distributed Memory

Before we get too practical, it is worth understanding why memory is hard for language models in the first place, and why vector-based retrieval works at all. The theoretical underpinning goes back to a 1988 book by Pentti Kanerva called Sparse Distributed Memory.

Kanerva was trying to model how biological brains store and retrieve information, and he observed something counterintuitive: a memory system does not need to store every piece of data at a distinct address. Instead, you can distribute information across a vast space of possible addresses, where each memory is spread across many locations and each location participates in many memories. Retrieval is not a lookup — it is a resonance. You present a partial or noisy version of a pattern, and the system returns the stored pattern that most closely matches it, even if the query is incomplete or slightly wrong.

The key insight is that the retrieval key and the stored content live in the same high-dimensional space. If you think of a remembered conversation as a point in that space, then a similar conversation — same topic, same emotional register, similar vocabulary — will be a point nearby. Ask a question that sounds like the earlier conversation, and the system will pull the earlier conversation back up.

Modern vector-based retrieval — which underlies most practical implementations of RAG (Retrieval-Augmented Generation) — is almost exactly this. Text is embedded into a high-dimensional vector space using a trained model. Semantically similar passages end up geometrically close. When an agent needs to recall something, it embeds the current query, finds the nearest stored vectors, and retrieves the corresponding chunks. The match is approximate and probabilistic, not exact — much like Kanerva’s distributed addresses.

There is a deeper implication here that is easy to miss. If the ending state of an LLM context — the accumulated attention patterns after a long, productive session — could somehow be used as a retrieval key, then the agent’s own mental state at the end of a task would naturally surface related past work. Vector memory gestures toward this: the embeddings of the agent’s current reasoning pull up passages from the past that are semantically related to what the agent is thinking about right now. Memory, in this model, is not a filing cabinet. It is an associative field that activates in the presence of resonant patterns. RAG, stripped to its essentials, is memory based on searching compressed representations of what came before.

A Practical Architecture: The Memory Directory

Understanding the theory is useful; having a concrete structure you can actually implement is better. Here is a layout that works in practice for giving an agent a persistent, navigable memory system:

memory/
  README.md              # entry point, index of everything
  objectives/
    current.md           # active goals with priorities
    backlog.md
  context/
    architecture.md      # stable, slow-changing facts
    decisions/           # ADRs (architecture decision records)
      001-chose-postgres.md
  projects/
    repo-a/
      status.md
      learnings.md
  agents/
    weekly-survey.md     # what the agent does, its prompt

The design principle here is the same as the Bible’s: a small, stable entry point that expands outward through deliberate links. The README.md at the root is not decorative — it is the agent’s map of the territory, and it should explain the structure explicitly. Something like: before reading any file in a subdirectory, read that directory’s own README first. This lets the hierarchy self-document. An agent that arrives in memory/projects/repo-a/ without context can read the local status.md and learnings.md and understand what matters without having to load the entire tree.

The key to making this work is the instruction file — the CLAUDE.md or AGENTS.md that tools like Claude Code and Codex pick up automatically. These files tell the agent how to navigate your memory structure, when to consult which file, and what to treat as authoritative. If the agent needs to make a statement about the business — brand voice, what language to use, which logo to link to — CLAUDE.md can instruct it to read context/brand.md before doing so. The information does not have to live in the instruction file; it just has to be findable from it.

This is the trail of links model. Each file is a node; each README.md and instruction file is a router. The agent never has to load everything at once. It follows the relevant path, loads what it needs, and the rest stays on disk, available but not consuming context.

The emerging community practice around these files — CLAUDE.md, AGENTS.md, and similar conventions — reflects a growing recognition that the memory problem is fundamentally an information architecture problem. You are not building a database. You are building a library, and the reference desk is your instruction file. A useful survey of these conventions is available on HackerNoon.

Memory Across Sessions: The Shift Handoff

File-based memory becomes especially powerful when you think about what happens when an agent runs for hours, pauses, and then resumes — or when a new session picks up where the last one left off. Anthropic’s engineering research on long-running agents offers a vivid analogy for the core challenge: an agent working across sessions is like engineers working in shifts, where each new engineer arrives with no memory of what happened on the previous shift. The solution in human organizations is the handoff note. The solution for agents is the same thing.

Anthropic’s work on long-running scientific computing agents formalizes this into a pattern. The agent maintains a CHANGELOG.md — or a claude-progress.txt, depending on the project — that serves as its portable long-term memory, something like a lab notebook. A well-maintained progress file records not just what was completed, but what was attempted and failed, and why. This second category is arguably more valuable. An agent that does not know which approaches have already been ruled out will waste sessions rediscovering dead ends. The failed-approaches section is the institutional memory that prevents reinventing the wheel, or redriving down the same dead-end road.

There is a recursive elegance to the most sophisticated version of this pattern: the agent is given a CLAUDE.md at the start of the project with its instructions, but it is also given permission — even encouraged — to edit those instructions as it works, updating them to reflect decisions made, approaches settled on, and context accumulated. The instructions become a living document. The agent’s own compressed understanding of the project accumulates in the file it reads at the start of every session. It is, again, the Bible pattern: a source text that grows slowly, carefully, by accretion, and that any future reader (or any future session of the same agent) can use to reconstruct the state of the project.

This approach is detailed in Anthropic’s research on long-running agents for scientific computing and their companion post on effective harnesses for long-running agents — both worth reading if you are building anything that needs to run longer than a single context window.

RAG as the Dynamic Layer

File-based memory handles structured, intentional knowledge: what the project is, what decisions have been made, what the agent should do. But what about the vast, unstructured residue of past work — every conversation, every document, every scrap of analysis that might be relevant but is too large to include in the context directly?

This is where RAG earns its place. If file-based memory is the library’s reference section — curated, well-indexed, maintained by hand — then RAG is the stacks: an enormous collection that no one could read in full, but that a good search system can surface the relevant corner of when you need it. The agent embeds its current query, searches the vector index, and retrieves the passages that are semantically closest to what it is trying to do right now.

The two systems are complementary rather than competing. You use file-based memory for things you know you will need — the project’s current status, the architectural decisions, the brand voice guidelines. You use RAG for things you might need — the context from an old conversation that turned out to be relevant, the notes from a meeting three months ago, the documentation for a library you haven’t touched since last year. One is curated; the other is harvested. Together they cover much of what an agent needs to function as if it has been paying attention all along.

Memory as Craft

None of this is automatic. The file-based memory structure has to be designed, populated, and maintained. The progress file has to actually be updated at the end of each session. The instruction file has to be written well enough that a stateless agent can read it cold and understand what matters.

This is, in a way, the deeper lesson. Memory for agents is not a technical feature you turn on — it is a practice of information management, not so different from the practices of any organization that has to keep track of what it knows and what it has tried. The analogy to the Bible is not accidental: what makes that text remarkable is not the words alone but the millennia of careful transmission, commentary, and re-application that have kept it generative. A memory system for your agent will be about as good as the care you put into maintaining it.

The good news is that the agent can help. It can update its own progress files, revise its own instruction files, and flag when a document is going stale. The tools are there. The architecture is not complicated. What it requires is the same thing good documentation has always required: someone who understands what the next reader will need to know, and takes the time to write it down.