Give an Autonomous Coding Agent Memory: KNOWLEDGE + RAG

For the first few weeks my daily coding agent had no memory. Each run began with the repository and its instructions, worked for half an hour, opened a pull request, and forgot everything. It was competent and it was Groundhog Day: the same misunderstanding about a fixture, the same wrong assumption about a helper, three times in one week.

The fix turned out to be small — three markdown files, about a hundred and fifty lines of Python, and one instruction at the end of the run. This is how it fits together, and which parts actually earned their place.

Three files, three different jobs

The temptation is one big notes file. That fails because the three kinds of memory have different lifecycles: one is curated by a human, one accumulates automatically, and one is overwritten constantly.

Keeping them separate matters operationally too. Knowledge is committed and reviewed; lessons and sessions are seeded once and then only appended to, so an accidental overwrite does not erase weeks of accumulated detail.

Vector search without a vector database

Once there are three growing files, dumping them all into the prompt stops being viable, and grep is too literal to find the note that matters. The middle ground is a tiny retrieval script: chunk the markdown by heading, embed each chunk, and rank by cosine similarity.

# kb.py — stdlib-only vector search over the agent's own notes.
# No vector database, no framework, about 150 lines.
#
#   kb index KNOWLEDGE.md LEARNINGS.md SESSIONS.md
#   kb search "how do we handle idempotency" 5

def chunk(markdown: str) -> list[str]:
    """Split on level-2 headings, then sub-split anything over ~1200 chars."""

def embed(texts: list[str]) -> list[list[float]]:
    """One HTTP call per chunk to a free embedding endpoint.
    On any failure, fall back to lexical token overlap so a run never
    breaks because an embedding API was down."""

def search(query: str, k: int) -> list[tuple[float, str]]:
    """Cosine similarity against the on-disk index, top k returned."""

# Index resolution walks UP from the working directory, the way git
# finds .git, so a search works from any subdirectory of the repo.

Two decisions made this practical. First, a lexical fallback: if the embedding call fails for any reason, the script falls back to token overlap so a run never breaks because an API was down. Second, index resolution that walks up from the working directory the way git finds its own root, so a search works from anywhere inside the repository.

The loop that makes knowledge compound

Retrieval alone is a lookup table. What turns it into learning is a per-item pipeline plus a reflection step, and the reflection step is the part most setups leave out.

# The per-item loop, written into the agent's persistent instructions.
# Each step costs turns, which is exactly why the budget must be stated.

1. RETRIEVE   kb search "<the item you are about to do>"
2. EDGE CASES consider empty, null, zero, negative, boundary, duplicate,
              already-terminal — pick the ones that apply
3. EDIT       the smallest change that covers them
4. SELF-REVIEW re-read your own diff as a reviewer would. If it is
              trivial or wrong, rewrite it before committing
5. COMMIT     one logical change, conventional commit message

# At the END of the run:
6. REFLECT    append one dated lesson to LEARNINGS.md
7. REINDEX    kb index — so the next run can retrieve today's lesson

The pipeline is deliberately ordered. Retrieve first, so the edge cases the agent considers are informed by what previous runs found. Self-review before committing, because a model reading its own diff as a reviewer catches the trivially wrong test it just wrote. Then reflect and re-index, so today's lesson is retrievable tomorrow. That last step is what closes the loop; without it the lessons file grows and nothing ever reads it.

What actually goes in the knowledge file

The content that changed behaviour was more specific than I expected, and more about standards than about facts.

• Domain invariants. Explicit states and transitions, pure guard functions, side effects at the edges, audit completeness — the rules a reviewer would apply without thinking.
• An anti-patterns section. Trivially always-true tests, over-mocking, silencing types instead of fixing them, fixing the same bug twice. Naming the anti-pattern was more effective than describing the ideal.
• Modern language idioms to prefer, with names. Concrete feature names are retrievable; write good modern code is not.
• Repository quirks that are not discoverable in a single run: which command runs out of memory, which helper has a surprising return value, which test needs a real service.

The last category is the one the agent contributes best. It cannot tell you what good design is, but it is very good at recording that a particular helper returned undefined when given a path with no dot in it, on the day that cost it twenty minutes.

Does it actually help?

The honest evidence is indirect but consistent. Semantic queries score highest against exactly the chunks a human would pick, the lessons file grew from one line to over twenty within days of the reflection step being added, and the specific misunderstandings that used to recur stopped recurring.

What it did not do is make a weak model strong. Retrieval prevents repeated mistakes; it does not add capability. The throughput gains in my project came from procedure — task sizing, turn budgets, landing deadlines — and memory is what stopped those gains from being eroded by the same three errors every week.

Building this in an afternoon

Five steps, in this order, and stop after step three if that is all you need.

1. Create the lessons file with a seed-if-absent step, so it can never be clobbered by a redeploy, and add one instruction: append a dated lesson at the end of every run.
2. Have the agent read the last few lessons at the start of a run. This alone removes most repeated mistakes and costs nothing.
3. Write the knowledge file yourself, once, with the standards and anti-patterns you would enforce in review.
4. Add retrieval only when the files are big enough that reading them whole is wasteful — and add the lexical fallback at the same time.
5. Re-index after the reflection step, then verify by searching for a phrase from today's lesson and confirming it comes back.

Step five is the verification most people skip, and it is the only one that proves the loop is closed rather than merely configured.

Memory turns an agent from a competent stranger into a colleague who was here yesterday. Three files with different lifecycles, a hundred and fifty lines of retrieval, and one reflection instruction were enough — and the discipline that makes it work is not the vector search, it is remembering to re-index so the lesson learned today is findable tomorrow.