Ask Again

A bet on asking instead of remembering

@vieko

The default move in AI tooling right now is to give the agent a memory. Cursor indexes your codebase. Claude Code writes its own notes about your project and reloads them every session. Nightshift writes its session state to disk so its overnight loop can resume in the morning.[1] Underneath all of it is the same premise: models forget, so we have to remember on their behalf.

Some of that is right. Some of it is the wrong tool for the job.

I've built two of these. Bonfire on purpose, Forge without meaning to. Bonfire memorizes what the human side cares about: the current goal, yesterday's blocker, the next step. The codebase can't tell you any of that, so something has to. Forge memorized what the codebase could already tell you: how it was built. That was the mistake.[2]

Five jobs, one file

In Forge, every task produces a spec. The spec is the durable artifact: the outcome, the acceptance criteria, the files the agent decides to touch, and what it discovered about the codebase while planning. When the next agent runs the spec, it inherits everything the planner figured out. No rediscovery. That was the plan.

The first time a spec went stale I assumed it was a bug. After a few tries, I had to admit it wasn't. It was working as designed.

I had asked the spec to do five jobs at once:

  • Contract--the outcome and acceptance criteria the agent is held to.
  • Schedulable unit--a node in the dependency graph.
  • Resumable handle--what survives worktree cleanup so a run can resume.
  • Audit artifact--reviewable evidence of intent, stored on disk.
  • Discovery cache--what the agent learned about the codebase while planning.

Pulled apart, four of them have clean homes now. The contract lives in the final turns of the conversation that produces the work. The schedule lives in a row in the tasks table, where it always belonged. The resumable handle is a foreign key. The audit artifact is the conversation itself, optionally exported when humans need to review.

The discovery cache needed something else. Not a new column. A reason to stop storing it.

The cache that was always wrong

A cached discovery is a map drawn the day the work was planned. Accurate the moment it's made. Then the territory drifts. A bridge comes down, a one-way flips, and nobody marks the map. Every street it names still exists, so the next task trusts it, follows it across town, and finds the bridge gone. The mistake was miles back, on a map that looked right the whole way.

The cache disagreed with the codebase. Nothing flagged it.

The usual response is to build cache invalidation. Track which files a spec depended on. Rebuild it when they change. Add staleness checks, expiry, a freshness indicator in the TUI.

I have written every one of those things. They don't solve the problem. They postpone it. Every cache invalidation strategy is a bet that you can predict which changes matter, and the bet loses every time the codebase surprises you.

The codebase already knew. It always had. Every line I was caching was a worse copy of something the source could answer directly. The spec wasn't memory. It was denormalization. The tax is supposed to come out of disk. Mine came out of correctness.

Keep the question, not the answer

What columns does the scores table have? is a valid question at any commit, forever. It backs the leaderboard for a game I built, and today the codebase answers:

CREATE TABLE scores (
  name VARCHAR(3),
  score INTEGER,
  elapsed_months REAL,
  elapsed_seconds INTEGER,
  unicorn_count INTEGER,
  momentum REAL,
  team_stats JSONB
);

A few versions back there was no unicorn_count, no elapsed_seconds. I added each when the game needed to track one more thing. The next version will add another. The question is stable. The answer is whatever the codebase says when you point at it.

A written-down question can be wrong too. But a wrong question fails loud. Ask it against the codebase and the false assumption gets caught: the model comes back with “there is no such table” instead of inventing one. A wrong answer just sits there, looking right.

The trick is to ask without naming the answer. How is unicorn_count computed? dies the day I rename the column. How does the game count unicorns per run? survives the rename, the retype, the table split. Same intent, one phrasing brittle and one durable.

Forge keeps a question library per project now. None of the questions describe my code. All of them describe how to ask about it. One library has survived three refactors that would have invalidated a spec.

Single source of truth, all over again

The principle isn't new. Don't denormalize without measuring the cost. “Single source of truth,” repeated in every database textbook for forty years.

What is new is the search surface. The model can read files, search through them, run commands against them, the same things a senior engineer does on the first day of a new project. The question “what runs the build?” doesn't need an embedding. It needs three minutes and a grep.

Much of what is being called agent memory right now is a denormalized view of a database that is still online. The database is your repo. The query interface is already in the agent's hands. We keep paying the cache-invalidation tax because we keep building the cache.

Memory is the answer when the source is gone. Embeddings are the answer when the source is too slow to scan. Neither is true of your code.

The bet

The new Forge is shipping in slices, behind a flag. The first slice is a benchmark. Warm context from a question-driven interview against cold execution, on a fixed task set, scored on whether verify passes and what it costs to get there.

If the results support the bet, the migration proceeds. If they don't, this is a story about something I tried and learned from.



Illustration by Jon Romero Ruiz.

[1] Nightshift is Pranit Sharma's own project. We're colleagues at Vercel; I'm reading it from the outside.

[2] This is a sequel of sorts to Outcomes. That piece was about pulling orchestration out of Forge and letting verification decide what “done” means. This one is about what to keep between runs, and what to stop keeping.