Externalize Agent Memory to the Filesystem

Context windows are ephemeral, finite, and increasingly expensive as they grow. The filesystem is unlimited in size, persistent by nature, and directly operable by the agent itself. When you have state that needs to survive across turns, across sessions, or across agents, write it to a file — do not keep it in the context window.

Every token in the context window costs money and attention. A 128K-token context at $5/M input tokens costs $0.64 per request just for the context. If that context is mostly stale notes, intermediate results, and task state that the model does not need to see on every turn, you are paying for noise.

The filesystem solves three problems at once:

1. Size: No practical limit. Write 10MB of notes, 500MB of intermediate data, 50,000 lines of scratch code — the filesystem holds it all.
2. Persistence: State survives across sessions, restarts, and failures. The context window does not.
3. Agency: The agent can read, write, update, and delete files using tools it already has. No special infrastructure needed.

Task state

Write a todo.md that tracks task progress. The agent updates it after each step. This keeps objectives visible in recent attention without consuming context budget.

# TODO: Refactor authentication module

- [x] Read current auth middleware
- [x] Identify deprecated JWT calls
- [ ] Replace with new token service
- [ ] Update tests
- [ ] Verify integration

Decisions log

Write a decisions.md that records key choices and their rationale. This prevents the agent from re-litigating decisions it already made 20 turns ago.

• Reason: Built-in persistence needed for session recovery
• Alternatives considered: Memcached (no persistence), PostgreSQL (too heavy)
• Tradeoff: Slightly higher memory usage

### Intermediate results

Write computed values, parsed data, and analysis output to files. The agent reads them back only when needed, not on every turn.

### Structured knowledge

Write a `context.md` or `architecture.md` that the agent can reference when it needs project context but does not need to see on every call.

Most agent frameworks already have file read/write tools. The pattern is:

1. At task start: Read the relevant files (todo.md, decisions.md, context.md)
2. During work: Update files as state changes — after each significant step
3. At task end: Write a summary to a results file
4. On next session: Read the files to restore state without re-deriving everything

Consider an agent running a 50-turn task with 100K tokens of context:

• All in context: 50 turns × 100K tokens × $5/M = $25.00 in input costs alone
• Externalized: 50 turns × 20K tokens (active context) × $5/M = $5.00, plus negligible filesystem I/O

The 5x cost reduction is conservative. Real savings are often higher because externalized state also reduces output tokens — the agent does not need to restate things it already wrote down.

• Short, one-shot tasks — the overhead of file management is not worth it
• State that changes every turn — if the agent needs to see it constantly, keep it in context
• Security-sensitive data — do not write secrets, credentials, or PII to plaintext files. Use proper secret management.

• Bookend critical context — keep the most important instructions in context at the edges, externalize everything else
• Just-in-time loading — read files only when the agent needs them, not preemptively
• Compress as you go — summarize old context and write the summary to a file, then drop the original from the window