Agent Memory Benchmark

We ran a scripted 8-turn conversation with each agent. The conversation included:

• Stated user preferences
• A decision made in turn 3
• Facts introduced in turns 2 and 5
• A final task in turn 8 that required combining earlier context

Agents were scored on how much relevant context they recalled and used correctly.

Criterion	Weight	Max points
Preference recall	25%	25
Fact recall	25%	25
Decision recall	20%	20
Correct use in new task	25%	25
Setup complexity	5%	5

• Same conversation script for every agent.
• No explicit reminders after turn 1 unless the agent asked.
• We restarted the conversation for each agent to test cold memory.
• Memory configuration followed each agent's official docs.

• Letta is purpose-built for memory and it shows. Cross-turn recall was the strongest.
• OpenClaw memory works well through skills, but you have to design for it.
• Hermes Agent handles session memory but needs configuration for persistence.
• Claude Code with a memory file is a workable hack, not a real memory system.

• We tested one scripted conversation. Real user behavior is messier.
• Memory quality depends on implementation and prompt design.

• Letta: agents where long-term memory is core to the product.
• OpenClaw: self-hosted agents that need persistent skills and user context.
• Hermes Agent: messaging agents where session continuity matters.
• Claude Code: short projects where a manual memory file is enough.