Multi-Turn Tool Consistency Benchmark

We ran a scripted 5-turn workflow with each agent. Each turn required the agent to call a tool, then use the result in the next turn. The workflow included:

• A filesystem search in turn 1 whose output was needed in turn 3.
• A configuration value returned in turn 2 that had to be referenced in turn 4.
• A user preference stated in turn 1 that constrained the final action in turn 5.

Agents were scored on whether they used prior tool outputs correctly without being reminded.

Criterion	Weight	Max points
Tool output recall	35%	35
Correct chaining	30%	30
User preference recall	20%	20
No hallucinated prior results	15%	15

• Same script and tool set for every agent.
• No explicit reminders after turn 1 unless the agent asked.
• We scored each turn independently and averaged across runs.

• Claude Code retained the most context across turns, especially for chained tool use.
• OpenClaw performed well when skills were designed to emit structured state.
• Hermes Agent handled session continuity but sometimes needed restatement for older turns.
• OpenHands was strong on individual tool calls but drifted on later turns.

• The workflow is synthetic. Real user conversations are less predictable.
• Results depend on how each agent is configured for memory and context.

• Claude Code: complex multi-turn workflows where context carries across many steps.
• OpenClaw: self-hosted agents that persist state through skills and gateways.
• Hermes Agent: messaging agents where most turns stay within a focused session.
• OpenHands: single-turn or lightly chained tool tasks.