ToolCall-15: Deterministic Tool-Use Bench Pack

ToolCall-15 is an official BenchLocal Bench Pack for evaluating deterministic tool use: tool selection, parameter precision, multi-step tool chains, restraint, and recovery from tool errors.

It is smaller and more focused than tool-eval-bench, which makes it excellent as a fast regression gate when you change models, samplers, or tool schemas.

Five categories, three scenarios each:

1. Tool Selection
2. Parameter Precision
3. Multi-Step Chains
4. Restraint and Refusal
5. Error Recovery

Each scenario scores:

• 2 pass
• 1 partial
• 0 fail

Each category is worth 6 points. The final score averages the five category percentages.

• Reproducible: system prompt, tool schema, mocked tool outputs, and scoring are versioned
• Balanced: failure modes are spread, not over-indexed on one skill
• Deterministic: mocked tools, temperature 0 defaults
• Inspectable: raw traces for failed scenarios

BenchLocal (recommended)

1. Install BenchLocal from its latest release
2. Install ToolCall-15 from the official Bench Pack registry
3. Add models, select ToolCall-15, run

CLI / development

The repository includes a CLI runner and BenchLocal adapter (npm install, npm run cli, npm run build:benchlocal — see upstream README).

Most public "agent benchmarks" still under-test tool discipline. ToolCall-15 is a practical pack you can run locally before promoting a model into production agent roles.

• Fifteen scenarios will not cover every enterprise tool edge case
• Mocked tools cannot fully simulate flaky third-party APIs
• Use as a gate, not as the only evaluation

• tool-eval-bench for a larger 69+ scenario expansion of the same idea
• BenchLocal as the desktop runner for multiple packs