Measure Before You Optimize

Agent builders love to optimize. Swap a cheaper model. Change the chunk size. Try a new vector database. These changes feel productive, but without measurement they are guesses.

The slow part of your agent workflow might not be the LLM. It might be:

• Network calls to a tool API
• Repeated retrieval of the same chunks
• Inefficient tool loops
• Large input prompts
• Cold starts on local models

Until you measure, you are optimizing a mirror.

Metric	Why it matters
Latency per task	End-to-end time, not just model time
Cost per task	Tokens + tool calls + compute
Success rate	Did the agent complete the task correctly?
Retry rate	How often do you have to re-prompt?
Review time	How long does human validation take?

Pick 5–10 representative tasks. Run each one three times. Record latency, cost, and quality. That baseline tells you where to focus.

• "A cheaper model will save money." It might fail more and cost more in retries.
• "A bigger context window will help." It might just make retrieval noisier.
• "Local is cheaper." It might be slower and require expensive hardware.
• "More tools are better." Each tool adds latency and failure surface.