Evaluation routes
Frontend route:Evaluation workflow
1
Create an evaluation set
Group related test cases under one evaluation set.
2
Add cases
Add prompts and expected criteria for the target agent behavior.
3
Run the evaluation
Execute the evaluation set against an agent.
4
Review the run
Inspect run output and compare results over time.
When to evaluate
- Before publishing an agent.
- After changing tools or model configuration.
- After changing the agent instruction.
- After backend execution changes.
Evaluation run features
Incremental streaming results
During an evaluation run, Pinter streams test case outcomes incrementally:- Case passes and failures are rendered in real time as each case finishes, without waiting for the entire batch to finish.
- Intermediate scores and tool trajectory matches update live on the dashboard.
Agent version binding and historical audit
Evaluation runs record the exact version number of the target agent (agent_version). This ensures that score history reflects whether a regression happened due to a prompt change, a tool change, or a model switch.