You cannot improve what you do not measure. AI features need tests just like code, but the answers are not exact, so the tests look different.
Start with a dataset
- Collect real inputs and write the expected output or a checklist for each.
- Include easy, hard, and adversarial cases.
- Keep a golden set that you do not tune on, to avoid fooling yourself.
- Add every bug you find as a new case.
What to measure
- Correctness: is the answer right?
- Relevance: does it address the question?
- Faithfulness / groundedness: is every claim supported by the provided sources?
- Format: does it match the schema?
- Retrieval (for RAG): precision, recall and F1.
- Latency and cost per request.
Ways to score
- Code checks: exact match, schema validation, regular expressions. Cheap and reliable when possible.
- LLM-as-a-judge: another model grades the output against a rubric. Fast, but it has biases (for example liking longer answers), so check it against human grades.
- Human review: slowest, best for subjective quality.
- Pairwise comparison: show two outputs and pick the better one, often easier than absolute scores.
Regression testing
Run the full set on every change to a prompt, model or retrieval setting. If one metric improves and another drops, you will see it.
Agents
Evaluate the trajectory: did it choose the right tools, in a sensible order, within its budget, as well as the final result.
Benchmarks
Public benchmarks do not predict how a model does on your task. Use them only as a rough guide, then test your own cases.