Synthetic Evaluation Data

Synthetic evaluation data consists of artificially generated test cases used to assess AI model performance. Instead of relying solely on real-world data for evaluation, organisations create targeted test scenarios that systematically cover edge cases, rare events, and specific capabilities.

Why synthetic evaluation data is needed

Real-world evaluation data has several limitations:

• Coverage gaps: Some important scenarios are rare in real data. A fraud detection system might encounter thousands of normal transactions for every fraudulent one.
• Privacy constraints: Real data often cannot be used for evaluation due to privacy regulations. Synthetic data avoids these constraints.
• Cost: Labelling real data for evaluation requires human effort and domain expertise. Generating synthetic data can be automated.
• Bias: Real evaluation data reflects the biases of its collection process. Synthetic data can be deliberately balanced.

Generating synthetic evaluation data

Several approaches are used:

• Template-based generation: Define templates with variable slots and fill them systematically. "What is the capital of {country}?" generates hundreds of factual questions.
• LLM-generated data: Use a powerful language model to generate realistic test cases. This can produce diverse, natural-sounding examples quickly, though the generated data inherits the generating model's biases.
• Perturbation-based: Take real examples and systematically modify them — changing names, swapping genders, adjusting numbers, introducing errors — to test robustness and fairness.
• Rule-based generation: Use domain rules to create valid test cases. For a tax calculation system, generate returns with known correct outcomes.
• Adversarial generation: Deliberately create challenging or tricky examples designed to expose model weaknesses.

Best practices

• Complement, do not replace: Synthetic data should supplement real evaluation data, not replace it entirely. Real data captures patterns that are difficult to synthesise.
• Validate the synthetic data: Ensure generated test cases are actually realistic and meaningful. Unrealistic test cases lead to misleading evaluation results.
• Control for contamination: If using an LLM to generate evaluation data, ensure the model being evaluated was not trained on similar data — otherwise you are testing memorisation, not capability.
• Stratified coverage: Design synthetic data to systematically cover important categories, difficulty levels, and edge cases.
• Version control: Track evaluation datasets with the same rigour as code, so results are reproducible.

LLM-as-judge

A related technique is using one language model to evaluate the outputs of another. The evaluating model scores responses for quality, accuracy, and helpfulness. While imperfect, this approach scales much better than human evaluation and correlates reasonably well with human judgement for many tasks.