Deep Learning (DL) · Intermediate level

Intermediate level · Module 5: Evaluating Deep Learning Models · Lesson 1 of 3

Validation Strategies

Choose validation and test sets that reflect how the model will really be used.

In this lesson you will

  • Design validation and test sets for real conditions
  • Report per-class metrics

Key concepts

in-distribution vs field testper-class recallmacro-averaged metricstest-set discipline

The lesson

A random test split from the same photo sessions measures performance on familiar conditions. A separately collected field test — different days, phones, people — shows how the model will actually perform.

Report metrics per class and macro-averaged across classes, so poor performance on a rare package is not hidden by common ones.

Keep the test set untouched until the end; use the validation set for all choices.

Step by step

  1. Evaluate on the random test split.
  2. Evaluate on the field test set.
  3. Compare per-class recall.
  4. Explain the differences.

Practice activity

Report in-distribution and field performance for both architectures.

Expected result: Per-class results on both test sets with an explanation of gaps.

Check your understanding

  1. Why collect a separate field test set?

    Show answer

    It shows performance under real, unfamiliar conditions.

  2. What does macro-averaging do?

    Show answer

    Gives each class equal weight in the average.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.