Deep Learning (DL) · Intermediate level

Intermediate level · Module 3: Building Image and Text Data Sets · Lesson 2 of 3

Label Quality and Leakage-Free Splits

Measure label agreement, find label errors and split data so evaluation is honest.

In this lesson you will

  • Measure label agreement and correct errors
  • Split data without near-duplicate leakage

Key concepts

inter-annotator agreementlabel noisenear-duplicate imagesgroup-based splitting

The lesson

Have a sample labelled twice and measure agreement; low agreement means the guidelines are unclear. Review images the model gets confidently wrong — many turn out to be label errors.

Photos taken seconds apart are near-duplicates. If one is in training and another in testing, the test score is inflated.

Split by group — the photo session, the package or the person who took it — so related images stay in the same split.

Step by step

  1. Double-label a sample and measure agreement.
  2. Fix the guideline problems found.
  3. Find near-duplicates with image hashing.
  4. Split by photo session.

Practice activity

Produce quality metrics and leakage-free splits for the project data.

Expected result: Agreement figures, corrected labels and group-based splits.

Check your understanding

  1. Why do near-duplicates across splits inflate results?

    Show answer

    The model is tested on images almost identical to ones it trained on.

  2. What does low labeller agreement suggest?

    Show answer

    Unclear guidelines or genuinely ambiguous cases.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.