Data Science · Beginner level

Beginner level · Module 2: Exploratory Data Analysis with pandas · Lesson 3 of 3

Patterns, Outliers and Data Problems

Use plots and checks to find relationships, outliers and data problems, and decide how to handle them.

In this lesson you will

  • Use scatter plots and box plots to explore relationships and outliers
  • Document how data problems are handled

Key concepts

scatter plotbox plotoutliercleaning log

The lesson

Scatter plots show relationships between two numeric variables; box plots compare distributions across groups and show outliers as individual points.

Not every outlier is an error: a household that walks 120 minutes may be real and important. Check with the data owners before removing anything.

Keep a cleaning log: what you changed, why and how many rows were affected. It makes your results trustworthy and repeatable.

Step by step

  1. Plot walking time against household size.
  2. Draw box plots of walking time by district.
  3. Investigate the most extreme outliers.
  4. Write the cleaning log entries.

Practice activity

Complete the exploration of the water survey with a cleaning log and a list of questions for the data collectors.

Expected result: A cleaning log and a list of well-founded questions.

Check your understanding

  1. Should outliers always be removed?

    Show answer

    No; they may be real and important. Check them first.

  2. Why keep a cleaning log?

    Show answer

    So changes are transparent and the analysis can be repeated.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.