Data Science · Intermediate level

Intermediate level · Module 2 of 6 · 16 guided hours

Data Wrangling with pandas and NumPy

Reshape, merge and transform data from several sources efficiently and reliably.

Learning objectives

  • Reshape data between wide and long formats
  • Merge data sets correctly and verify the result
  • Use vectorised NumPy and pandas operations, dates and text methods

Topics

  • tidy data
  • melt and pivot
  • merge types and keys
  • validating merges
  • vectorised operations
  • dates, times and text

Lessons

  1. Lesson 1 Tidy Data and Reshaping

    Organise data so each variable is a column and each observation a row, and convert between wide and long formats.

    tidy datawide vs long formatmeltpivot_table

  2. Lesson 2 Merging Data Sets

    Join tables from different sources on keys and verify that the result is what you expect.

    merge keyshow='left' / 'inner' / 'outer'indicator=Trueone-to-one vs one-to-many

  3. Lesson 3 Vectorised Operations, Dates and Text

    Transform data efficiently with vectorised operations and handle dates and text correctly.

    vectorisationnp.wherepd.to_datetime.str methods

Practical activity

Combine the school census, attendance records and survey into one analysis table, checking every merge and documenting the final data dictionary.

Expected outcomes

  • Reliable combined data sets
  • Efficient, readable transformation code

Assessment

Wrangling practical marked on merge correctness, checks and documentation.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.