Data Science · Advanced level

Advanced level · Module 3 of 6 · 22 guided hours

Working with Larger and Spatial Data

Handle data too large for simple workflows, query it with SQL and analyse and map location data.

Learning objectives

  • Process larger data efficiently in Python and SQL
  • Analyse and map spatial data
  • Understand when distributed big-data tools are needed

Topics

  • memory-efficient pandas and Parquet
  • chunked processing
  • SQL for analysts with DuckDB or SQLite
  • spatial data: points, boundaries and coordinate systems
  • GeoPandas and maps
  • big data concepts

Lessons

  1. Lesson 1 Efficient Data Processing

    Work with data that strains memory by choosing types, file formats and chunked processing.

    memory usagecategorical dtypeParquetchunked reading

  2. Lesson 2 SQL for Analysts

    Use SQL engines such as DuckDB and SQLite to filter, join and aggregate large data quickly.

    DuckDBanalytical SQLquerying files directlySQL to DataFrame

  3. Lesson 3 Spatial Data and Maps

    Work with points and boundaries, join them correctly and make clear, safe maps.

    points and polygonscoordinate reference systemspatial joinchoropleth map

  4. Lesson 4 Big Data Concepts

    Understand when data truly needs distributed tools and what those tools do.

    distributed processingSparkcloud data warehousescost and complexity

Practical activity

Query a multi-million-row synthetic transaction file with DuckDB, join household points to administrative boundaries and map need by county.

Expected outcomes

  • Efficient handling of larger data
  • Correct spatial joins and clear maps

Assessment

Practical marked on efficiency, correct spatial joins and map design.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.