Deep Learning (DL) · Intermediate level

Intermediate level · Module 4: GPUs and Efficient Training · Lesson 1 of 3

Using GPUs Effectively

Understand why GPUs speed up deep learning and check that your training actually uses them well.

In this lesson you will

  • Explain why GPUs suit deep learning
  • Monitor GPU utilisation and find bottlenecks

Key concepts

parallel computationGPU memoryutilisation monitoringdata loading bottleneck

The lesson

GPUs perform thousands of simple calculations in parallel, which matches the matrix operations at the heart of neural networks; training can be many times faster than on a CPU.

Check utilisation with nvidia-smi. If the GPU is often idle, the bottleneck is elsewhere — usually loading and decoding images. Use more DataLoader workers and pre-resized images.

Debug on a small subset on a CPU first; spend GPU time only on runs that are ready.

Step by step

  1. Train one epoch on CPU and GPU and compare times.
  2. Watch utilisation during training.
  3. Increase DataLoader workers and pre-resize images.
  4. Measure the improvement.

Practice activity

Profile and speed up the project's training pipeline.

Expected result: A before-and-after timing with the bottlenecks identified.

Check your understanding

  1. Why are GPUs fast for deep learning?

    Show answer

    They perform many simple calculations in parallel.

  2. What does a frequently idle GPU suggest?

    Show answer

    A bottleneck elsewhere, often data loading.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.