Intermediate level · Module 4: GPUs and Efficient Training · Lesson 2 of 3
Mixed Precision and Batch Size
Train faster and fit larger models with mixed precision, and choose batch sizes sensibly.
In this lesson you will
- Enable automatic mixed precision
- Choose batch size and adjust the learning rate
Key concepts
16-bit vs 32-bit precisionautomatic mixed precisionbatch size and memorygradient accumulation
The lesson
Mixed precision uses 16-bit numbers for most calculations, which halves memory and speeds up modern GPUs, while keeping 32-bit where needed for stability. PyTorch's autocast and gradient scaler make it a few lines of code.
Larger batches use the GPU more efficiently but need more memory and may need a larger learning rate. If memory is short, gradient accumulation sums gradients over several small batches before each update.
Measure the effect on speed and accuracy; keep changes that help both.
Step by step
- Enable mixed precision and compare speed and memory.
- Find the largest batch that fits.
- Adjust the learning rate accordingly.
- Implement gradient accumulation.
Practice activity
Add mixed precision and a tuned batch size to the training code.
Expected result: Faster training with unchanged or better validation accuracy.
Check your understanding
-
What does mixed precision reduce?
Show answer
Memory use and training time.
-
What does gradient accumulation allow?
Show answer
Simulating a larger batch when memory is limited.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.