Deep Learning (DL) · Beginner level

Beginner level · Module 5: Training Neural Networks Well · Lesson 1 of 3

Losses and Optimisers

Choose a loss for the task and an optimiser that updates weights efficiently.

In this lesson you will

  • Match loss functions to tasks
  • Compare SGD and Adam optimisers

Key concepts

cross-entropy and MSEclass weights in the lossSGD with momentumAdam

The lesson

The loss must match the task: cross-entropy for classification, mean squared error for regression. Class weights in the loss help when some classes are rare.

Stochastic gradient descent (SGD) updates weights using batches; momentum smooths the updates. Adam adapts the step size for each weight and often trains quickly with little tuning.

Neither optimiser is always best: Adam is a good default for experiments, while well-tuned SGD often generalises slightly better for image models.

Step by step

  1. Train with SGD and with Adam.
  2. Compare their loss curves.
  3. Add class weights for a rare disease class.
  4. Record the effect on recall for that class.

Practice activity

Compare optimisers and class weighting for the leaf classifier.

Expected result: A comparison with loss curves and per-class recall.

Check your understanding

  1. Which loss suits multi-class image classification?

    Show answer

    Cross-entropy.

  2. What does Adam adapt?

    Show answer

    The step size for each weight.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.