Deep Learning (DL) · Beginner level

Beginner level · Module 1: Neural Network Foundations · Lesson 2 of 3

Layers and the Forward Pass

Connect neurons into layers and follow data through a network to a prediction and a loss.

In this lesson you will

  • Describe input, hidden and output layers
  • Compute a forward pass and a loss in PyTorch

Key concepts

input, hidden and output layersfully connected layerforward passloss

The lesson

A network arranges neurons in layers. The input layer receives the features, hidden layers transform them and the output layer produces the prediction — one value for regression or one score per class for classification.

The forward pass sends data through every layer in turn. In PyTorch, nn.Linear creates a fully connected layer and a model's forward method defines the order.

The loss measures how wrong the prediction is: mean squared error for regression, cross-entropy for classification. Training aims to make the loss small on the training data without overfitting.

Step by step

  1. Build a two-layer network with nn.Linear and ReLU.
  2. Pass a batch of inputs through it.
  3. Calculate the cross-entropy loss against labels.
  4. Print the shapes at each step.

Practice activity

Build the practical activity's tiny network in PyTorch and confirm it matches your hand calculation.

Expected result: PyTorch output identical to the hand calculation, with shapes explained.

Check your understanding

  1. Which loss suits classification?

    Show answer

    Cross-entropy.

  2. What does nn.Linear create?

    Show answer

    A fully connected layer.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.