Deep Learning (DL) · Beginner level

Beginner level · Module 3: Sequence Models for Text and Time · Lesson 1 of 3

Representing Sequences and Words

Turn text into sequences of numbers and learn why embeddings capture meaning better than one-hot codes.

In this lesson you will

  • Tokenise text and build a vocabulary
  • Explain word embeddings

Key concepts

tokenvocabularyone-hot vectorembedding

The lesson

Sequences are ordered data: words in a sentence, readings from a sensor each hour. Order matters — 'not good' differs from 'good, not'.

Text is split into tokens (words or word pieces) and each token is mapped to an index in a vocabulary. A one-hot vector for each word is huge and treats all words as equally different.

An embedding maps each token to a short dense vector learned during training. Words used in similar ways end up with similar vectors, which helps models generalise.

Step by step

  1. Tokenise a set of SMS messages.
  2. Build a vocabulary with an unknown-word token.
  3. Convert messages to index sequences and pad them.
  4. Create an embedding layer and inspect its output shape.

Practice activity

Prepare the SMS data set as padded index sequences ready for a model.

Expected result: Correctly tokenised, indexed and padded sequences with a documented vocabulary.

Check your understanding

  1. What is an embedding?

    Show answer

    A short, learned vector that represents a token.

  2. Why pad sequences?

    Show answer

    So messages of different lengths can be batched together.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.