Advanced level · Module 4: Testing and Performance in C++ · Lesson 4 of 4
Memory Layout and Cache-Friendly Code
Understand why contiguous data is fast, and lay out data so the processor's caches work for you.
In this lesson you will
- Explain the effect of caches on performance
- Choose contiguous layouts and avoid needless indirection
Key concepts
CPU cachecontiguous memorypointer chasingstructure of arrays
The lesson
Processors read memory in blocks called cache lines. Data stored contiguously, as in std::vector, is read efficiently because neighbouring elements arrive together. Linked structures such as std::list scatter elements across memory, so each step may wait for memory.
A vector of pointers to heap objects has the same problem: iterating follows a pointer for every element. Storing objects by value in a vector is often several times faster.
When a loop uses only one field of large records, a structure of arrays — separate vectors for timestamps and values — reads less memory than an array of structures. Measure before and after; layout changes are worth it only for hot loops.
Step by step
- Sum values in a std::list and a std::vector of the same size and compare times.
- Compare a vector of objects with a vector of unique_ptrs.
- Rewrite a hot loop to use a structure of arrays.
- Record the measurements.
Practice activity
Apply one layout optimisation to the sensor processor and report its measured effect.
Expected result: A measured result, positive or not, with an explanation of why the layout change helped or did not.
Check your understanding
-
Why is iterating a std::vector usually faster than a std::list?
Show answer
Its elements are contiguous, so the cache loads neighbours together.
-
What is a structure of arrays?
Show answer
Storing each field in its own array instead of an array of records.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.