Intermediate level · Module 3: Hyperparameter Tuning · Lesson 1 of 3
Grid and Random Search
Tune hyperparameters systematically with cross-validated grid and random search.
In this lesson you will
- Distinguish parameters from hyperparameters
- Run GridSearchCV and RandomizedSearchCV on a pipeline
Key concepts
hyperparameterGridSearchCVRandomizedSearchCVstep__param naming
The lesson
Parameters are learned during training, such as regression coefficients. Hyperparameters are set before training, such as a tree's max_depth or k in k-NN.
GridSearchCV tries every combination in a grid with cross-validation; RandomizedSearchCV samples a fixed number of combinations from ranges. Random search usually finds good settings faster when there are many hyperparameters.
Tune the whole pipeline by naming parameters with the step name: model__max_depth. The best combination is then refitted on all training data.
Step by step
- Define a small grid for a random forest.
- Run GridSearchCV on the pipeline.
- Run RandomizedSearchCV with the same budget.
- Compare best scores and run times.
Practice activity
Tune two models for the agent problem and record each search's settings and results.
Expected result: A tuning log with search spaces, best settings, scores and run times.
Check your understanding
-
Is max_depth a parameter or a hyperparameter?
Show answer
A hyperparameter; it is set before training.
-
When is random search preferable to grid search?
Show answer
When there are many hyperparameters or a limited time budget.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.