Grid Search Hyperparameters in Python

Perform exhaustive grid search over hyperparameter combinations using itertools.product and a scoring function.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 14 views 0 copies

grid-search hyperparameters itertools ml

Python code

30 lines

Python 3.9+

import itertools

def grid_search(param_grid, score_fn):
    """Perform exhaustive grid search over hyperparameter combinations."""
    keys = param_grid.keys()
    names = list(keys)
    values = [param_grid[name] for name in names]
    results = []

    for combination in itertools.product(*values):
        params = dict(zip(names, combination))
        score = score_fn(params)
        results.append((score, params))

    results.sort(key=lambda x: x[0], reverse=True)
    return results

if __name__ == "__main__":
    param_grid = {
        "learning_rate": [0.01, 0.1],
        "depth": [2, 3],
    }

    def mock_score(params):
        # Simple mock: higher score for lower learning_rate, deeper trees
        return 100 - params["learning_rate"] * 100 + params["depth"]

    results = grid_search(param_grid, mock_score)
    for score, params in results:
        print(f"Score: {score:6.2f} | Params: {params}")

Output

stdout

Score: 102.00 | Params: {'learning_rate': 0.01, 'depth': 3}
Score: 101.00 | Params: {'learning_rate': 0.01, 'depth': 2}
Score: 93.00 | Params: {'learning_rate': 0.1, 'depth': 3}
Score: 92.00 | Params: {'learning_rate': 0.1, 'depth': 2}

How it works

The grid_search function builds all combinations of hyperparameter values using itertools.product, evaluating each with the provided score_fn. Results are stored as (score, params) pairs and sorted in descending order by score, making the best configuration appear first. This pattern is ideal for mocked or small-scale experiments where exhaustive search is feasible. The mock scoring function rewards lower learning rates and deeper trees, which are common heuristics in practice.

Common mistakes

Forgetting to convert `itertools.product` results to a list, which can cause iteration issues
Using a score function that mutates the params dict, leading to unexpected results
Sorting in ascending order by default — always set `reverse=True` for best-first results

Variations

Use `GridSearchCV` from scikit-learn for integration with cross-validation
Add early stopping by tracking the best score and skipping unpromising combinations

Real-world use cases

Comparing model configurations in a research prototype before scaling up to full training.
Automating benchmark runs for a small ML pipeline to decide which hyperparameters ship.
Testing algorithm variants in a feature store experiment to pick the fastest config.

Grid Search Hyperparameters in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Keep learning

Tutorials

Quizzes