Normalize Data in Python with Comprehensions and Generators

Clean a list by dropping None values with a comprehension, then min-max normalize it using a lazy generator expression — a beginner-friendly data preparation pattern.

Easy Python 3.6+ Aug 9, 2026 Comprehensions & generators 13 views 0 copies

Python code

31 lines
Python 3.6+
import statistics

# Sample raw data including missing and outlier-ish values
raw = [22, 18, None, 25, 30, 19, 22, 17, None, 28, 24]

# Clean the data: drop None values using a list comprehension
clean = [x for x in raw if x is not None]

# Normalize using min-max scaling with a generator expression
min_val = min(clean)
max_val = max(clean)

def scale(value):
    return (value - min_val) / (max_val - min_val)

# Generator expression: lazy evaluation, yields values one at a time
normalized_gen = (scale(x) for x in clean)

# Convert generator to list to observe results
normalized = list(normalized_gen)

# Show stats and output
print("Raw:", raw)
print("Clean:", clean)
print("Normalized (rounded to 3 decimals):", [round(x, 3) for x in normalized])
print("Mean:", round(statistics.mean(normalized), 3))
print("Min:", round(min(normalized), 3), "Max:", round(max(normalized), 3))

# Demonstrate generator laziness: sum without materializing
total = sum(scale(x) for x in clean)
print("Sum (from generator):", round(total, 3))

Output

stdout
Raw: [22, 18, None, 25, 30, 19, 22, 17, None, 28, 24]
Clean: [22, 18, 25, 30, 19, 22, 17, 28, 24]
Normalized (rounded to 3 decimals): [0.385, 0.077, 0.615, 1.0, 0.154, 0.385, 0.0, 0.846, 0.538]
Mean: 0.444
Min: 0.0 Max: 1.0
Sum (from generator): 4.0

How it works

The list comprehension [x for x in raw if x is not None] creates a new list with only valid values, making the dataset ready for numerical processing. The generator expression (scale(x) for x in clean) computes normalized values lazily — each value is produced on demand rather than all at once, saving memory for large datasets. The closure over min_val and max_val in scale keeps the normalization logic simple and reusable. Converting to a list with list(normalized_gen) materializes results for inspection, while sum(scale(x) for x in clean) shows how generators avoid intermediate storage.

Common mistakes

  • Using `== None` instead of `is None` — identity checks are the Pythonic way for None.
  • Modifying the raw list instead of creating a new cleaned one, which changes source data.

Variations

  1. Use `(x for x in raw if x is not None)` as a generator directly without materializing the clean list.
  2. Replace min-max scaling with `(x - mean) / std` for z-score normalization using statistics.

Real-world use cases

  • Preparing feature columns for machine learning models by cleaning and scaling numeric data.
  • Filtering faulty sensor readings (None values) before calculating rolling averages in IoT pipelines.
  • Normalizing API response values to a 0–1 range for consistent dashboard chart scaling.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Comprehensions & generators

Related tutorials and quizzes for this topic.