How to Normalize Data with Dictionaries and Sets in Python

Normalize dictionary entries to a fixed set of keys and extract unique values using sets in Python.

Easy Python 3.9+ Aug 9, 2026 Dictionaries & sets 14 views 0 copies

dictionaries sets data-cleaning normalization

Python code

27 lines

Python 3.9+

def normalize_entry(entry: dict, valid_keys: set) -> dict:
    result = {}
    for key in valid_keys:
        result[key] = entry.get(key, "")
    return result


def unique_values(entries: list[dict], key: str) -> set:
    return {entry.get(key) for entry in entries if entry.get(key) is not None}


if __name__ == "__main__":
    raw_data = [
        {"name": "Alice", "age": 30, "city": "London"},
        {"name": "Bob", "age": 25},
        {"name": "Alice", "age": 31, "city": "Paris"},
        {"name": "Carol", "age": 35, "city": "London"},
    ]
    expected_keys = {"name", "age", "city"}

    normalized = [normalize_entry(entry, expected_keys) for entry in raw_data]
    print("Normalized entries:")
    for entry in normalized:
        print(entry)

    cities = unique_values(raw_data, "city")
    print(f"\nUnique cities: {sorted(cities)}")

Output

stdout

Normalized entries:
{'name': 'Alice', 'age': 30, 'city': 'London'}
{'name': 'Bob', 'age': 25, 'city': ''}
{'name': 'Alice', 'age': 31, 'city': 'Paris'}
{'name': 'Carol', 'age': 35, 'city': 'London'}

Unique cities: ['London', 'Paris']

How it works

The normalize_entry function iterates over a set of valid keys and uses the dictionary's get method to safely retrieve each value, filling missing keys with an empty string. This ensures every output dictionary has the same structure, which is useful for consistent downstream processing. The unique_values function uses a set comprehension to collect distinct non-None values for a given key, automatically removing duplicates. Using a set for iteration in normalize_entry makes the order non-deterministic, but since dictionary creation order doesn't matter for equality, this is fine. Together, these functions provide a simple, clean way to standardize and explore data.

Common mistakes

Forgetting to handle missing keys, leading to KeyError instead of using .get()
Using a list instead of a set, which would include duplicate values
Assuming set iteration order is stable, which it is not in Python

Variations

Use a dictionary comprehension: `{key: entry.get(key, '') for key in valid_keys}`
Filter out None values from the set comprehension: `{entry[key] for entry in entries if key in entry}`

Real-world use cases

Preparing API response payloads before inserting them into a database with a fixed schema.
Extracting unique user cities from a list of records to generate a dropdown filter in a UI.
Cleaning CSV-like data where different rows have varying columns, ensuring uniform column sets.

How to Normalize Data with Dictionaries and Sets in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Dictionaries & sets

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Dictionaries & sets

Keep learning

Tutorials

Quizzes