Normalize unicode accents to ASCII in Python

This code converts accented Unicode characters to ASCII equivalents using the standard library's unicodedata module.

Easy Python 3.9+ Aug 9, 2026 Strings & text 14 views 0 copies

unicode accents normalization text-processing

Python code

15 lines

Python 3.9+

import unicodedata

def normalize_accents(text: str) -> str:
    """Convert accented unicode characters to ASCII equivalents."""
    decomposed = unicodedata.normalize('NFD', text)
    ascii_text = ''.join(
        char for char in decomposed
        if unicodedata.category(char) != 'Mn'
    )
    return unicodedata.normalize('NFC', ascii_text)

if __name__ == "__main__":
    sample = "Café déjà vu — naïve Zürich über Straße"
    print(f"Input:  {sample}")
    print(f"Output: {normalize_accents(sample)}")

Output

stdout

Input:  Café déjà vu — naïve Zürich über Straße
Output: Cafe deja vu — naive Zurich uber Strasse

How it works

The function first normalizes the input text using NFD (Canonical Decomposition), which separates base characters from combining diacritical marks. Then it filters out all characters with category 'Mn' (non-spacing marks), effectively removing the accents. Finally, it normalizes again with NFC to combine any remaining characters back into composed form, ensuring consistent Unicode representation.

Common mistakes

Using NFKD instead of NFD, which also strips compatibility characters and might change symbols unexpectedly
Forgetting to filter out the 'Mn' category properly, leaving leftover combining marks
Not applying NFC after filtering, which can result in inconsistencies in output

Variations

Use the popular `unidecode` library for more aggressive transliteration, including converting characters like 'ß' to 'ss'
Use `str.maketrans` with a static mapping table for small, fixed sets of accented characters

Real-world use cases

Normalizing user-generated content for search indexing so queries match regardless of accent usage
Creating slugified URLs from titles that may contain accented characters
Preprocessing text data for NLP models that expect ASCII-only input

Normalize unicode accents to ASCII in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Strings & text

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Strings & text

Keep learning

Tutorials

Quizzes