How to Count Word Frequencies in Python with Counter and Sets

This code processes a text string by lowercasing, splitting into words, counting frequencies with Counter, and extracting unique and sorted word lists using sets.

Easy Python 3.9+ Aug 9, 2026 Dictionaries & sets 12 views 0 copies

counter sets text-processing frequency collections

Python code

22 lines

Python 3.9+

from collections import Counter

def process_text(text):
    words = text.lower().split()
    word_counts = Counter(words)
    unique_words = set(words)
    sorted_words = sorted(unique_words)
    
    return {
        "total_words": len(words),
        "unique_words": len(unique_words),
        "word_frequencies": dict(word_counts.most_common(5)),
        "sorted_unique": sorted_words[:5]
    }

if __name__ == "__main__":
    sample = "The cat sat on the mat. The dog sat on the mat too."
    result = process_text(sample)
    print(f"Total words: {result['total_words']}")
    print(f"Unique words: {result['unique_words']}")
    print(f"Top 5 frequencies: {result['word_frequencies']}")
    print(f"First 5 sorted unique words: {result['sorted_unique']}")

Output

stdout

Total words: 10
Unique words: 8
Top 5 frequencies: {'the': 3, 'sat': 2, 'on': 2, 'cat': 1, 'mat.': 1}
First 5 sorted unique words: ['cat', 'dog', 'mat.', 'on', 'sat']

How it works

The collections.Counter class provides an efficient way to count hashable objects like words. By splitting the lowercased text, punctuation like periods remain attached to words, which is a common preprocessing caveat. Using set(words) removes duplicates to get unique words, and sorted() orders them alphabetically. The most_common(5) method returns the top five frequent items, which we convert to a dict for easy display.

Common mistakes

Forgetting to lowercase text before splitting, causing case-sensitive duplicates
Not cleaning punctuation, so 'mat.' and 'mat' are counted as separate words
Using `len(set(words))` instead of a Counter when frequency counts are also needed

Variations

Use `re.findall(r'\w+', text)` to extract only word characters, ignoring punctuation
Replace Counter with `collections.defaultdict(int)` to manually increment counts

Real-world use cases

Analyzing log files to identify the most frequent error messages for alerting.
Building a keyword frequency report for SEO content optimization.
Creating a word cloud generator that weighs words by their occurrence count.

How to Count Word Frequencies in Python with Counter and Sets

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Dictionaries & sets

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Dictionaries & sets

Keep learning

Tutorials

Quizzes