Text Processor with Dictionaries and Sets in Python

Build a simple text processor that counts word frequencies with a dictionary and tracks unique words with a set.

Easy Python 3.9+ Aug 9, 2026 Dictionaries & sets 12 views 0 copies

Python code

29 lines
Python 3.9+
def analyze_text(text):
    words = text.lower().split()
    word_freq = {}
    unique_words = set()
    
    for word in words:
        clean_word = word.strip('.,!?;:')
        if clean_word:
            word_freq[clean_word] = word_freq.get(clean_word, 0) + 1
            unique_words.add(clean_word)
    
    return word_freq, unique_words


def get_top_words(word_freq, n=3):
    sorted_words = sorted(word_freq.items(), key=lambda x: x[1], reverse=True)
    return sorted_words[:n]


if __name__ == "__main__":
    sample_text = "The quick brown fox jumps over the lazy dog. The dog sleeps!"
    word_freq, unique_words = analyze_text(sample_text)
    
    print("Word frequency dictionary:")
    print(word_freq)
    print("\nUnique words set:")
    print(unique_words)
    print("\nTop 3 most frequent words:")
    print(get_top_words(word_freq))

Output

stdout
Word frequency dictionary:
{'the': 2, 'quick': 1, 'brown': 1, 'fox': 1, 'jumps': 1, 'over': 1, 'lazy': 1, 'dog': 2, 'sleeps': 1}

Unique words set:
{'quick', 'brown', 'jumps', 'lazy', 'over', 'fox', 'sleeps', 'dog', 'the'}

Top 3 most frequent words:
[('the', 2), ('dog', 2)]

How it works

The function analyze_text converts the input to lowercase and splits on whitespace, then cleans punctuation from each word. It uses a dictionary word_freq to count occurrences with .get() and a set unique_words to track distinct clean words. The get_top_words helper sorts dictionary items by frequency in descending order and returns the top n pairs. The main block runs a sample text and prints the frequency map, the set of unique words, and the top frequent words.

Common mistakes

  • Forgetting to strip punctuation before counting
  • Using a list instead of a set for unique words, causing duplicates
  • Not using `.get()` to avoid KeyError when incrementing counts
  • Ignoring casing, so 'The' and 'the' are counted separately

Variations

  1. Use `collections.Counter` to count frequencies in one line: `Counter(words)`
  2. Implement the entire pipeline with list comprehension and `set()` for unique words

Real-world use cases

  • Building a word cloud generator that needs frequency analysis of user text.
  • Analyzing customer feedback in surveys to identify frequently mentioned topics.
  • Preprocessing text for a search engine or NLP pipeline to index unique terms.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Dictionaries & sets

Related tutorials and quizzes for this topic.