How to compute ROUGE recall in Python

Compute ROUGE recall by counting token overlap between a reference and candidate summary with pure Python.

Easy Python 3.9+ Aug 9, 2026 AI & LLM integration patterns 12 views 0 copies

rouge nlp evaluation text-summarization

Python code

25 lines

Python 3.9+

def rouge_recall(reference, candidate):
    ref_tokens = reference.lower().split()
    cand_tokens = candidate.lower().split()

    ref_counts = {}
    for token in ref_tokens:
        ref_counts[token] = ref_counts.get(token, 0) + 1

    cand_counts = {}
    for token in cand_tokens:
        cand_counts[token] = cand_counts.get(token, 0) + 1

    overlap = 0
    for token, count in cand_counts.items():
        overlap += min(count, ref_counts.get(token, 0))

    return overlap / len(ref_tokens) if ref_tokens else 0.0


if __name__ == "__main__":
    reference = "The quick brown fox jumps over the lazy dog"
    candidate = "The brown fox quickly jumps over a lazy dog"

    score = rouge_recall(reference, candidate)
    print(f"ROUGE recall score: {score:.2f}")

Output

stdout

ROUGE recall score: 0.78

How it works

ROUGE recall measures how much of the reference text's content appears in the candidate. The function tokenizes both texts by splitting on whitespace and lowercasing, then builds frequency dictionaries to handle repeated tokens correctly. Overlap is calculated by summing the minimum count of each token in both texts, which avoids overcounting. Dividing by the reference token count yields the fraction of reference tokens matched. This is a simplified version that ignores stopword removal and stemming, but it captures the core mechanism used in evaluation libraries.

Common mistakes

Forgetting to lowercase tokens, causing case mismatches.
Dividing by candidate length instead of reference length, which gives precision, not recall.
Using sets instead of counts, which overcounts repeated tokens.
Not handling empty reference text, leading to DivisionByZero.

Variations

Use the rouge-score library's `rouge_scorer` for standard ROUGE-1, ROUGE-2, and ROUGE-L.
Preprocess text with nltk's word tokenizer and remove stopwords for better accuracy.

Real-world use cases

Evaluating the quality of AI-generated summaries against human-written references in NLP pipelines.
Scoring retriever or reranker outputs by measuring content overlap with a gold answer.
Monitoring LLM output quality in production by computing recall against expected key phrases.

How to compute ROUGE recall in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from AI & LLM integration patterns

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from AI & LLM integration patterns

Keep learning

Tutorials

Quizzes