How to detect anomalies in a column using z-score in Python

Detect outliers in a list of numbers using z-score statistics, flagging values that deviate significantly from the mean.

Easy Python 3.9+ Aug 9, 2026 Data pipelines & processing 14 views 0 copies

anomaly-detection z-score statistics outliers data-processing

Python code

30 lines

Python 3.9+

import random

def z_score_anomaly_detection(data, threshold=2.0):
    """
    Detect anomalies in a list of numbers using z-score.
    """
    mean = sum(data) / len(data)
    variance = sum((x - mean) ** 2 for x in data) / len(data)
    std_dev = variance ** 0.5
    
    if std_dev == 0:
        return []
    
    anomalies = []
    z_scores = []
    
    for x in data:
        z = (x - mean) / std_dev
        z_scores.append(z)
        if abs(z) > threshold:
            anomalies.append(x)
    
    return anomalies, z_scores

if __name__ == "__main__":
    random.seed(42)
    data = [random.gauss(50, 5) for _ in range(10)] + [100, 2]  # Add 2 outliers
    anomalies, z_scores = z_score_anomaly_detection(data, threshold=2.5)
    print(f"Data: {data}")
    print(f"Anomalies: {anomalies}")

Output

stdout

Data: [49.40377376192539, 48.919867823371836, 55.50573360958126, 48.541655072122766, 51.69598929230049, 48.93805841267825, 48.47398445876512, 51.756175856448006, 49.93434094076763, 51.468977016594466, 100, 2]
Anomalies: [100, 2]

How it works

The z-score method standardizes each value by subtracting the mean and dividing by the standard deviation. Values with an absolute z-score above a threshold (e.g., 2.5) are considered anomalies. The code computes population variance and standard deviation directly from the list, making it self-contained. When the standard deviation is zero, no anomalies exist, and the code returns an empty list. The function returns both the anomaly list and the full z-scores for further inspection.

Common mistakes

Using sample standard deviation (n-1) when the data represents the full population, slightly skewing results.
Choosing a threshold that is too strict or too lenient, causing false positives or missed outliers.
Not handling the case where standard deviation is zero, leading to division by zero errors.
Applying z-score to non-normal distributions where the method is less appropriate.

Variations

Use scipy.stats.zscore for a vectorized approach on numpy arrays.
Implement with the statistics module's stdev and mean for cleaner code.

Real-world use cases

Flagging unusual transactions in financial fraud detection systems.
Monitoring server metrics to alert on spikes in CPU or memory usage.
Cleaning sensor data before feeding it into a machine learning pipeline.

How to detect anomalies in a column using z-score in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Data pipelines & processing

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Data pipelines & processing

Keep learning

Tutorials

Quizzes