How to Handle Missing Values in a CSV Numeric Column in Python

Clean missing entries in a CSV numeric column by filling them with the mean, median, a custom value, or dropping rows.

Easy Python 3.9+ Aug 9, 2026 Files & data 13 views 0 copies

Python code

46 lines
Python 3.9+
import csv
from pathlib import Path
import statistics

def clean_csv_numeric(input_path: str, output_path: str, column: str, strategy: str = "mean") -> None:
    """
    Handles missing values in a numeric column of a CSV file.
    Strategies: 'mean', 'median', 'drop', or 'fill' with a specified value.
    """
    rows = list(csv.DictReader(Path(input_path).open(newline="")))
    values = []
    for row in rows:
        if row[column] not in ("", None):
            try:
                values.append(float(row[column]))
            except ValueError:
                pass
    
    if strategy == "mean":
        fill_value = statistics.mean(values) if values else 0.0
    elif strategy == "median":
        fill_value = statistics.median(values) if values else 0.0
    elif strategy == "drop":
        rows = [r for r in rows if r[column] not in ("", None)]
        fill_value = None
    else:  # custom fill value passed as strategy (e.g., "0" or "10")
        fill_value = float(strategy)
    
    if fill_value is not None:
        for row in rows:
            if row[column] in ("", None):
                row[column] = f"{fill_value:.2f}"
    
    fieldnames = rows[0].keys() if rows else []
    with Path(output_path).open("w", newline="") as f:
        writer = csv.DictWriter(f, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(rows)

if __name__ == "__main__":
    sample = "sample_data.csv"
    with Path(sample).open("w", newline="") as f:
        f.write("name,age,salary\nAlice,30,50000\nBob,,60000\nCharlie,25,\nDiana,35,70000\n")
    
    clean_csv_numeric(sample, "cleaned.csv", column="salary", strategy="mean")
    print(Path("cleaned.csv").read_text())

Output

stdout
name,age,salary
Alice,30,50000.00
Bob,,60000.00
Charlie,25,60000.00
Diana,35,70000.00

How it works

The function clean_csv_numeric reads the CSV with csv.DictReader, which represents each row as a dictionary keyed by column names. It first collects all non-empty, numeric values from the target column (ignoring blanks and non-parseable strings) to compute the fill statistic. Depending on the chosen strategy, it either fills missing cells with a formatted float (two decimal places) or removes rows that have missing values. Writing with csv.DictWriter preserves the header and column order, and the fieldnames are taken from the first row's keys. The stdlib modules csv, pathlib, and statistics handle everything without third-party dependencies.

Common mistakes

  • Comparing `row[column]` to `None` when CSV fields are always strings, so blanks are `''` not `None`.
  • Forgetting to convert the custom fill value (a string) to float before assigning to the cell.
  • Assuming all values in the column parse as floats, causing silent skips when using `try/except`.

Variations

  1. Use `pandas.DataFrame.fillna(method='ffill')` for a more feature-rich alternative with `pip install pandas`.
  2. Call `csv.DictReader` directly on an open file object instead of wrapping with `Path.open()`.

Real-world use cases

  • Preprocessing sensor logs where occasional dropped readings leave blank cells before feeding data into a regression model.
  • Cleaning exported financial reports that contain missing revenue values so totals and averages become accurate.
  • Standardizing user-uploaded spreadsheets in a web app to fill gaps in age or salary columns before processing.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Files & data

Related tutorials and quizzes for this topic.