Generate a Monthly Report CSV from Log Files in Python
Reads a CSV log file, filters events by a given month, aggregates daily event counts and revenue, and writes a summarized monthly report to a new CSV.
Python code
30 linesimport csv
from collections import defaultdict
from datetime import datetime
def generate_monthly_report(log_file: str, month: str, output_file: str) -> None:
events_by_date = defaultdict(int)
revenue_by_date = defaultdict(float)
with open(log_file, 'r') as f:
for line in f:
date_str, _, revenue_str = line.strip().split(',')
if date_str.startswith(month):
events_by_date[date_str] += 1
revenue_by_date[date_str] += float(revenue_str)
with open(output_file, 'w', newline='') as f:
writer = csv.writer(f)
writer.writerow(['date', 'events', 'revenue'])
for date in sorted(events_by_date):
writer.writerow([date, events_by_date[date], round(revenue_by_date[date], 2)])
if __name__ == "__main__":
sample_logs = "2024-03-01,login,10.50\n2024-03-01,purchase,25.00\n2024-03-02,purchase,12.75\n2024-04-01,login,5.00"
with open("logs.txt", 'w') as f:
f.write(sample_logs)
generate_monthly_report("logs.txt", "2024-03", "march_report.csv")
with open("march_report.csv") as f:
print(f.read())
Output
date,events,revenue
2024-03-01,2,35.5
2024-03-02,1,12.75
How it works
This script uses defaultdict to pre-initialize counters for each date, avoiding the need to check key existence. The csv module ensures proper quoting and escaping when writing the output file. Filtering is done by checking if the date string starts with the month prefix, which works because dates are formatted as YYYY-MM-DD. Sorted iteration ensures the report is chronological. The script demonstrates a classic batch pattern: read, aggregate, and write results.
Common mistakes
- Forgetting to open the output file with `newline=''` on Windows, causing extra blank lines.
- Assuming all lines have the same number of fields without validation.
- Not handling the case where a month has no events, resulting in an empty report.
Variations
- Use `csv.DictReader` and `csv.DictWriter` for more readable code when columns have names.
- Filter with `datetime.strptime` to parse dates and compare month values more explicitly.
Real-world use cases
- Generating per-month sales reports from raw transaction logs in payment systems.
- Creating usage reports for e-commerce platforms by aggregating user actions logged daily.
- Producing monthly Kafka/database export summaries for accounting and auditing teams.
Sponsored
More from Automation & scripting
- Aggregate Log Errors Count by Hour in Python easy
- Automate Tweeting New Blog Posts in Python easy
- Automatically Clean Temporary Files from Applications Using Python medium
- Automatically Download the Latest Software Release from GitHub with Python medium
- Automatically Generate Charts from CSV Files with One Command medium
- Automatically Generate Hardware Inventory Reports in Python easy
Keep learning
Related tutorials and quizzes for this topic.