ETL in Python: Extract CSV, Transform Dicts, Load JSON
Build a simple ETL pipeline that reads a CSV, normalizes keys and converts price to float, then writes structured JSON.
Python code
27 linesimport csv
import json
from pathlib import Path
def etl_csv_to_json(csv_path: str, json_path: str) -> None:
"""Extract CSV, transform rows to dicts, load to JSON."""
with open(csv_path, mode='r', newline='', encoding='utf-8') as f:
reader = csv.DictReader(f)
records = list(reader)
# Transform: normalize keys and convert price to float
transformed = []
for rec in records:
cleaned = {k.strip().lower(): v.strip() for k, v in rec.items()}
cleaned['price'] = float(cleaned['price'])
transformed.append(cleaned)
# Load: write JSON
with open(json_path, 'w', encoding='utf-8') as f:
json.dump(transformed, f, indent=2)
if __name__ == "__main__":
src = Path("data.csv")
dest = Path("output.json")
src.write_text("Name,Price\nApple,1.25\nBanana,0.75\n")
etl_csv_to_json(str(src), str(dest))
print(dest.read_text())
Output
[
{
"name": "Apple",
"price": 1.25
},
{
"name": "Banana",
"price": 0.75
}
]
How it works
The csv.DictReader reads each row into a native dictionary, using the header row as keys. The transform step strips whitespace, lowercases keys, and casts the price field to float to keep data types clean. json.dump writes the list of dicts to disk with indentation for readability. Using with open handles file lifecycle safely, and __main__ guard makes this runnable as a script.
Common mistakes
- Forgetting `newline=''` when opening CSV files in Python 3
- Hardcoding paths instead of using `Path` or checking file existence
- Assuming all CSV fields are strings without converting numeric types
Variations
- Use `pandas.read_csv` then `.to_json` for larger datasets
- Add filtering or deduplication logic in the transform step
Real-world use cases
- Moving exported CRM reports into a JSON API-consumable format for dashboards.
- Normalizing vendor CSV drops before loading them into a document database.
- Converting legacy flat-file exports into structured JSON for warehouse ingestion.
Sponsored
More from Data pipelines & processing
- Add a UUID Surrogate Key to Each Row in a CSV with Python easy
- Attach Source File Metadata to Records in Python easy
- Build a Python Utility That Detects Duplicate Records Across Multiple Excel Sheets medium
- Check Null Rate Threshold in PySpark DataFrame medium
- Count Records Processed per Category in Python easy
- Create Data Helper Functions in Python for Beginners easy
Keep learning
Related tutorials and quizzes for this topic.