Join two CSV files on shared key column in Python
Merge rows from two CSV files by a common key column, outputting combined records to a new file.
Python code
43 linesimport csv
def join_csv(file1, file2, key, output="joined.csv"):
# Read first CSV into dict keyed by the join column
with open(file1, newline="") as f1:
reader1 = csv.DictReader(f1)
data1 = {row[key]: row for row in reader1}
# Read second CSV and merge matching rows
with open(file2, newline="") as f2:
reader2 = csv.DictReader(f2)
fieldnames = list(data1[next(iter(data1))].keys()) + [
col for col in reader2.fieldnames if col != key
]
with open(output, "w", newline="") as out:
writer = csv.DictWriter(out, fieldnames=fieldnames)
writer.writeheader()
for row in reader2:
if row[key] in data1:
merged = {**data1[row[key]], **{
k: v for k, v in row.items() if k != key
}}
writer.writerow(merged)
if __name__ == "__main__":
# Create sample CSVs inline for demonstration
with open("customers.csv", "w", newline="") as f:
w = csv.writer(f)
w.writerow(["id", "name"])
w.writerow(["1", "Alice"])
w.writerow(["2", "Bob"])
w.writerow(["3", "Carol"])
with open("orders.csv", "w", newline="") as f:
w = csv.writer(f)
w.writerow(["id", "item"])
w.writerow(["1", "apple"])
w.writerow(["1", "banana"])
w.writerow(["2", "cherry"])
join_csv("customers.csv", "orders.csv", key="id")
with open("joined.csv") as f:
print(f.read())
Output
id,name,item
1,Alice,apple
1,Alice,banana
2,Bob,cherry
How it works
This code mimics a SQL INNER JOIN: rows from the second file are kept only if their key exists in the first file. The first CSV is loaded into a dictionary mapping the key column to its row, giving O(1) lookups. When iterating the second file, matching rows are merged by combining the first row with non-key columns from the second row. The output uses a csv.DictWriter, whose fieldnames are derived from the first file's columns plus the second file's non-key columns, preserving column order.
Common mistakes
- Assuming `next(iter(data1))` works when file1 is empty — it raises `StopIteration`. Guard against empty first file.
- Duplicate keys in the first file: only the last row is kept, silently dropping earlier matches.
- Column names that differ between files can cause missing values or `ValueError` if not handled explicitly.
Variations
- Use pandas: `pd.merge(pd.read_csv(file1), pd.read_csv(file2), on=key)` for a one-liner.
- Include non-matching rows (LEFT/OUTER join) by iterating data1 and looking up file2.
Real-world use cases
- Merging customer profiles with their latest order details for a reporting export.
- Combining user information from an HR export with badge access logs for onboarding audits.
- Enriching product inventory files with supplier pricing data before a pricing update.
Sponsored
More from Files & data
- Append a Line to a Log File in Python easy
- Audit File Permissions Across a Project in Python easy
- Automatically Detect Corrupted Files Using SHA-256 Checksums in Python easy
- Automatically Highlight Data Validation Errors Inside Excel Files in Python easy
- Build a Command-Line To-Do List Application with Data Persistence in Python easy
- Build a File Index by Relative Path Hash Map in Python easy
Keep learning
Related tutorials and quizzes for this topic.