How to Parse NDJSON Lines into a List in Python
Reads a JSON-lines (NDJSON) file line by line and converts each non-empty line into a Python object, returning a list.
Python code
20 linesimport json
from pathlib import Path
def parse_ndjson(file_path: str) -> list:
data = []
with Path(file_path).open("r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if line:
data.append(json.loads(line))
return data
if __name__ == "__main__":
sample = Path("sample.ndjson")
sample.write_text('{"name": "Alice", "age": 30}\n{"name": "Bob", "age": 25}\n')
result = parse_ndjson(sample)
print(result)
sample.unlink()
Output
[{'name': 'Alice', 'age': 30}, {'name': 'Bob', 'age': 25}]
How it works
Path.open() opens the file with a specified encoding and returns a file object that iterates line by line. Each line is stripped to remove newline and surrounding whitespace, and empty lines are skipped. json.loads() parses one JSON object per line, appending it to the data list. Since the file is opened in a context manager, it is automatically closed after the loop finishes. The result is a list of dictionaries, one per JSON object in the file.
Common mistakes
- Using `json.load()` instead of `json.loads()` — `json.load()` expects a file object, not a string.
- Not stripping the line before parsing, which can cause errors with trailing newlines.
- Forgetting to skip empty lines, which causes `json.loads()` to fail on blank lines.
- Missing the `encoding` parameter when the file contains non-ASCII characters.
Variations
- Use `with open(file_path) as f:` instead of `Path(file_path).open()` for a more conventional approach.
- Use a list comprehension with a generator expression: `[json.loads(line) for line in Path(file_path).read_text().splitlines() if line]`.
Real-world use cases
- Loading bulk event logs or telemetry data exported in NDJSON format into memory for analysis.
- Reading streaming data dump files from APIs like Elasticsearch or AWS S3 that use JSON lines for batching.
- Preprocessing user activity logs written in NDJSON by data pipelines before inserting into a database.
Sponsored
More from Files & data
- Append a Line to a Log File in Python easy
- Audit File Permissions Across a Project in Python easy
- Automatically Detect Corrupted Files Using SHA-256 Checksums in Python easy
- Automatically Highlight Data Validation Errors Inside Excel Files in Python easy
- Build a Command-Line To-Do List Application with Data Persistence in Python easy
- Build a File Index by Relative Path Hash Map in Python easy
Keep learning
Related tutorials and quizzes for this topic.