How to Split a Large File into Fixed-Size Parts in Python

Splits any binary or text file into multiple part files of a fixed byte size using Python's standard library.

Easy Python 3.9+ Aug 9, 2026 Files & data 14 views 0 copies

Python code

31 lines
Python 3.9+
import os
import math

def split_file(filepath, chunk_size_bytes):
    """Split a file into parts of fixed size (bytes). Creates part files in same directory."""
    filepath = os.path.abspath(filepath)
    basename = os.path.basename(filepath)
    file_size = os.path.getsize(filepath)
    num_parts = math.ceil(file_size / chunk_size_bytes)
    
    with open(filepath, 'rb') as f:
        for part_num in range(num_parts):
            chunk = f.read(chunk_size_bytes)
            part_filename = f"{basename}.part{part_num + 1:03d}"
            with open(part_filename, 'wb') as part_file:
                part_file.write(chunk)
            print(f"Created {part_filename} ({len(chunk)} bytes)")

if __name__ == "__main__":
    # Example: split a sample file into 16-byte chunks
    sample = "example_data.txt"
    with open(sample, 'w') as f:
        f.write("Hello world, this is a test file for splitting.")
    
    split_file(sample, 16)
    
    # Cleanup after demonstration
    for fname in os.listdir('.'):
        if fname.startswith(sample) and fname != sample:
            os.remove(fname)
    os.remove(sample)

Output

stdout
Created example_data.txt.part001 (16 bytes)
Created example_data.txt.part002 (16 bytes)
Created example_data.txt.part003 (11 bytes)

How it works

The function uses os.path.getsize to determine the file size and math.ceil to calculate the number of parts. Reading in binary mode with 'rb' ensures that the splitting works for both text and binary files without any encoding issues. Each chunk is read with f.read(chunk_size_bytes), which returns exactly that many bytes except for the last part, which contains the remaining data. Part files are named with a zero-padded index to maintain order, and the script cleans up after the demo by removing the created parts and the original sample file.

Common mistakes

  • Forgetting to use binary mode ('rb'/'wb') when splitting text files, leading to encoding errors
  • Not padding part numbers (e.g., part1, part2) which breaks alphabetical ordering for more than 9 parts
  • Assuming the last chunk is the same size; it is usually smaller
  • Overwriting existing part files without checking, which can lose data

Variations

  1. Use `shutil.copyfileobj` with a loop for memory-efficient large-file splitting
  2. Store parts in a subdirectory to keep the original directory clean

Real-world use cases

  • Splitting a large database dump into chunks small enough to upload to an object store with per-file size limits.
  • Breaking a log file into daily parts for easier archival and transfer between systems.
  • Segmenting a video or archive into pieces for distribution across multiple HTTP requests (e.g., resumable downloads).

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Files & data

Related tutorials and quizzes for this topic.