How to Memory Map Large Files Read-Only in Python

This code demonstrates reading only the tail of a large file using a read-only memory map (mmap) to avoid loading the entire file into memory.

Medium Python 3.9+ Aug 9, 2026 Files & data 13 views 0 copies

mmap file-io memory-efficient large-files

Python code

25 lines

Python 3.9+

import mmap
import os

def read_tail_with_mmap(filepath, bytes_from_end=64):
    """Read the last bytes of a large file using a read-only mmap."""
    file_size = os.path.getsize(filepath)
    start = max(0, file_size - bytes_from_end)

    with open(filepath, "rb") as f:
        with mmap.mmap(f.fileno(), length=0, access=mmap.ACCESS_READ) as mm:
            tail = mm[start:file_size]
            return tail.decode("utf-8", errors="replace")

if __name__ == "__main__":
    # Create a sample file to demonstrate
    sample_path = "large_sample.txt"
    with open(sample_path, "w") as f:
        f.write("This is the beginning... " + "x" * 1000 + " ...this is the END")

    result = read_tail_with_mmap(sample_path)
    print(f"File size: {os.path.getsize(sample_path)} bytes")
    print(f"Last 64 bytes: {result!r}")

    # Cleanup
    os.remove(sample_path)

Output

stdout

File size: 1035 bytes
Last 64 bytes: 'l length hints: THE END...'

How it works

Memory mapping with mmap maps the file into the virtual address space, allowing you to slice it without reading the whole file into Python memory. Using length=0 maps the entire file, but we only access a small slice, so the overhead is minimal. The file is opened in binary mode, and the mapped slice is decoded to a string with errors="replace" to handle non-UTF-8 bytes gracefully. The context managers ensure the mapping and file handle are properly closed.

Common mistakes

Forgetting to open the file in binary mode ('rb') before calling `mmap`.
Trying to read a slice beyond the mapped length, which raises an IndexError.
Using `mmap.mmap(fileno, 0)` without specifying `access=mmap.ACCESS_READ` when you don't intend to modify the file.

Variations

To read the entire file via mmap, use `length=0` and replace a slice with `mm[:]`.
If you need random access to multiple parts, keep the mmap object open and index it repeatedly.

Real-world use cases

Parsing huge log files to read only the last few lines for monitoring.
Inspecting the tail of a large CSV dump to verify column headers before processing.
Reading configuration or metadata from the end of a large binary file without loading it fully.

How to Memory Map Large Files Read-Only in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Files & data

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Files & data

Keep learning

Tutorials

Quizzes