Reading a file with read() gives your program a straightforward model: ask for bytes, receive a bytes object, and let Python manage the buffer. That is often the right choice.
Some workloads are different. A program may need to inspect small regions scattered across a large file, search the same file repeatedly, or pass file-backed bytes to APIs that understand the buffer protocol. Repeated seek() and read() calls can work, but they make every access an explicit file operation in your code.
Python’s mmap module offers another model. A memory mapping asks the operating system to associate a region of a file with a region of the process’s virtual address space. Python then exposes that mapping as an object that behaves partly like a byte array and partly like a file.
The useful mental model is not “the whole file is loaded into RAM.” The mapping creates an addressable view of the file. The operating system decides when mapped pages need to be brought into memory and may reuse cached file pages. Physical-memory use therefore depends on which parts you touch and on operating-system behavior.
This article focuses on the practical consequences: how to map a file safely, read arbitrary regions, search without manually managing a file position, choose access modes, avoid accidental copies, and recognize cases where ordinary file I/O is simpler.
Start with a read-only mapping of a non-empty file
Suppose a binary data file starts with a fixed header followed by records. You want to inspect bytes at known offsets without repeatedly changing a shared file position.
The smallest useful mapping is:
import mmap
with open("events.bin", "rb") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
magic = mapped[0:4]
version = mapped[4]Here, length=0 asks Python to map the current file size. The exact constructor differs internally between Unix and Windows, but access=mmap.ACCESS_READ is available on both and makes the intent explicit.
The two expressions demonstrate different byte-oriented behaviors:
mapped[0:4]returns abytesobject containing that slice;mapped[4]returns one byte as an integer, like indexing abytesorbytearrayobject.
Use read-only access when the program only needs to inspect data. It prevents accidental assignment through the mapping.
An empty file needs separate handling. Mapping a zero-length file is not portable, so check the size before creating a whole-file mapping if empty input is valid in your application.
import mmap
import os
with open("events.bin", "rb") as file:
size = os.fstat(file.fileno()).st_size
if size == 0:
data = b""
else:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
data = mapped[:16]That branch also makes the application’s empty-file behavior explicit instead of relying on platform-specific mapping errors.
Think in offsets instead of a mutable cursor
A normal file object has a current position. Calling read() advances it, while seek() changes it.
An mmap object also supports read(), seek(), and tell(), but random-access code often becomes easier to reason about when it uses indexes and slices instead.
Imagine a file format whose first eight bytes contain two unsigned 32-bit offsets:
bytes 0..3 -> start of name
bytes 4..7 -> start of payloadYou can decode the header and then address each region directly:
import mmap
import struct
with open("record.bin", "rb") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
name_offset, payload_offset = struct.unpack_from(">II", mapped, 0)
name_end = mapped.find(b"\0", name_offset)
if name_end == -1:
raise ValueError("record name is not null-terminated")
name = mapped[name_offset:name_end]
payload = mapped[payload_offset:]struct.unpack_from() can read from an object that supports the buffer protocol at a specified offset. The code does not have to reposition the mapping before each field.
This style is useful when offsets are part of the file format itself. The code mirrors the format: validate an offset, then inspect the bytes at that offset.
It does not remove the need for bounds checking. A corrupt or untrusted file can contain offsets beyond the mapping or offsets in an unexpected order.
Validate ranges before slicing structured data
Python slices are intentionally forgiving. A slice whose end exceeds the object length is clipped instead of raising an exception.
That behavior is convenient for ordinary sequences but dangerous when a binary format promises an exact-size field.
This code can silently accept truncated input:
payload = mapped[offset:offset + expected_size]If the file ends early, payload is simply shorter than expected_size.
Validate the range first:
def checked_region(mapped, offset, size):
if offset < 0 or size < 0:
raise ValueError("offset and size must be non-negative")
if offset > len(mapped) or size > len(mapped) - offset:
raise ValueError("region extends beyond the mapped file")
return mapped[offset:offset + size]The subtraction form avoids relying on offset + size when porting the same reasoning to languages where integer overflow may matter. Python integers themselves do not overflow, but the validation remains clear: the start must be inside the mapping, and the requested size must fit in what remains.
Use exact range checks whenever short input is an error rather than a valid partial result.
Search mapped data without reading it all into one Python bytes object
An mmap object provides find() and rfind() methods.
For example, a program can find newline-delimited records starting at a known position:
start = 0
while start < len(mapped):
end = mapped.find(b"\n", start)
if end == -1:
line = mapped[start:]
break
line = mapped[start:end]
start = end + 1The mapping itself does not require a prior file.read() that materializes the whole file as one Python bytes object.
However, each slice such as mapped[start:end] creates a new bytes object for that slice. If you immediately decode or process small records, that copy is often acceptable and keeps ownership simple.
If you are working with large regions and an API can consume a buffer directly, a memoryview can avoid creating that slice copy.
Use memoryview when you need a borrowed view, not a copy
Create a memoryview over the mapping when downstream code can operate on a bytes-like view:
with open("events.bin", "rb") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
view = memoryview(mapped)
header = view[0:64]
try:
consume_header(header)
finally:
header.release()
view.release()Slicing a memoryview creates another view rather than a new bytes object containing the region’s data.
That changes the lifetime rules. A live memory view refers to the underlying mapping. Python will not let the mapping close while exported views still exist; attempting to do so raises BufferError.
This is why the example releases both the sliced header view and its parent view before the mmap context exits.
For small fields, ordinary mapped[start:end] slices are often simpler. Use memoryview when avoiding copies matters enough to justify tighter lifetime management.
Choose the access mode based on what writes should mean
The portable access argument gives three common choices.
ACCESS_READ makes modification through the mapping invalid
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
mapped[0] = 0The assignment raises TypeError because the mapping is read-only.
This mode is a good default for parsers, indexes, search tools, and other code whose contract is inspection only.
ACCESS_WRITE changes the underlying file
With a writable file descriptor, ACCESS_WRITE allows assignments to the mapping and those changes affect the underlying file.
with open("flags.bin", "r+b") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_WRITE) as mapped:
mapped[0:4] = b"DONE"
mapped.flush()flush() asks for changes in the mapping to be flushed back to the file. It is important when the program needs an explicit synchronization point rather than relying on eventual write-back or mapping destruction.
Do not confuse mmap.flush() with a complete durability guarantee for every storage stack. If a workflow requires crash-durable persistence, reason about the platform’s file-system and storage guarantees as well.
ACCESS_COPY gives a private copy-on-write view
ACCESS_COPY lets your process modify the mapped view without updating the underlying file.
with open("template.bin", "r+b") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_COPY) as mapped:
mapped[0:4] = b"TEST"
result = mapped[:8]The modified bytes are visible through mapped, but the file is not changed by those assignments.
This is useful for temporary transformations or experiments where file contents should remain untouched. It is not a substitute for explicitly saving the modified result when persistence is required.
Keep mapped-file size changes out of ordinary readers
A mapping describes a particular byte range. Code becomes harder to reason about if another component truncates or replaces the underlying file while that range is still mapped.
The consequences of changing a mapped file’s size are platform-sensitive and can include failed accesses or worse process-level behavior. A parser should therefore prefer a simple ownership rule:
open stable file
|
create mapping
|
read mapped range
|
close mapping
|
close fileIf the file is actively rewritten by another process, use a publication strategy that gives readers a stable object, such as writing a new file and then replacing a name according to the guarantees of the target platform.
Do not assume a memory mapping turns a changing file into an immutable snapshot. ACCESS_READ prevents your code from assigning through that mapping; it does not guarantee that no other actor can change the underlying file.
Partial mappings require aligned offsets
Mapping only one section of a very large file can be useful, but the mapping offset has an alignment constraint.
Python requires the offset argument to be a multiple of mmap.ALLOCATIONGRANULARITY.
So this may fail when record_offset is arbitrary:
mmap.mmap(
file.fileno(),
length=record_size,
access=mmap.ACCESS_READ,
offset=record_offset,
)A portable approach aligns the mapping start downward and then uses a relative offset inside the mapping:
import mmap
alignment = mmap.ALLOCATIONGRANULARITY
map_offset = (record_offset // alignment) * alignment
relative_offset = record_offset - map_offset
map_length = relative_offset + record_size
with mmap.mmap(
file.fileno(),
length=map_length,
access=mmap.ACCESS_READ,
offset=map_offset,
) as mapped:
record = mapped[relative_offset:relative_offset + record_size]You still need to validate that the requested region exists in the file before creating or consuming the mapping.
For many applications, mapping the whole stable file with length=0 is simpler. Partial mapping is most useful when the file is very large, only a bounded window is needed, or address-space constraints matter.
Memory mapping changes the cost model, not the amount of data you ultimately touch
It is tempting to describe mmap as automatically faster than ordinary file I/O. That claim is too broad.
A memory mapping can reduce application-level copying and make repeated or random access convenient. It can also let the operating system manage paging and file caching without your code maintaining a large explicit buffer.
But if your algorithm scans every byte once in order, a buffered file loop may already be efficient and simpler:
with open("events.log", "rb") as file:
for line in file:
process(line)The mapped version still has to touch the data pages needed by the algorithm. Page faults, storage latency, cache behavior, and access pattern still matter.
Measure the workload that matters before choosing mmap primarily for speed.
A strong reason to use it is often access shape, not a generic performance promise: the program needs direct byte offsets, repeated searching, buffer-compatible access, or a convenient view over a stable file.
Understand the resource lifetime
An mmap object owns an operating-system mapping and should be closed deliberately.
Using nested context managers keeps the lifetime visible:
with open("events.bin", "rb") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
inspect(mapped)Closing the mmap does not itself close the original Python file object. Keeping both context managers makes that ownership explicit and portable.
Do not return a memoryview into a mapping that is about to close:
def unsafe_header(path):
with open(path, "rb") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
return memoryview(mapped)[:64]The returned view needs the mapping to remain alive, so the function’s ownership model is wrong. In fact, the mapping’s attempt to close while the exported view is still alive can raise BufferError.
Either return copied bytes:
def read_header(path):
with open(path, "rb") as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mapped:
return mapped[:64]or design an object whose lifetime clearly owns both the mapping and any views handed to callers.
Common mistakes
Mapping every file just because it is large
Large size alone is not enough reason. Sequential streaming can be simpler and keeps memory usage naturally bounded at the application level.
Assuming slices are zero-copy
Slicing an mmap produces bytes. Use memoryview when a borrowed zero-copy-style view is required, and manage its lifetime carefully.
Ignoring short or corrupt files
Slices clip silently. Validate offsets and exact region sizes when parsing structured data.
Using writable access for a reader
A parser usually gains nothing from writable access and loses a useful protection against accidental modification.
Treating ACCESS_COPY as persistence
Copy-on-write changes the mapped view, not the underlying file.
Changing the file size while readers still use a mapping
Keep the mapped file stable for the mapping’s lifetime unless your design explicitly handles the platform-specific consequences.
When mmap is a good fit
Consider mmap when a stable file is non-empty and your program benefits from direct byte addressing, repeated searches, random access to many regions, or passing file-backed data through the buffer protocol.
Prefer ordinary buffered file I/O when the program reads data once from beginning to end, handles streams that are not seekable files, processes empty files frequently, or does not benefit from byte-level random access.
The two approaches are not competitors in every situation. A program can use buffered streaming for ingestion and memory mapping for a later read-mostly index file whose offsets are known.
Conclusion
Python’s mmap is most useful when you change the way you think about a file: instead of repeatedly moving a cursor and requesting reads, you work with a stable byte-addressable region.
That model makes random access, offset-based parsing, searching, and buffer-oriented APIs convenient. It also introduces responsibilities that ordinary read() can hide: the file must remain suitable for mapping, offsets need careful validation, slices can copy, exported views extend the mapping’s lifetime, and writable modes have different persistence semantics.
Use memory mapping because its access model matches the problem. When the workload is simple sequential I/O, the simpler file iterator or buffered read() path is often the better engineering choice.