Updating a small file looks simple: open it, truncate it, write the new contents, and close it. That works when nothing interrupts the write.

The failure mode appears when a process crashes, the machine loses power, or another process reads the file while it is being rewritten. A reader can observe an empty or partially written file, and a crash can leave the pathname referring to incomplete data.

For configuration files, state snapshots, generated metadata, and similar single-file updates, a better pattern is to write a complete replacement beside the original file and then rename it into place.

The important mental model is that atomic visibility and crash durability are different guarantees:

  • rename() can make the switch from old file to new file atomic for pathname lookup.
  • fsync() is needed when the update must survive a crash after the system call reports success.

Understanding that distinction prevents a common mistake: using an atomic rename and assuming the new data is therefore durable.

Why rewriting a file in place can expose partial state

Consider a program that updates settings.json like this:

from pathlib import Path

Path("settings.json").write_text(
    '{"enabled": true, "workers": 8}\n',
    encoding="utf-8",
)

The high-level call is convenient, but an in-place replacement can involve truncating the existing file before all new bytes have been written.

If another process opens the file during that interval, it can observe the intermediate state. A crash can also happen after truncation but before the complete replacement reaches storage.

The problem is not JSON-specific. The same risk applies to any format whose readers expect one complete version of a file.

Treat the update as publishing a new file

Instead of modifying the visible file incrementally, build the next version under a temporary name in the same directory:

settings.json
.settings.json.abcd.tmp

Then follow this sequence:

1. write the complete temporary file
2. fsync the temporary file
3. rename the temporary file over settings.json
4. fsync the containing directory

Each step protects a different boundary.

Writing to a separate file keeps incomplete bytes away from readers of settings.json. Flushing that file asks the kernel to persist its data and required metadata. Renaming publishes the completed inode under the destination pathname. Flushing the directory persists the directory-entry change itself.

The temporary file must be on the same filesystem as the destination because a normal rename cannot atomically move a file across filesystems.

Start with atomic replacement

On Linux, rename() atomically replaces an existing destination pathname when the operation succeeds. A process looking up the destination does not pass through a moment where the pathname is absent merely because it is being replaced.

Python exposes the same operation through os.replace():

import os

os.replace("settings.json.tmp", "settings.json")

This gives readers a useful property: they see the old file or the new file through that pathname, rather than a partially copied transition between the two.

Atomic replacement does not mean every reader instantly switches to the new inode. A process that already has the old file open can continue reading through its existing file descriptor. Rename changes directory entries; it does not invalidate open descriptors.

Atomic replacement also does not serialize multiple writers. If two writers independently create complete temporary files and replace the same destination, whichever rename happens last determines what later pathname lookups see.

Add fsync when crash durability matters

Before publishing the temporary file, flush it:

file.flush()
os.fsync(file.fileno())

flush() moves Python’s buffered output to the operating system. It does not by itself require the operating system to persist those bytes to durable storage.

os.fsync() asks the operating system to synchronize the file’s modified data and associated metadata needed by the filesystem.

The order matters. Renaming an unflushed temporary file can make the new pathname visible while its newly written contents are still only pending in volatile caches.

After the rename, there is one more durability boundary: the directory entry.

Linux documents that syncing a file does not necessarily sync the directory entry that names it. If the rename itself must survive a crash, open the containing directory and sync that descriptor too.

Put the sequence into one helper

Here is a small Linux-specific helper for replacing a regular file with byte content:

from pathlib import Path
import os
import tempfile


def replace_file_atomically(
    path: Path,
    data: bytes,
    mode: int = 0o600,
) -> None:
    path = Path(path)
    parent = path.parent

    fd, temp_name = tempfile.mkstemp(
        dir=parent,
        prefix=f".{path.name}.",
        suffix=".tmp",
    )
    temp_path = Path(temp_name)

    try:
        os.fchmod(fd, mode)

        with os.fdopen(fd, "wb") as file:
            file.write(data)
            file.flush()
            os.fsync(file.fileno())

        os.replace(temp_path, path)

        dir_fd = os.open(
            parent,
            os.O_RDONLY | os.O_DIRECTORY,
        )
        try:
            os.fsync(dir_fd)
        finally:
            os.close(dir_fd)

    except BaseException:
        try:
            temp_path.unlink()
        except FileNotFoundError:
            pass
        raise

The temporary file is created inside parent, so the final replacement stays on the same filesystem under normal directory layout.

mkstemp() also creates the temporary file without a filename-selection race. The returned descriptor is already open, so the program does not need to invent a name and then separately hope that another process has not claimed it.

The cleanup path removes an unpublished temporary file after an error. If os.replace() already succeeded, temp_path no longer exists under that name, so cleanup simply has nothing to remove.

Choose file metadata deliberately

Replacing a pathname with a newly created file also replaces the inode behind that pathname. The new file therefore does not automatically inherit every property of the old inode.

The helper above explicitly sets a mode:

os.fchmod(fd, 0o600)

That is appropriate for a private state file, but it may be wrong for a shared configuration file.

If ownership, permissions, extended attributes, access-control lists, or other metadata matter, define how they should be handled before the rename. Blindly assuming that replacement preserves the old file’s metadata can create permission or security regressions.

There is also a concurrency issue when copying metadata from an existing destination: another writer can replace that destination between inspection and publication. If multiple writers need coordination, atomic rename alone is not a complete concurrency-control protocol.

Do not put the temporary file in /tmp by default

This looks convenient:

tempfile.mkstemp(dir="/tmp")

but it can break the central guarantee.

If /tmp and the destination are on different mounted filesystems, os.replace() can fail because rename does not provide a cross-filesystem move.

Keeping the temporary file in the destination directory avoids that problem and also makes the directory durability step unambiguous.

If the destination directory is not writable by the process, you need a different design rather than silently falling back to a cross-filesystem copy.

Atomic rename does not make a multi-file transaction

Suppose an update changes both:

users.json
index.json

Replacing each file atomically still leaves an interval where one pathname refers to the new version and the other refers to the old version.

If readers require both files to change as one logical unit, a single-file rename is not enough.

Common alternatives include storing the state in one file, writing versioned directories and atomically switching one pointer, or using a transactional storage system such as a database. The right choice depends on the consistency boundary readers require.

Handle fsync failures as real write failures

fsync() can fail. For example, storage errors that occurred during delayed writeback can surface when the program requests synchronization.

Do not log an fsync() error and then report the update as safely persisted. If durability is part of the operation’s contract, a synchronization failure means that contract was not established.

There is an awkward boundary after os.replace() succeeds but the directory fsync() fails: the new file may already be visible even though the program cannot confirm that the directory update is durable.

That uncertainty is unavoidable at this layer. The application should report failure accurately and avoid pretending it knows whether a subsequent crash would preserve the rename.

Know what the pattern does and does not guarantee

For a regular local Linux filesystem that implements the documented operations normally, the pattern provides two useful properties.

First, readers that open the destination by pathname see a complete old version or a complete new version around the rename, rather than the temporary file’s partially written contents.

Second, syncing the new file before rename and the directory after rename requests persistence for both the new file state and the pathname update.

These guarantees still have boundaries. Filesystem implementations, network filesystems, storage hardware, mount options, and I/O errors can affect durability behavior. fsync() should be treated as an operation that can fail, not as ceremonial syntax.

The pattern also does not protect application-level invariants across multiple files, prevent competing writers, or preserve old inode metadata automatically.

When a simpler write is enough

Not every file needs this machinery.

A direct write is often sufficient for disposable cache files, temporary outputs, or data that can be regenerated after corruption. Adding extra writes and synchronization calls can increase I/O latency, especially when updates happen frequently.

Use atomic replacement when readers must not observe partial content. Add the durability steps when acknowledging success means the update should survive a crash as far as the operating system and storage stack can guarantee.

That distinction keeps the design proportional to the actual failure you are trying to prevent.

Conclusion

Safe file replacement on Linux is not one operation but a sequence with separate responsibilities.

Write the next version to a temporary file in the destination directory. Sync that file when durability matters. Rename it over the destination to publish the completed version atomically. Then sync the directory when the rename itself must be crash-durable.

The key lesson is to keep visibility, durability, and concurrency separate in your mental model. Atomic rename solves the publication boundary. fsync() addresses persistence boundaries. Neither one, by itself, turns arbitrary file updates into a transaction.

Linux Filesystems File I/O Reliability