Apps Artificial Intelligence CSS DevOps Go JavaScript Laravel Linux MongoDB MySQL PHP Python Rust Svelte Vue

Atomic File Writes in Go: Prevent Partial and Corrupted Files

5 min read .
Published on
Atomic File Writes in Go: Prevent Partial and Corrupted Files

Writing a file with os.WriteFile is simple, but it is not always the safest choice for configuration files, generated metadata, caches, state files, or other data that must never be left half-written.

If a process crashes or the machine loses power while a file is being replaced, readers may observe incomplete content. A common way to reduce this risk is an atomic file write: write the new content to a temporary file first, then replace the destination with a rename.

This article shows a reusable implementation using only Go’s standard library.

What does “atomic write” mean?

The goal is to make the visible update happen as one replacement operation from the point of view of other processes.

Instead of doing this:

err := os.WriteFile("config.txt", data, 0644)

the safer pattern is:

  1. Create a temporary file in the same directory as the destination.
  2. Write the complete new content to the temporary file.
  3. Flush the temporary file with Sync.
  4. Set the desired permissions.
  5. Close the temporary file.
  6. Rename it over the destination.

Readers normally see either the old file or the new file rather than an intermediate partially written version.

A reusable atomic write function

Here is a complete implementation:

package main

import (
    "fmt"
    "os"
    "path/filepath"
)

func writeFileAtomic(path string, data []byte, perm os.FileMode) error {
    dir := filepath.Dir(path)

    temp, err := os.CreateTemp(dir, ".tmp-*")
    if err != nil {
        return err
    }
    tempName := temp.Name()

    cleanup := func() {
        temp.Close()
        os.Remove(tempName)
    }

    if _, err := temp.Write(data); err != nil {
        cleanup()
        return err
    }

    if err := temp.Sync(); err != nil {
        cleanup()
        return err
    }

    if err := temp.Chmod(perm); err != nil {
        cleanup()
        return err
    }

    if err := temp.Close(); err != nil {
        os.Remove(tempName)
        return err
    }

    if err := os.Rename(tempName, path); err != nil {
        os.Remove(tempName)
        return err
    }

    return nil
}

func main() {
    path := filepath.Join(os.TempDir(), "example-config.txt")

    if err := writeFileAtomic(path, []byte("version=2\n"), 0644); err != nil {
        panic(err)
    }

    data, err := os.ReadFile(path)
    if err != nil {
        panic(err)
    }

    fmt.Print(string(data))
}

The output is:

version=2

Why the temporary file should be in the same directory

It may be tempting to create the temporary file in the system temporary directory:

temp, err := os.CreateTemp("", "config-*")

That is not ideal for this pattern.

os.Rename can fail when the source and destination are on different filesystems. Creating the temporary file with the destination directory avoids that common problem:

dir := filepath.Dir(path)
temp, err := os.CreateTemp(dir, ".tmp-*")

Keeping both files on the same filesystem is also important because filesystem rename operations are generally what provide the atomic replacement behavior.

Why call Sync before renaming?

Write can return successfully even though data is still buffered by the operating system.

Calling:

if err := temp.Sync(); err != nil {
    return err
}

asks the operating system to flush the file’s data to stable storage before the rename is attempted.

This is especially useful for files representing durable application state.

However, Sync has a performance cost. For disposable caches or files that can easily be regenerated, you may decide that the additional durability is unnecessary.

Why close the file before os.Rename?

Closing the temporary file before replacing the destination is good cross-platform practice:

if err := temp.Close(); err != nil {
    return err
}

if err := os.Rename(tempName, path); err != nil {
    return err
}

Some operating systems are more restrictive than others about renaming files that still have open handles.

It also ensures errors from the final close are not silently ignored.

Always clean up failed temporary files

Every operation between CreateTemp and Rename can fail. Without cleanup, failed writes can leave many .tmp-* files behind.

This helper centralizes cleanup:

cleanup := func() {
    temp.Close()
    os.Remove(tempName)
}

The function removes the temporary file whenever writing, syncing, changing permissions, closing, or renaming fails.

Do not remove the temporary file after a successful rename because its old path no longer exists.

File permissions matter

os.CreateTemp initially creates a file with restrictive permissions. If the final file needs a specific mode, set it explicitly before renaming:

if err := temp.Chmod(0644); err != nil {
    return err
}

Passing the permission as an argument makes the helper more reusable:

writeFileAtomic("settings.json", data, 0600)

For files containing secrets, tokens, or private configuration, 0600 is often more appropriate than 0644 on Unix-like systems.

Writing JSON atomically

The same helper is useful for JSON configuration or application state.

package main

import (
    "encoding/json"
    "os"
)

type Settings struct {
    Theme string `json:"theme"`
    Debug bool   `json:"debug"`
}

func saveSettings(path string, settings Settings) error {
    data, err := json.MarshalIndent(settings, "", "  ")
    if err != nil {
        return err
    }

    data = append(data, '\n')
    return writeFileAtomic(path, data, 0644)
}

Separating serialization from the write operation has another advantage: if JSON marshaling fails, the existing file is never touched.

Common pitfalls

Writing directly to the destination

This is convenient:

file, err := os.Create(path)

But os.Create truncates an existing file immediately. If the program fails afterward, the previous valid content has already been destroyed.

Creating the temporary file on another filesystem

A rename between filesystems is not guaranteed to work. Put the temporary file beside the destination.

Ignoring Close errors

Developers often use:

defer temp.Close()

For ordinary reads that is often fine, but for a durability-sensitive write you should explicitly check the final close error before replacing the destination.

Forgetting temporary-file cleanup

A service that repeatedly encounters write errors can otherwise accumulate temporary files indefinitely.

Assuming atomic rename solves every durability problem

Atomic replacement protects readers from many partial-write scenarios, but it is not a complete transactional storage system.

For extremely strict durability guarantees, filesystem and operating-system behavior matters. Some applications also sync the parent directory after the rename so that directory metadata is persisted. Cross-platform behavior for that step needs additional platform-specific handling.

For most application configuration and local state files, the temporary-file-plus-rename pattern is a substantial improvement over truncating the destination directly.

When should you use atomic writes?

This technique is particularly useful for:

  • configuration files;
  • application state snapshots;
  • generated manifests;
  • package or build metadata;
  • local indexes;
  • lock-free files read by multiple processes;
  • JSON or YAML settings that are periodically rewritten.

It is usually less important for append-only logs or temporary data that can be regenerated easily.

A useful rule of thumb

If losing the old file and failing to finish the new file would leave your application in a bad state, write the replacement to a temporary file first.

The essential Go pattern is small:

temp, err := os.CreateTemp(filepath.Dir(path), ".tmp-*")
// write, Sync, Chmod, and Close...
err = os.Rename(temp.Name(), path)

That extra step turns a destructive in-place rewrite into a much safer replacement operation using only the Go standard library.

Golang Files Backend Reliability

Related Posts

Go Context Timeouts and Request Cancellation
Published on
Go Context Timeouts and Request Cancellation
A Go HTTP handler can outlive the request that started it unless the work inside the handler pays attention to cancellation. That matters when a client disconnects, an upstream request takes too long, or a database query is no longer useful. Go solves this with context.Context. Every incoming *http.Request already has a context, and that context is canceled when the client connection closes, the request is canceled by HTTP/2, or the handler returns. You can also derive a shorter deadline for work that should not consume the entire request lifetime.
Golang Backend HTTP
Go Error Wrapping with errors.Is and errors.As
Published on
Go Error Wrapping with errors.Is and errors.As
Errors often cross several layers of a Go application. A low-level function may know that a file is missing, while a higher-level function needs to add context about which operation failed. Go error wrapping lets you add that context without losing information callers need for reliable handling. Why error strings are fragile Do not make program logic depend on error wording. Adding a filename or changing punctuation can break string comparisons even when the underlying condition is unchanged. Prefer semantic checks:
Golang Errors Backend Best Practices
Graceful HTTP Server Shutdown in Go
Published on
Graceful HTTP Server Shutdown in Go
Stopping a web server with Ctrl+C looks harmless during development, but production deployments need a more careful shutdown process. If a process exits immediately, active HTTP requests can be interrupted, clients may receive connection errors, and in-flight work can be left unfinished. Go’s standard library already provides the pieces needed for a clean shutdown. The main tools are os/signal, context, and http.Server.Shutdown. This guide shows a practical pattern for shutting down an HTTP server when the process receives SIGINT or SIGTERM.
Golang HTTP Context Backend
Testing Go HTTP Handlers with httptest
Published on
Testing Go HTTP Handlers with httptest
HTTP handlers are one of the easiest parts of a Go service to test well. You usually do not need to bind a real network port, start the entire application, or depend on an external test framework. Go’s standard library provides net/http/httptest, which can construct HTTP requests, capture handler responses, and even start temporary HTTP servers when a real client-server round trip matters. This guide builds a small JSON endpoint and tests it at several useful levels.
Golang Testing HTTP Backend
Reverse Proxy with Nginx and Go for Microservices
Published on
Reverse Proxy with Nginx and Go for Microservices
As an application grows, splitting it into smaller services can make independent deployment and scaling easier. For example: Product service on port 8080 Blog service on port 8081 Users should not need to know those internal ports. An Nginx reverse proxy can expose both services under one domain and route requests by URL path. 1. Configure the Nginx Reverse Proxy Create a site configuration: Copy sudo nano /etc/nginx/sites-available/yourdomain.com Add:
DevOps Nginx Golang Microservices
Arrays and Slices in Go: Add, Delete, Search, Update, Sort, and Filter
Updated on
Arrays and Slices in Go: Add, Delete, Search, Update, Sort, and Filter
Go arrays and slices are straightforward once you understand their different roles. Arrays have a fixed length that is part of their type, while slices are flexible views over an underlying array and are the collection type used most often in application code. The following examples use a slice of structs: Copy type Person struct { Name string Age int } people := []Person{ {Name: "Alice", Age: 30}, {Name: "Bob", Age: 25}, {Name: "John", Age: 20}, {Name: "Zara", Age: 35}, } 1. Add an Element Use append:
Golang Array Slice
chevron-up