Reading bytes in Go looks simple: allocate a buffer, call Read, and inspect the error. The subtlety is that io.Reader does not promise to fill the buffer in one call. A valid reader may return fewer bytes than requested even when more data will arrive later.
That behavior matters for network protocols, binary file formats, framed messages, and any code that expects an exact number of bytes. Correct stream handling starts by matching the API to the requirement: use ordinary Read when partial progress is acceptable, and use helpers such as io.ReadFull when a fixed-size field must be complete.
Understand the io.Reader contract
The core interface is deliberately small:
type Reader interface {
Read(p []byte) (n int, err error)
}A call to Read reads up to len(p) bytes. The result n can be smaller than the buffer size without indicating failure.
Several rules are especially important:
- always process the first
nbytes whenn > 0; - do not treat
n < len(p)as end of input; - a reader may return data and a non-nil error in the same call;
- end of input is represented by
io.EOF, but the final data may be returned either withio.EOFor before a later call that returns0, io.EOF.
This flexibility lets readers represent files, sockets, compression streams, buffers, and many other sources without pretending that all of them naturally produce data in application-sized chunks.
Why one Read call is not an exact-length operation
Consider code that expects a four-byte header:
buf := make([]byte, 4)
n, err := r.Read(buf)
if err != nil {
return err
}
parseHeader(buf)The code assumes that a successful call filled all four bytes. That assumption is not part of the io.Reader contract.
If Read returns n == 2 and err == nil, only buf[:2] contains newly read data. Passing the entire four-byte slice to the parser can mix valid bytes with zero values or data left in a reused buffer.
Checking n fixes part of the problem:
n, err := r.Read(buf)
if n > 0 {
consume(buf[:n])
}
if err != nil {
return err
}This is correct for incremental consumption, but it is not enough when the protocol requires exactly four bytes before parsing.
Use io.ReadFull for fixed-size fields
For exact-length reads, the standard library provides io.ReadFull:
header := make([]byte, 4)
n, err := io.ReadFull(r, header)
if err != nil {
return fmt.Errorf("read header: %w", err)
}
// n == len(header) here.
parseHeader(header)io.ReadFull keeps reading until the buffer is full or an error prevents completion.
Its error behavior is useful for distinguishing an empty stream from a truncated field:
io.EOFmeans no bytes were read;io.ErrUnexpectedEOFmeans some bytes were read, but not enough to fill the buffer;- a nil error means the buffer was completely filled.
A small example demonstrates the distinction:
r := strings.NewReader("ABCDEFG")
header := make([]byte, 4)
n, err := io.ReadFull(r, header)
fmt.Printf("n=%d data=%q err=%v\n", n, header, err)
next := make([]byte, 4)
n, err = io.ReadFull(r, next)
fmt.Printf("n=%d data=%q err=%v\n", n, next[:n], err)The first read succeeds with ABCD. The second returns three bytes, EFG, together with io.ErrUnexpectedEOF because the requested four-byte field is incomplete.
Process data before errors in manual loops
When you do need a manual read loop, handle the bytes before the error:
buf := make([]byte, 32*1024)
for {
n, err := r.Read(buf)
if n > 0 {
if _, writeErr := w.Write(buf[:n]); writeErr != nil {
return writeErr
}
}
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return err
}
}Checking the error first can discard valid bytes if a reader returns n > 0 and a non-nil error together.
For ordinary stream copying, prefer io.Copy instead of writing this loop yourself. Manual loops are most useful when you need custom framing, progress accounting, hashing, limits, or transformation logic.
Bound variable-length input before allocating
Fixed-size fields are straightforward, but many formats include a length prefix followed by a variable-sized payload. Never trust an unbounded length from the stream.
A safer pattern is:
- read the fixed-size length field exactly;
- decode it using the format’s defined byte order;
- reject lengths above an application limit;
- allocate only after validation;
- read the payload exactly.
For a two-byte big-endian length field:
var lengthBuf [2]byte
if _, err := io.ReadFull(r, lengthBuf[:]); err != nil {
return fmt.Errorf("read length: %w", err)
}
length := binary.BigEndian.Uint16(lengthBuf[:])
if length > 4096 {
return fmt.Errorf("payload too large: %d", length)
}
payload := make([]byte, int(length))
if _, err := io.ReadFull(r, payload); err != nil {
return fmt.Errorf("read payload: %w", err)
}The limit is part of the protocol or application policy, not a property of io.ReadFull. Choose a bound appropriate for the data you expect.
Use io.ReadAtLeast when the minimum matters
Sometimes a parser needs a minimum number of bytes but can accept more. io.ReadAtLeast reads until it has obtained at least the requested minimum or encounters an error.
buf := make([]byte, 64)
n, err := io.ReadAtLeast(r, buf, 8)
if err != nil {
return err
}
consume(buf[:n])This differs from io.ReadFull, which requires the entire buffer to be filled. Use the helper whose contract matches the parser instead of recreating the behavior with ad hoc loops.
Do not confuse message boundaries with read boundaries
A stream reader exposes bytes, not application messages. One call to Read may return:
- part of one message;
- exactly one message;
- one complete message plus part of the next;
- data assembled from multiple lower-level packets.
Network packet boundaries therefore should not be used as protocol framing. Define framing explicitly with fixed-size records, delimiters, length prefixes, or another documented encoding.
For line-oriented text, bufio.Reader or bufio.Scanner may be more appropriate than manually searching chunks for delimiters. Keep in mind that scanners apply token-size limits and are intended for tokenized input rather than arbitrary unbounded records.
Treat zero-length progress carefully
Except for a zero-length buffer, implementations are discouraged from repeatedly returning 0, nil. A caller should not interpret 0, nil as EOF.
If you write infrastructure that wraps unusual readers, avoid busy loops that spin forever on no progress. Standard helpers already contain the behavior needed for common cases, which is another reason to prefer them over custom exact-read loops.
Keep framing separate from decoding
Reliable parsers usually become simpler when they separate two jobs:
Framing
Determine exactly which bytes belong to the next unit of input. This may involve io.ReadFull, a delimiter, or a validated length prefix.
Decoding
Interpret the complete frame as a number, record, request, or other domain value.
Separating the phases prevents decoders from accidentally depending on whatever chunk sizes the underlying reader happens to return.
Common pitfalls
Parsing the whole buffer instead of buf[:n]
Only the first n bytes are valid data from the current Read call.
Assuming nil error means a full buffer
A short read with err == nil is valid. Exact-size requirements need an explicit exact-read operation.
Dropping bytes returned with an error
Process n > 0 bytes before handling err in manual loops.
Retrying io.ReadFull blindly after ErrUnexpectedEOF
A truncated fixed-size field is usually a protocol or input error. Retrying the operation against the same ended stream does not reconstruct the missing bytes.
Allocating directly from an untrusted length prefix
Validate lengths before allocation to avoid excessive memory use and malformed-frame handling bugs.
Treating TCP reads as messages
TCP is a byte stream. Application framing must be defined above the transport.
Choose the smallest correct abstraction
Use Read when partial chunks are natural, io.ReadFull when a fixed number of bytes is mandatory, io.ReadAtLeast when only a minimum is required, and higher-level helpers such as io.Copy or buffered token readers when they better match the task.
The central rule is simple: buffer size expresses how much data a call may return, not how much it must return. Once code treats short reads as normal rather than exceptional, stream parsers become more portable across files, sockets, wrappers, and other io.Reader implementations.