Call Read once and assume the buffer is full. That can pass tests against a short strings.Reader, then truncate a message when a real connection returns fewer bytes.

Two methods carry most of Go’s I/O story:

type Reader interface {
	Read(p []byte) (n int, err error)
}

type Writer interface {
	Write(p []byte) (n int, err error)
}

That’s it. Files and TCP connections can implement both; gzip readers, HTTP request bodies, and strings.Reader implement Reader. Because the interfaces are small, the io package provides helpers such as Copy, TeeReader, MultiWriter, and LimitReader that work with implementations of these contracts, including yours.

If your functions accept io.Reader and io.Writer instead of []byte or *os.File, you get streaming, testability, and composition without writing any of it. But the contracts have sharp edges, and most of the bugs come from not reading them closely.

Read returns what it has, not what you asked for

Read is allowed to return fewer bytes than the length of your buffer. From the docs:

If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

So this is broken:

// BROKEN: assumes one Read fills the buffer
buf := make([]byte, 1024)
n, err := r.Read(buf)
if err != nil {
	return err
}
process(buf[:n]) // might only be 12 bytes of a 1024-byte message

Over a network connection or a pipe, you cannot assume one read corresponds to one write or one complete message.

There’s a second rule that matters just as much: Read can return bytes and an error in the same call. The docs say callers should process the n > 0 bytes before looking at err. A reader at the end of its input may return (5, io.EOF) or (5, nil) followed by (0, io.EOF). Both are legal, and you don’t get to pick which one you receive.

The correct loop handles both:

func countBytes(r io.Reader) (int, error) {
	buf := make([]byte, 4096)
	total := 0
	for {
		n, err := r.Read(buf)
		total += n // process data first

		if err == io.EOF {
			return total, nil // graceful end, not a failure
		}
		if err != nil {
			return total, err
		}
	}
}

Note err == io.EOF, not errors.Is. The docs call this out: Read must return io.EOF itself rather than an error wrapping it, because callers compare with ==. If you’re writing your own reader, don’t wrap EOF with fmt.Errorf("%w"): that breaks callers relying on the documented comparison.

Most of the time you don’t write that loop yourself, because the standard library already did.

Let io.Copy and io.ReadFull do the looping

io.Copy reads from a source until EOF and writes everything to a destination:

written, err := io.Copy(os.Stdout, r)

A successful Copy returns err == nil, not err == io.EOF. It treats EOF as the normal end of input and swallows it. Same for io.ReadAll.

Copy also does something smarter than a naive loop. It checks whether the source implements io.WriterTo or the destination implements io.ReaderFrom, and delegates to those methods if so. Some source and destination pairs can take an optimised path without you asking. When neither interface is present, Copy allocates a temporary buffer. In a hot loop, io.CopyBuffer lets you supply and reuse one:

buf := make([]byte, 32*1024)
if _, err := io.CopyBuffer(dst, src1, buf); err != nil {
	return err
}
if _, err := io.CopyBuffer(dst, src2, buf); err != nil {
	return err
}

Two caveats from the docs: CopyBuffer panics on a zero-length buffer, and ignores your buffer entirely if the WriterTo/ReaderFrom fast path applies. So the allocation you carefully avoided may not have been happening in the first place. Measure before you reach for it.

When you need exactly N bytes, a fixed-size header or a length-prefixed frame, use io.ReadFull. Its error behaviour is precisely what protocol parsers want:

header := make([]byte, 8)
if _, err := io.ReadFull(r, header); err != nil {
	if err == io.ErrUnexpectedEOF {
		return fmt.Errorf("truncated header: %w", err)
	}
	return err
}

ReadFull returns io.EOF only when zero bytes were read, and io.ErrUnexpectedEOF when it read some but not all. That distinction is the difference between “the client finished sending” and “the connection died mid-message”, which you probably want to log differently. io.ReadAtLeast generalises this with a minimum byte count and returns io.ErrShortBuffer if your minimum exceeds the buffer length.

Composing streams: TeeReader, MultiWriter, and LimitReader

Small interfaces mean you can build pipelines out of pieces that know nothing about each other. Here’s a function that streams an upload to disk while computing its SHA-256 and rejecting an oversized input, without holding the payload in memory:

package main

import (
	"crypto/sha256"
	"encoding/hex"
	"fmt"
	"io"
	"os"
)

const maxUpload = 10 << 20 // 10 MiB

// saveUpload streams body to path and returns the content hash.
func saveUpload(path string, body io.Reader) (string, error) {
	f, err := os.Create(path)
	if err != nil {
		return "", err
	}
	defer f.Close()

	// Read one extra byte to distinguish an oversized body from one at the limit.
	limited := io.LimitReader(body, maxUpload+1)

	// Every byte read for the copy is also fed to the hasher.
	hasher := sha256.New()
	tee := io.TeeReader(limited, hasher)

	n, err := io.Copy(f, tee)
	if err != nil {
		return "", err
	}
	if n > maxUpload {
		return "", fmt.Errorf("upload exceeds %d bytes", maxUpload)
	}
	if err := f.Sync(); err != nil {
		return "", err
	}
	if err := f.Close(); err != nil {
		return "", err
	}
	return hex.EncodeToString(hasher.Sum(nil)), nil
}

TeeReader writes to w whatever it reads from r, with no internal buffering. The write must complete before the read returns. Any write error surfaces as a read error, so io.Copy reports a failing secondary writer as a copy failure. LimitReader stops with EOF at its limit, so limiting to exactly maxUpload would silently accept a longer body. Reading one extra byte detects that case; the example may leave a partial file on error, so production code should write to a temporary file and remove it on failure before publishing the result.

io.MultiWriter is the mirror image. One write fans out to several destinations.

var audit bytes.Buffer
logTarget := io.MultiWriter(os.Stdout, &audit)
fmt.Fprintln(logTarget, "request handled")

The failure mode here deserves attention. MultiWriter writes to each writer one at a time, and if one returns an error, the whole write stops there and returns that error. It does not continue down the list. So a slow or dead third writer stalls or kills the first two, which makes MultiWriter a bad fit for anything where one destination is best-effort.

io.MultiReader concatenates readers, which is handy when you’ve peeked at a stream and need to put the bytes back:

peek := make([]byte, 4)
if _, err := io.ReadFull(r, peek); err != nil {
	return err
}
// Hand downstream a reader that still starts from byte 0.
full := io.MultiReader(bytes.NewReader(peek), r)

And io.Discard is a Writer that accepts everything and does nothing, useful when you need to consume a response body without keeping it.

Buffering is not the io package’s job

Nothing in io.Reader says anything about buffer sizes, and that’s deliberate. Repeated small writes to a file can be costly. bufio batches them by wrapping the interface:

w := bufio.NewWriter(f)

for _, line := range lines {
	if _, err := w.WriteString(line + "\n"); err != nil {
		return err
	}
}
return w.Flush()

Check the final Flush error: buffered bytes may not reach the underlying writer until then. On an earlier write error, the function returns that error instead of pretending the output succeeded. bufio.Writer also implements io.StringWriter, so io.WriteString can use its WriteString method instead of converting the string to []byte.

bufio.Reader plays the same role in reverse and adds methods the bare interface lacks: ReadByte, ReadRune, ReadString. The io docs point you there explicitly, noting that a Reader which doesn’t implement io.ByteReader “can be wrapped using bufio.NewReader to add this method.”

Writing adapters

One method means an adapter is a struct with a Read or Write method. Here’s a writer that counts bytes and forwards them:

// countingWriter wraps w and tracks how many bytes pass through.
type countingWriter struct {
	w io.Writer
	n int64
}

func (c *countingWriter) Write(p []byte) (int, error) {
	n, err := c.w.Write(p)
	c.n += int64(n) // count what actually landed
	return n, err
}

Two contract rules apply here. Write must return a non-nil error if it returns n < len(p), which is what io.ErrShortWrite exists for. And implementations must not retain p or modify it, even temporarily. The caller owns that slice and will almost certainly reuse it on the next iteration, so stashing a reference to it gives you data that mutates under your feet.

Readers have a matching rule: don’t return (0, nil) unless len(p) == 0. Callers read that as “nothing happened”, and some consumers give up with io.ErrNoProgress after enough of them.

If you want more background on why one-method interfaces like these are so easy to satisfy, this piece on Go interfaces covers implicit satisfaction and interface size.

Testing with in-memory streams

Accepting io.Reader instead of a filename makes the input easy to test without fixtures. This example still writes an output file in t.TempDir().

func TestSaveUpload(t *testing.T) {
	src := strings.NewReader("hello world")
	path := filepath.Join(t.TempDir(), "upload.bin")

	got, err := saveUpload(path, src)
	if err != nil {
		t.Fatalf("saveUpload: %v", err)
	}
	if want := "b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"; got != want {
		t.Errorf("hash = %s, want %s", got, want)
	}
}

strings.NewReader and bytes.NewReader are useful for the happy path, but small inputs with a large read buffer may not expose a partial-read bug. For that, write a reader that deliberately returns short reads:

// flakyReader returns one byte at a time, then returns err or EOF.
type flakyReader struct {
	data []byte
	err  error
}

func (f *flakyReader) Read(p []byte) (int, error) {
	if len(f.data) == 0 {
		if f.err != nil {
			return 0, f.err
		}
		return 0, io.EOF
	}
	if len(p) == 0 {
		return 0, nil
	}
	p[0] = f.data[0]
	f.data = f.data[1:]
	return 1, nil // deliberately short read
}

Feeding that into your parser is a quick way to find code that assumed one Read per message. Set err to io.ErrUnexpectedEOF and the same struct exercises an error after the available input. Stream parsers have enough surface area that this is also a reasonable place for fuzz testing.

io.Pipe covers the case where you need to connect writer-shaped code to reader-shaped code:

r, w := io.Pipe()
defer r.Close()
go func() {
	err := json.NewEncoder(w).Encode(payload)
	w.CloseWithError(err) // reports an encode error to the client reading the pipe
}()
resp, err := http.Post(url, "application/json", r)
if err != nil {
	return err
}
defer resp.Body.Close()

io.Pipe is synchronous and unbuffered. Each Write blocks until a read consumes the data, so the producer runs in its own goroutine. CloseWithError passes an encoding failure to the HTTP client as it reads the request body; a plain Close would instead signal a normal EOF. The server may already have received part of the body, so it should validate the request independently.

The API design rule that falls out of this

When you write a function that handles data, the signature choice is usually between these:

func Process(data []byte) ([]byte, error) // forces everything into memory
func Process(path string) error           // couples the function to file paths
func Process(dst io.Writer, src io.Reader) error // composable

The third works with files, sockets, HTTP bodies, gzip streams, bytes.Buffer in tests, and io.Discard when you want to benchmark the read path alone. A streaming implementation can also use bounded memory on a 10 GB input; the signature alone does not guarantee it.

Take the narrowest interface you need. If you only read, take io.Reader, not io.ReadCloser. Closing belongs to whoever opened the thing. If you genuinely need to hand a plain reader to something demanding a ReadCloser, io.NopCloser wraps it with a no-op Close. For random access, the grouped interfaces are there: io.ReadSeeker, io.ReaderAt, io.ReadWriteCloser. io.SectionReader builds on io.ReaderAt to expose a window of a larger source as its own independent stream, with Read, ReadAt, Seek, and Size.

None of those combinations would be worth having if the base interfaces had four methods each. Next time you’re about to add a second method to an interface, that’s the tradeoff you’re making.