Serving RFC 9110 Range Requests in Go Without an io.ReadSeeker
http.ServeContent does Range requests properly. It also demands an io.ReadSeeker, which you do not have when your bytes live in S3, sit behind a remote origin, or get generated on demand. What you have is a function that takes an offset and a length. kofiadeyemiq/byterange is built for that shape: it parses RFC 9110 §14.1.2 Range headers into a validated set of ranges, then serves them from an Open(offset, length) callback.
The interesting part is where the package draws the line between parsing policy and response writing, and how it limits response amplification from a Range header.
The amplification problem Range parsing has to solve
A Range header can express 1300 one-byte ranges from a 10 KB file. RFC 9110 lets a server ignore or reject a suspicious set of many small ranges, and explains why: every part in a multipart/byteranges response carries a boundary plus headers, with typical overhead around 80 bytes per part depending on the media type and boundary length. The request can therefore produce a much larger response. That is the class of amplification behind CVE-2011-3192, the Apache httpd Range DoS.
The repo ships a measurement for it in bench/amplify/main.go. It builds a header with 1300 ascending one-byte ranges, pushes it through http.ServeContent and through byterange.Serve, and prints each response size as a multiple of the 10,000-byte body. Header generation is a plain strings.Builder loop:
func manyRangesHeader(n int) string {
var b strings.Builder
b.WriteString("bytes=")
for i := 0; i < n; i++ {
if i > 0 {
b.WriteByte(',')
}
pos := i * 2
b.WriteString(strconv.Itoa(pos))
b.WriteByte('-')
b.WriteString(strconv.Itoa(pos))
}
return b.String()
}
At the repository snapshot used for this article, go run ./bench/amplify produced a 190,209-byte http.ServeContent response, 19 times the body size. byterange.Serve rejected the excessive range count and returned the full body in a 10,045-byte response.
Parse limits are functional options, not struct fields
parse.go keeps tunables in an unexported config struct and exposes them through Option funcs:
type Option func(*config)
Three of them: WithMaxRanges, WithMaxTotalBytes, and WithCoalesce. The defaults are where the thinking shows.
defaultMaxRanges is 16, and the doc comment argues for the number instead of just stating it. A video player seeks a few times. A download manager resumes a few chunks. A PDF viewer grabs a few pages. With the default, a multipart response carries at most 16 requested parts before coalescing, keeping its fixed per-part overhead bounded. Parsing the header still depends on the input size, a distinction we’ll come back to below.
defaultMaxTotalBytes is 0, meaning off. A client asking for non-overlapping ranges cannot add up to more than the representation once coalescing is enabled, so a default byte cap would impose an extra policy on otherwise legitimate traffic. Applications can still set one when backend reads or egress need a tighter budget. The count and byte limits solve different problems.
WithCoalesce defaults to on. The parser merges overlapping or adjacent ranges, shrinking both the range count and any duplicated bytes on the wire. That also follows RFC 9110’s guidance that servers may coalesce ranges when sending separate parts would be less efficient.
An element-count guard, with an important limit
RFC 9110 §5.6.1.2 requires a recipient to parse and ignore a reasonable number of empty list elements, even though senders are not supposed to generate them. That means bytes=0-0,,,,,,,1-1 still contains only two actual ranges. Counting only non-empty ranges without any other guard would let a header contain an excessive number of commas.
parse.go answers that with a second, separate bound:
const emptyElementSlack = 64
const rawListElementsHardCeiling = 1 << 16
func rawListElementsCap(maxRanges int) int {
if maxRanges <= 0 {
return rawListElementsHardCeiling
}
if scaled := maxRanges * emptyElementSlack; scaled < rawListElementsHardCeiling {
return scaled
}
return rawListElementsHardCeiling
}
Look at the maxRanges <= 0 branch. Turning off the range-count policy still leaves an absolute ceiling of 65,536 comma-separated elements, while the default 16-range setting accepts at most 1,024 raw elements.
That is an element-count guard, not a complete input-size bound. Parse calls strings.Split before checking the count, so it has already scanned the entire field and allocated the slice; a single element can also be arbitrarily long. In a server, the HTTP stack’s header-size limit remains the byte-level defense. The useful pattern here is narrower: disabling a semantic limit does not also disable the parser’s empty-element ceiling.
Sentinel errors that encode the response, not just the failure
byterange.go declares five errors, and every doc comment tells the caller what to send back.
ErrUnsupportedUnit means the unit wasn’t bytes; RFC 9110 §14.2 says a server must ignore an unsupported unit, so the package responds with the full 200. ErrMalformed means the ranges-specifier was invalid; the RFC allows a server to ignore or reject it, and this package chooses 200. ErrTooManyRanges trips WithMaxRanges, while ErrRangesTooLarge trips WithMaxTotalBytes after coalescing; Serve also maps both policy rejections to a full 200. ErrUnsatisfiable is the odd one out: valid syntax with nothing satisfiable against the representation size gets a 416 with Content-Range: bytes */size.
Four of the five collapse to “pretend the header wasn’t there.” That is required for an unsupported unit and is the package’s chosen policy for malformed or application-rejected ranges. Serve does the mapping for you. Call Parse directly and use errors.Is, since the package wraps some of these errors with %w to attach detail.
Serving from an offset-and-length source
ServeOptions in serve.go gets built per request. The documented fields include Size (required), ContentType, ETag, and the Open callback. The doc comment goes out of its way to say ServeOptions carries no context, so Open normally closes over req.Context() when the backing store wants one. An S3 GetObject call, say.
Here’s how bench/amplify/main.go wires up an in-memory body:
opts := byterange.ServeOptions{
Size: bodySize,
Open: func(off, n int64) (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(body[off : off+n])), nil
},
}
if err := byterange.Serve(rec, req, opts); err != nil {
panic(err)
}
The io.ReadCloser return type earns its keep: a remote fetch needs somewhere to release the connection, and Serve closes each range’s reader. For local data, io.NopCloser is the entire adapter.
Serve then chooses among four outcomes: full 200 body, single-range 206, multipart/byteranges 206, or 416. RFC 9110 says a server must not generate a multipart response when the client requested only one range. It may use a one-part multipart response when the client requested several ranges but only one remains satisfiable or survives coalescing; Serve takes the simpler route and uses a single-range 206 whenever parsing leaves one range.
The body-writing primitives are separately usable
response.go hands you the pieces if you’d rather drive the response yourself. ContentRange(Range{Start: 0, End: 499}, 1234) gives you "bytes 0-499/1234". ContentRangeUnsatisfied(size) gives you "bytes */size" for a 416.
NewBoundary lives apart from WriteMultipart for a concrete reason: the boundary has to go into the Content-Type header, which you write before the first byte of body. It reads 16 bytes from crypto/rand and hex-encodes them. If that read fails, the function uses a deterministic fallback rather than exposing an error or encoding a partially filled buffer.
WriteMultipart takes an io.ReaderAt instead of an Open callback and bounds each part with io.NewSectionReader(ra, r.Start, r.Length()) before io.Copy into the part writer from mime/multipart. Headers go in as a textproto.MIMEHeader. That SectionReader wrap is the idiomatic way to hand a bounded window on a bigger ReaderAt to code that only knows how to read until EOF:
// Bound an io.ReaderAt to exactly one range before copying.
section := io.NewSectionReader(ra, start, length)
if _, err := io.Copy(dst, section); err != nil {
return err
}
WriteMultipart does not enforce the single-range rule, and the docs say why: it’s “a body-writing primitive, not a policy.” Policy lives in Serve. A caller using the lower-level helper therefore has to know whether the original request asked for one range or several.
If-Range is where most implementations get it wrong
ifrange.go implements the RFC 9110 §13.1.5 precondition. The edge cases are worth reading even if you never import the package.
Telling an entity-tag apart from an HTTP-date comes down to looking for a DQUOTE in the first three characters, which catches "abc" and W/"abc". Weak validators never satisfy an If-Range: if the request’s value is weak, or the resource’s ETag is, the function returns false, because the RFC requires strong comparison. For the date form, callers pass a zero modTime when Last-Modified is unavailable or is not known to be a strong validator, so it cannot match.
Then the comparison itself:
return t.Equal(modTime.Truncate(time.Second))
HTTP-date has one-second resolution. A parsed HTTP-date is already aligned to a whole second, but a modTime from a filesystem stat can carry nanoseconds. Truncating modTime before the exact comparison prevents that sub-second precision from turning a valid partial request into a full download. The Go time package makes the operation simple; remembering why it matters is the harder part.
One last case: an absent If-Range returns true, because there is no validator to invalidate the Range header. Returning false there would turn every ordinary Range request into a full-body response.
Most of these behaviors are one line of code guarding one sentence of RFC, which is why they are easy to miss. The empty-element safeguard and the Open callback shape are the two reasons I’d reach for this instead of rolling your own on net/http.