Why MinIO chose Go for S3-compatible storage at scale
Sooner or later, most backend teams hit the same problem: the application is coupled to S3, and you need to run it somewhere that isn’t AWS. A local dev environment, an on-prem deployment, a second cloud. MinIO exists for exactly this. It’s an object storage server written entirely in Go, and it’s interesting both as a tool and as a piece of Go software. Let’s look at both.
What is MinIO?
MinIO speaks the S3 protocol, so any application that works with Amazon S3 works with MinIO. Same SDKs, same API calls, different endpoint. You can run it on your laptop, in Kubernetes, or across multiple data centres.
The project is open source under AGPLv3, and it’s become the default answer for cloud-native storage, particularly in multi-cloud setups where you want one consistent API regardless of provider. The licence does have implications if you’re modifying it and hosting it commercially, so check that before you get too attached.
Why Go for high-performance storage?
Go might seem an odd choice for storage software. Surely C or Rust would be faster? In raw terms, probably. But MinIO bet that Go’s advantages outweigh the difference, and I think that bet paid off.
Goroutines are the big one. A storage server handles thousands of concurrent client connections, and each of those can be a cheap goroutine rather than a heavy thread. Then add single-binary deployment with no dependencies, a standard library that makes network servers almost boring to write, and compile times fast enough to keep iteration tight. The raw speed gap starts to look like a good deal.
Connecting to MinIO with Go
Here’s how to interact with MinIO from a Go application:
package main
import (
"context"
"fmt"
"log"
"strings"
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
)
func main() {
ctx := context.Background()
// Connect to MinIO
client, err := minio.New("localhost:9000", &minio.Options{
Creds: credentials.NewStaticV4("minioadmin", "minioadmin", ""),
Secure: false,
})
if err != nil {
log.Fatal(err)
}
// Create a bucket
bucketName := "my-bucket"
err = client.MakeBucket(ctx, bucketName, minio.MakeBucketOptions{})
if err != nil {
// Check if bucket already exists
exists, errExists := client.BucketExists(ctx, bucketName)
if errExists != nil || !exists {
log.Fatal(err)
}
}
// Upload an object
content := "Hello, MinIO!"
_, err = client.PutObject(ctx, bucketName, "greeting.txt",
strings.NewReader(content), int64(len(content)),
minio.PutObjectOptions{ContentType: "text/plain"})
if err != nil {
log.Fatal(err)
}
fmt.Println("Object uploaded successfully")
}
Notice how we pass context as the first parameter. This is standard practice in Go and gives you proper cancellation and timeout handling for free.
Performance tuning with concurrent uploads
For large files, MinIO supports multipart uploads, and the client lets you tune the concurrency:
package main
import (
"context"
"log"
"os"
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
)
func uploadLargeFile(filePath, bucketName, objectName string) error {
ctx := context.Background()
client, err := minio.New("localhost:9000", &minio.Options{
Creds: credentials.NewStaticV4("minioadmin", "minioadmin", ""),
Secure: false,
})
if err != nil {
return err
}
file, err := os.Open(filePath)
if err != nil {
return err
}
defer file.Close()
stat, err := file.Stat()
if err != nil {
return err
}
// Upload with concurrent parts
_, err = client.PutObject(ctx, bucketName, objectName, file, stat.Size(),
minio.PutObjectOptions{
ContentType: "application/octet-stream",
NumThreads: 4, // Concurrent upload threads
PartSize: 64 * 1024 * 1024, // 64MB parts
})
return err
}
Four threads and 64MB parts is a sensible starting point. The right numbers depend on your network and disks, so measure before tuning further.
Running MinIO in Kubernetes
MinIO is at home in Kubernetes. Here’s a minimal deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: minio
spec:
replicas: 1
selector:
matchLabels:
app: minio
template:
metadata:
labels:
app: minio
spec:
containers:
- name: minio
image: minio/minio:latest
args:
- server
- /data
- --console-address
- ":9001"
env:
- name: MINIO_ROOT_USER
value: "minioadmin"
- name: MINIO_ROOT_PASSWORD
value: "minioadmin"
ports:
- containerPort: 9000
- containerPort: 9001
Obviously don’t ship those default credentials anywhere that matters. And for anything beyond a single node, use the MinIO Operator, which handles distributed deployments properly.
Performance tips
MinIO’s own numbers talk about saturating 100Gbps network links. You will not get there by accident. The advice that actually matters: run on SSDs, because the erasure coding wants to be CPU-bound and spinning disks won’t let it be. Enable direct I/O to bypass the kernel page cache for large objects. Spread load across multiple servers with distributed mode. And don’t forget the client side. Increasing MaxIdleConnsPerHost in your HTTP transport stops you paying connection setup costs on every request:
// Custom HTTP transport for better performance
transport := &http.Transport{
MaxIdleConns: 100,
MaxIdleConnsPerHost: 100,
IdleConnTimeout: 90 * time.Second,
}
client, err := minio.New("localhost:9000", &minio.Options{
Creds: credentials.NewStaticV4("minioadmin", "minioadmin", ""),
Secure: false,
Transport: transport,
})
When to use MinIO
MinIO fits when you want S3 compatibility without the lock-in: on-premises object storage, one storage API across clouds, or high-throughput data pipelines. It is not a general-purpose filesystem and doesn’t pretend to be. It’s optimised for write once, read many. If your workload doesn’t look like that, look elsewhere.
Wrapping up
If you need object storage, the easiest way in is to run MinIO locally and point your existing S3 code at it. The switch is mostly a config change. The official documentation covers distributed deployments and the more advanced configuration from there.
MinIO is also one of my favourite answers to “what does production Go look like at scale?” If you want to see serious concurrency handled with mostly the standard library, clone the repo and start reading. Thirty minutes in that codebase will teach you more than most blog posts.