A graceful shutdown sequence for Go HTTP servers: signals, readiness, request draining, deadlines, and a bounded wait for background workers.
· 11 min read

Graceful Shutdown for Go HTTP Servers: Signals, Draining, and Deadlines


http.Server.Shutdown does three things in order. It closes the listeners, closes idle connections, then waits for active connections to go idle. That third step is where the trouble lives: it waits indefinitely unless the context you pass expires first.

Two failure modes follow. With no deadline, a stuck handler can hold shutdown open until the orchestrator kills the process. With too short a deadline, Shutdown returns before requests finish. The timeout itself does not close those requests; an explicit Close or process exit does.

What follows is a shutdown sequence you can adapt: catching signals, flipping readiness, draining with a deadline, and cancelling background workers. The examples target Go 1.22+ and Unix signals. The server example uses the health handlers above it and the handler and worker functions below it; the worker’s batch and flush operations are application-specific placeholders.

What Shutdown actually guarantees

Four behaviors from the net/http docs are worth memorizing:

  • ListenAndServe returns http.ErrServerClosed the moment Shutdown is called. That’s not a failure. It’s the normal exit path.
  • Shutdown returns the context’s error if the context expires before draining finishes. Otherwise you get whatever error came back from closing the listeners.
  • Hijacked connections (WebSockets, raw TCP upgrades) are neither waited for nor closed. Those are your problem.
  • A server that has been shut down cannot be reused. Later calls to Serve return ErrServerClosed.

A common mistake is calling Shutdown from a goroutine and letting main return anyway. The process exits before draining finishes. main has to wait for the shutdown path to complete.

Catching signals with signal.NotifyContext

Before Go 1.16 you wired up an os.Signal channel by hand. signal.NotifyContext collapses that into a context that cancels when a signal arrives, which composes with everything else in your program that already takes a context.

package main

import (
	"context"
	"os/signal"
	"syscall"
)

func main() {
	// ctx is cancelled on SIGINT or SIGTERM.
	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer stop()

	// ... run server, block on <-ctx.Done()
	_ = ctx
}

Containers commonly receive SIGTERM, not the SIGINT sent by Ctrl+C. Kubernetes’ termination sequence normally runs any preStop hook, sends TERM, then forcibly kills remaining processes when the grace period expires. The hook consumes part of that same budget. Docker’s stop signal and timeout are configurable; its default first signal is SIGTERM. Catching only os.Interrupt misses that shutdown path. Ensure the application receives the configured signal, including through any entrypoint wrapper.

Calling stop() unregisters this notification. If nothing else is handling SIGINT, a second Ctrl+C then takes the default exit path.

Flip readiness before you drain

Routing changes and listener shutdown can race during a deployment.

When SIGTERM lands, a load balancer may still route traffic to the pod. Kubernetes marks terminating EndpointSlice entries as not ready, but routing updates and local shutdown happen concurrently. Closing the listener immediately can leave requests hitting an endpoint whose removal has not propagated yet.

Fail application readiness first, then allow a measured propagation window while still serving requests. The right delay depends on your ingress, probes, and load balancer; a fixed sleep is not a guarantee of zero dropped requests.

package main

import (
	"net/http"
	"sync/atomic"
)

type healthState struct {
	ready atomic.Bool
}

func (h *healthState) readyHandler(w http.ResponseWriter, r *http.Request) {
	if !h.ready.Load() {
		http.Error(w, "shutting down", http.StatusServiceUnavailable)
		return
	}
	w.WriteHeader(http.StatusOK)
	_, _ = w.Write([]byte("ok"))
}

// liveness stays 200 the whole time — we're alive, just not accepting new work.
func (h *healthState) liveHandler(w http.ResponseWriter, r *http.Request) {
	w.WriteHeader(http.StatusOK)
}

atomic.Bool (Go 1.19+) fits the access pattern exactly: read by every health check goroutine, written once by the shutdown path. A plain bool is a data race. A mutex works but buys you nothing here.

Keep readiness and liveness separate: refusing new work does not mean the process is unhealthy. Failing liveness can trigger restarts outside the normal pod-deletion path and is not a way to request traffic draining.

Putting the sequence together

The ordering: signal, flip readiness, sleep for the LB propagation window, Shutdown with a deadline, cancel background workers, wait for workers, exit.

package main

import (
	"context"
	"errors"
	"log/slog"
	"net/http"
	"os"
	"os/signal"
	"sync"
	"syscall"
	"time"
)

const (
	readinessDrainDelay   = 5 * time.Second
	shutdownTimeout       = 15 * time.Second
	workerShutdownTimeout = 5 * time.Second
)

func main() {
	logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
	sigCtx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer stop()

	health := &healthState{}
	health.ready.Store(true)

	mux := http.NewServeMux()
	mux.HandleFunc("GET /healthz", health.liveHandler)
	mux.HandleFunc("GET /readyz", health.readyHandler)
	mux.HandleFunc("GET /work", slowHandler)

	srv := &http.Server{
		Addr:              ":8080",
		Handler:           mux,
		ReadHeaderTimeout: 5 * time.Second,
		// Allow enough time to write normal responses. This limits writes,
		// not handler execution; downstream work needs its own deadlines.
		WriteTimeout: 20 * time.Second,
		IdleTimeout:  60 * time.Second,
	}

	// Background workers get their own cancellable context.
	workerCtx, cancelWorkers := context.WithCancel(context.Background())
	defer cancelWorkers()
	var workers sync.WaitGroup

	workers.Add(1)
	go func() {
		defer workers.Done()
		runQueueConsumer(workerCtx, logger)
	}()

	serverErr := make(chan error, 1)
	go func() {
		logger.Info("server starting", "addr", srv.Addr)
		if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
			serverErr <- err
			return
		}
		serverErr <- nil
	}()

	select {
	case err := <-serverErr:
		if err != nil {
			logger.Error("server failed", "err", err)
			cancelWorkers()
			os.Exit(1)
		}
	case <-sigCtx.Done():
		logger.Info("shutdown signal received")
	}

	// Stop catching signals so a second Ctrl+C kills us immediately.
	stop()

	// 1. Fail readiness so the load balancer stops routing to us.
	health.ready.Store(false)
	logger.Info("readiness disabled, waiting for load balancer", "delay", readinessDrainDelay)
	time.Sleep(readinessDrainDelay)

	// 2. Drain in-flight HTTP requests with a hard deadline.
	shutdownCtx, cancelShutdown := context.WithTimeout(context.Background(), shutdownTimeout)
	defer cancelShutdown()

	if err := srv.Shutdown(shutdownCtx); err != nil {
		logger.Error("graceful shutdown failed, forcing close", "err", err)
		// Close() rips open connections out from under their handlers.
		if closeErr := srv.Close(); closeErr != nil {
			logger.Error("forced close failed", "err", closeErr)
		}
	}

	// 3. Now that no new requests can arrive, stop background work.
	cancelWorkers()

	done := make(chan struct{})
	go func() {
		workers.Wait()
		close(done)
	}()

	select {
	case <-done:
		logger.Info("shutdown complete")
	case <-time.After(workerShutdownTimeout):
		logger.Warn("workers did not stop in time, exiting anyway")
	}
}

Three details in there deserve more than a code comment.

The order of Shutdown and cancelWorkers depends on your dependencies. Keep services that in-flight handlers need alive until HTTP draining finishes. Independent consumers may instead need to stop accepting new jobs as soon as shutdown begins.

srv.Close() is the forced-close fallback when draining fails. A deadline error means the drain did not finish within the budget, not that it can never finish. Close closes ordinary HTTP connections, but not hijacked connections, and does not wait for handler goroutines. Clients may receive errors or incomplete responses; buffered writes also mean a handler is not guaranteed to see an error on its next Write.

Your timeout budget has to fit the platform. The constants above allocate 5 + 15 + 5 = 25 seconds, leaving five seconds of headroom against a 30-second grace period. Include any preStop time and other cleanup in that budget. These values are starting points, not a substitute for measuring your own request durations and routing delays.

Propagating cancellation to handlers

Shutdown waits for handlers to return. It never cancels them. A handler sitting on a 60-second database query keeps right on going, and Shutdown politely waits.

Handlers need to observe cancellation through r.Context() and pass it to downstream operations. For an additional application-controlled cancellation path, give the server a base context (this fragment also needs the net import):

baseCtx, cancelBase := context.WithCancel(context.Background())
defer cancelBase()

srv := &http.Server{
	Addr:    ":8080",
	Handler: mux,
	BaseContext: func(net.Listener) context.Context {
		return baseCtx
	},
}

// Later, once Shutdown has returned or timed out:
// cancelBase()

With that base context, cancellation propagates to the request contexts. Cancelling before draining finishes can interrupt work you meant to preserve. Cancelling after a drain timeout signals remaining cooperative work to stop, but calling Close immediately afterward gives it no guaranteed time to unwind. If you need a separate cancellation grace period, explicitly wait for your handlers and account for it in the overall budget.

Handlers have to hold up their end:

func slowHandler(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context()

	select {
	case <-time.After(3 * time.Second): // stand-in for real work
		_, _ = w.Write([]byte("done"))
	case <-ctx.Done():
		// Client hung up, or the base context was cancelled.
		http.Error(w, "request cancelled", http.StatusServiceUnavailable)
	}
}

For a refresher on how cancellation moves through a call chain, see Context in Go: Cancellation, Timeouts, and Values.

Background workers that stop cleanly

A worker can check ctx.Done() between jobs and pass the context into each job. Here, cancellation can interrupt the current batch; it does not guarantee that batch finishes. Define retry or checkpoint behavior for partial work. A select also gives cancellation no priority over a ready tick, so the extra context check avoids starting work when cancellation is already observable.

func runQueueConsumer(ctx context.Context, logger *slog.Logger) {
	ticker := time.NewTicker(time.Second)
	defer ticker.Stop()

	for {
		select {
		case <-ctx.Done():
			logger.Info("consumer stopping", "reason", ctx.Err())
			// Flush anything buffered here, using a *fresh* context —
			// ctx is already cancelled, so it can't be used for I/O.
			flushCtx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
			defer cancel()
			if err := flushPending(flushCtx); err != nil {
				logger.Error("flush failed", "err", err)
			}
			return
		case <-ticker.C:
			if ctx.Err() != nil {
				continue // Return through the cancellation/flush branch.
			}
			if err := processBatch(ctx); err != nil {
				logger.Error("batch failed", "err", err)
			}
		}
	}
}

processBatch and flushPending represent your own queue and storage code. They must honor their contexts; cancellation cannot forcibly stop arbitrary Go code. The fresh context gives the final flush its own budget because reusing the cancelled worker context would cause cooperative I/O to abort. That budget must fit inside the worker wait in main. A timeout or forced exit still needs a recovery strategy for unpersisted work.

WebSockets and other hijacked connections

Shutdown does not manage hijacked connections such as WebSockets. Track and close those separately; successful HTTP draining is not proof that they have finished.

Server.RegisterOnShutdown gives you a hook that fires when Shutdown starts:

srv.RegisterOnShutdown(func() {
	// Called in its own goroutine when Shutdown starts.
	// Start protocol-specific shutdown without waiting for it here.
	// hub is your application's connection registry.
	hub.CloseAll()
})

The callback must initiate shutdown without blocking for completion. Wait separately for tracked connections, with a deadline. Register them with your wait group when they are accepted, not inside this callback, so the wait cannot race ahead of registration.

Testing that draining works

This standalone test uses channels to prove the handler is active before shutdown starts. It releases the handler only after Serve returns, then checks both successful draining and the complete response body. Save it as shutdown_test.go in a Go module and run go test -race.

package main

import (
	"context"
	"errors"
	"io"
	"net"
	"net/http"
	"testing"
	"time"
)

func TestShutdownDrainsInFlightRequests(t *testing.T) {
	started := make(chan struct{})
	release := make(chan struct{})
	mux := http.NewServeMux()
	mux.HandleFunc("/slow", func(w http.ResponseWriter, r *http.Request) {
		close(started)
		select {
		case <-release:
			_, _ = w.Write([]byte("finished"))
		case <-r.Context().Done():
		}
	})

	ln, err := net.Listen("tcp", "127.0.0.1:0")
	if err != nil {
		t.Fatal(err)
	}

	srv := &http.Server{Handler: mux}
	t.Cleanup(func() { _ = srv.Close() })
	serveDone := make(chan error, 1)
	go func() {
		serveDone <- srv.Serve(ln)
	}()

	type result struct {
		status int
		body   string
		err    error
	}
	resultCh := make(chan result, 1)
	go func() {
		client := &http.Client{Timeout: 5 * time.Second}
		resp, err := client.Get("http://" + ln.Addr().String() + "/slow")
		if err != nil {
			resultCh <- result{err: err}
			return
		}
		defer resp.Body.Close()
		body, err := io.ReadAll(resp.Body)
		resultCh <- result{status: resp.StatusCode, body: string(body), err: err}
	}()

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()
	select {
	case <-started:
	case <-ctx.Done():
		t.Fatal("handler never started")
	}

	shutdownDone := make(chan error, 1)
	go func() { shutdownDone <- srv.Shutdown(ctx) }()
	select {
	case err := <-serveDone:
		if !errors.Is(err, http.ErrServerClosed) {
			t.Fatalf("serve: %v", err)
		}
	case <-ctx.Done():
		t.Fatal("server did not stop accepting connections")
	}
	close(release)

	select {
	case err := <-shutdownDone:
		if err != nil {
			t.Fatalf("shutdown: %v", err)
		}
	case <-ctx.Done():
		t.Fatal("shutdown never completed")
	}

	select {
	case got := <-resultCh:
		if got.err != nil || got.status != http.StatusOK || got.body != "finished" {
			t.Fatalf("in-flight response: %+v", got)
		}
	case <-ctx.Done():
		t.Fatal("response never completed")
	}
}

Test the timeout path separately: keep the handler blocked, call Shutdown with a short deadline, and check for context.DeadlineExceeded. Then release the handler and verify that its response can still finish. This distinguishes an expired drain budget from a forced close; add a separate case for your application’s Close policy.

Common mistakes

Passing context.Background() to Shutdown. The doc example does this, but the doc example is demonstrating signal wiring, not timeout policy. With no deadline, one stuck handler holds the process open until the orchestrator kills it.

Treating ErrServerClosed as a failure. log.Fatalf on any error from ListenAndServe turns every clean shutdown into a logged crash and a non-zero exit code, which then confuses whoever is debugging the deploy. Check with errors.Is(err, http.ErrServerClosed).

Ignoring routing propagation. Closing the listener before routing updates have reached your load balancer can cause rollout errors. Measure that window and test draining through your actual ingress; sleeping for five seconds does not prove traffic has stopped.

Treating WriteTimeout as a handler deadline. It limits response writes, not computation or a blocked database call. Give downstream work deadlines and make it observe cancellation; keep the shutdown deadline as the final bound on how long the process waits.

Forgetting to call stop() after the first signal. Without it, an operator hammering Ctrl+C gets nothing, because your handler is still catching signals and throwing them away.

Check the shutdown budget against your deployment manifest, then test it with slow requests and workers that fail to stop. The graceful path matters, but so does knowing exactly what your process abandons when time runs out.