Most file sync tools come with a business model attached: your files sit on someone else’s servers and you pay for the privilege. Syncthing takes a different view. It’s an open source continuous file synchronisation program written in Go, and your devices talk directly to each other over the Block Exchange Protocol (BEP) on TLS 1.3. No cloud in the middle, no account, no subscription.

I think it’s one of the more interesting Go codebases you can read. It’s approximately 200,000 lines of Go doing genuinely hard things: structured concurrency across many device connections, a custom wire protocol, and system integration across several platforms. Let’s look at how it’s put together.

Go’s role in Syncthing’s architecture

Syncthing compiles to native binaries for Windows, macOS, Linux, FreeBSD, and Android from a single Go codebase. The cross-compilation is as boring as you’d hope, just GOOS and GOARCH:

GOOS=linux GOARCH=amd64 go build
GOOS=windows GOARCH=amd64 go build
GOOS=darwin GOARCH=arm64 go build

The binary comes in around 12-15 MB with no external dependencies, because Go’s static linking produces fully self-contained executables.

The architecture leans hard on Go’s concurrency primitives, and it has to. The program maintains persistent TCP connections to multiple devices, monitors filesystem changes, manages block transfers, and resolves conflicts, all at the same time. Each device connection runs in its own goroutine, with channels coordinating state between components.

Filesystem monitoring with Go

Syncthing uses filesystem watchers to detect changes without constant polling. On Linux that means inotify; on macOS, FSEvents; on Windows, ReadDirectoryChangesW. Writing that three times would be miserable, and thankfully Go’s fsnotify package abstracts the platform-specific APIs:

package main

import (
	"context"
	"log"

	"github.com/fsnotify/fsnotify"
)

type FileWatcher struct {
	watcher *fsnotify.Watcher
	changes chan string
}

func NewFileWatcher() (*FileWatcher, error) {
	watcher, err := fsnotify.NewWatcher()
	if err != nil {
		return nil, err
	}
	return &FileWatcher{
		watcher: watcher,
		changes: make(chan string, 100),
	}, nil
}

func (fw *FileWatcher) Watch(ctx context.Context, path string) error {
	if err := fw.watcher.Add(path); err != nil {
		return err
	}

	go func() {
		defer fw.watcher.Close()
		for {
			select {
			case <-ctx.Done():
				return
			case event, ok := <-fw.watcher.Events:
				if !ok {
					return
				}
				// Syncthing is interested in writes, creates, removes, and renames
				if event.Op&(fsnotify.Write|fsnotify.Create|fsnotify.Remove|fsnotify.Rename) != 0 {
					select {
					case fw.changes <- event.Name:
					case <-ctx.Done():
						return
					}
				}
			case err, ok := <-fw.watcher.Errors:
				if !ok {
					return
				}
				log.Printf("watcher error: %v", err)
			}
		}
	}()

	return nil
}

func (fw *FileWatcher) Changes() <-chan string {
	return fw.changes
}

There’s a lot of well-worn Go in this small example. The watcher stops cleanly when the context is cancelled, defer fw.watcher.Close() makes sure the file descriptor is released, and the changes channel is buffered so a burst of file changes doesn’t block the event loop. Note the nested select when forwarding a change: it means the goroutine can still exit during shutdown even if the consumer has stopped reading.

The Block Exchange Protocol

Syncthing’s Block Exchange Protocol (BEP) divides files into blocks and transfers only the blocks that differ between devices. Block size varies with file size: smaller files use smaller blocks (typically 128 KiB for files under 250 MB), while larger files use blocks up to 16 MiB to keep metadata overhead down.

Each block is identified by a SHA-256 hash. When synchronising, devices exchange block lists (file metadata containing offset, size, and hash for each block), and only blocks with mismatched hashes cross the wire. It’s the same core insight as rsync, applied peer-to-peer.

Here’s a simplified implementation of block hashing in Go:

package main

import (
	"crypto/sha256"
	"fmt"
	"io"
	"os"
)

type Block struct {
	Offset int64
	Size   int64
	Hash   [32]byte // SHA-256 produces exactly 32 bytes
}

// calculateBlockSize returns the block size for a given file size.
// This mimics Syncthing's variable block sizing strategy.
func calculateBlockSize(fileSize int64) int64 {
	const (
		minBlockSize = 128 << 10  // 128 KiB
		maxBlockSize = 16 << 20   // 16 MiB
	)

	// Use 128 KiB blocks for files up to 250 MB
	if fileSize < 250<<20 {
		return minBlockSize
	}

	// Scale block size with file size, capped at 16 MiB
	blockSize := fileSize / 2000
	if blockSize > maxBlockSize {
		return maxBlockSize
	}
	if blockSize < minBlockSize {
		return minBlockSize
	}
	return blockSize
}

func hashFile(path string) ([]Block, error) {
	f, err := os.Open(path)
	if err != nil {
		return nil, fmt.Errorf("open file: %w", err)
	}
	defer f.Close()

	stat, err := f.Stat()
	if err != nil {
		return nil, fmt.Errorf("stat file: %w", err)
	}

	blockSize := calculateBlockSize(stat.Size())
	numBlocks := (stat.Size() + blockSize - 1) / blockSize
	blocks := make([]Block, 0, numBlocks)

	buf := make([]byte, blockSize)
	offset := int64(0)

	for {
		n, err := io.ReadFull(f, buf)
		if n > 0 {
			hash := sha256.Sum256(buf[:n])
			blocks = append(blocks, Block{
				Offset: offset,
				Size:   int64(n),
				Hash:   hash,
			})
			offset += int64(n)
		}
		if err == io.EOF || err == io.ErrUnexpectedEOF {
			break
		}
		if err != nil {
			return nil, fmt.Errorf("read file: %w", err)
		}
	}

	return blocks, nil
}

A few details in this code are worth pausing on. Using [32]byte for the hash rather than []byte avoids a heap allocation, since the size is known at compile time. make([]Block, 0, numBlocks) pre-allocates capacity so append doesn’t keep reallocating. io.ReadFull matters too: unlike Read, it tries to fill the whole buffer, which keeps block sizes consistent. And the %w error wrapping preserves the error chain for when things go wrong at 2am.

Once both sides have hashed their files, syncing is just comparing block lists and requesting whatever doesn’t match.

Device discovery and connection establishment

The unglamorous part of any P2P system is finding the other peer, and Syncthing layers three mechanisms to do it.

Local discovery is UDP broadcast on port 21027, to the IPv4 broadcast address and the IPv6 multicast group ff12::8384. Devices on the same LAN respond with their listening address, and the broadcast repeats every 30 seconds.

Global discovery kicks in for devices that aren’t on your network. Each device announces itself to discovery servers over HTTPS, including its device ID (derived from its TLS certificate) and current external IP addresses. Other devices query those servers to find connection endpoints. The discovery server protocol is documented in the BEP specification.

And when a direct connection fails, usually thanks to NATs or firewalls, Syncthing falls back to relay servers using the Relay Protocol. Relays forward encrypted data between devices without ever seeing the plaintext, because the BEP connection itself is TLS-encrypted end to end.

Here’s how device IDs work in Go:

package main

import (
	"crypto/sha256"
	"encoding/base32"
	"strings"
)

// DeviceID generates a Syncthing device ID from a TLS certificate's
// SHA-256 fingerprint. Syncthing uses base32 encoding (Luhn mod N check digit algorithm).
func DeviceID(certSHA256 []byte) string {
	// Syncthing uses a base32 alphabet without padding
	encoded := base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(certSHA256)

	// Split into chunks of 13 characters separated by hyphens
	// (Syncthing uses 7-character chunks with Luhn check digits, simplified here)
	var chunks []string
	for i := 0; i < len(encoded); i += 13 {
		end := i + 13
		if end > len(encoded) {
			end = len(encoded)
		}
		chunks = append(chunks, encoded[i:end])
	}

	return strings.Join(chunks, "-")
}

Every Syncthing device generates a TLS certificate on first run, and the device ID is derived from that certificate’s SHA-256 fingerprint. It’s a neat design: identity falls out of the crypto for free, with no central registry handing out names.

The REST API and embedded web server

Syncthing embeds an HTTP server (Go’s net/http) that serves both a web UI and REST API on port 8384. The web UI is compiled into the binary using Go’s embed package (introduced in Go 1.16):

package main

import (
	"embed"
	"io/fs"
	"net/http"
)

//go:embed gui/*
var webUI embed.FS

func setupHTTPServer() *http.Server {
	// Extract the gui subdirectory
	guiFS, _ := fs.Sub(webUI, "gui")

	mux := http.NewServeMux()

	// Serve static files from embedded filesystem
	mux.Handle("/", http.FileServer(http.FS(guiFS)))

	// REST API endpoints
	mux.HandleFunc("/rest/system/status", handleSystemStatus)
	mux.HandleFunc("/rest/db/completion", handleCompletion)

	return &http.Server{
		Addr:    "127.0.0.1:8384",
		Handler: mux,
	}
}

func handleSystemStatus(w http.ResponseWriter, r *http.Request) {
	// Return JSON system status
	w.Header().Set("Content-Type", "application/json")
	// ... implementation
}

The //go:embed directive bakes the entire web UI in at compile time. HTML, JavaScript, CSS, images, all inside one binary with no separate installation step. Before Go 1.16 this required third-party tools and code generation hacks, so it’s easy to forget what a quality-of-life improvement embed was.

Protocol implementation in Go

Syncthing implements BEP using Protocol Buffers for message serialization. The .proto files define message structures, and protoc-gen-go generates Go code:

// Generated from BEP protocol buffer definition
type Index struct {
	Folder string
	Files  []FileInfo
}

type FileInfo struct {
	Name         string
	Size         int64
	ModifiedS    int64
	Blocks       []BlockInfo
	Version      Vector
	Permissions  uint32
}

The protocol runs over TLS 1.3, with Go’s crypto/tls package doing the heavy lifting. Authentication is mutual TLS: both sides present certificates, and the device ID is verified against the certificate.

For concurrency, Syncthing spawns a goroutine per device connection. Each one handles message encoding, decoding, and block transfer, with a connection manager coordinating state:

type ConnectionManager struct {
	connections map[DeviceID]*Connection
	mu          sync.RWMutex
}

func (cm *ConnectionManager) Add(deviceID DeviceID, conn net.Conn) {
	cm.mu.Lock()
	defer cm.mu.Unlock()

	c := &Connection{
		deviceID: deviceID,
		conn:     conn,
		sender:   make(chan Message, 100),
	}

	cm.connections[deviceID] = c

	// Start goroutines for send/receive
	go c.sendLoop()
	go c.receiveLoop()
}

Splitting send and receive into separate goroutines is the standard Go move for full-duplex protocols. Each direction runs independently, with channels coordinating message flow, and neither side can starve the other.

Go implementation insights

What I take away from Syncthing is how consistently it applies patterns you already know. Every component, from the file watcher to the connection manager to the block exchanger, runs in its own goroutine and communicates over channels. Shutdown propagates through context.Context so the whole thing terminates cleanly. Platform-specific code for filesystem operations and network discovery hides behind interfaces, with implementations chosen at compile time. Block buffers are pooled with sync.Pool to keep GC pressure down during transfers, and there’s proper observability via expvar metrics and pprof profiling. None of these techniques is exotic on its own; seeing them all held together across 200,000 lines is the lesson.

If you’re building networked systems in Go, spend some time in this codebase. It deals with the problems that only show up in the real world: NAT traversal, conflict resolution, incremental syncing, platforms that all disagree with each other. I’d start from the entry point and follow a single file change all the way to the wire.