NanotDB is a small, append-only time-series database written in Go. It targets a narrow problem: long-running sensor data collection on hardware that doesn’t have much to spare. No cluster coordination. No query planner. Just a storage engine that writes numeric samples to plain files and can maintain rollups for cheaper long-range queries.

What caught my attention from a Go perspective is how the internals are organized. The project uses Go’s internal/ package convention to keep the engine private, TOML for engine and database defaults, and a clean split between the storage engine and its CLI tooling. None of this is flashy. All of it is well-considered.

Project structure and the internal/engine package

The core of NanotDB lives in internal/engine/. The storage logic, WAL (write-ahead log), page management, catalog, and rollup aggregation code all sit here. By placing everything under internal/, the project makes sure nothing outside the module can import the engine directly. Only the binaries in cmd/ — cmd/nanotdb/main.go for the server and cmd/nanocli/main.go for the CLI — get access.

If you haven’t used this convention before, it’s worth knowing. When you’re building a database or any library where you want tight control over the public API, the internal/ directory is the right tool. Anything under internal/ is invisible to external consumers of your module. NanotDB uses this to keep its entire storage engine private while exposing functionality only through its CLI and server binaries.

The files inside internal/engine/ map to clear responsibilities:

• engine.go — the main engine type and entry point
• db.go — database-level operations
• wal.go — write-ahead log implementation
• pages.go — page-based storage management
• catalog.go — metadata tracking for series
• timestamps.go — timestamp handling for time-series data
• rollup.go and rollup_aggregates.go — pre-computed aggregations

This single-package-for-the-engine layout keeps things simple. No deep package hierarchy to wander through. All the engine types can reference each other directly without import cycles. For a database this size, that’s the right call. You could split it up, but why?

WAL, pages, and append-only storage

NanotDB’s raw data files are append-only: completed page frames are written out and then treated as immutable. Inserts must be monotonically non-decreasing per metric, which keeps the write path simple and avoids the extra machinery you’d need for arbitrary late updates.

The write-ahead log in internal/engine/wal.go handles crash recovery when WAL is enabled. The hot path writes a compact record before mutating the in-memory page. Known metrics can be stored as a MetricID, a timestamp delta from a baseline, flags, and a 4-byte value; first-seen metrics also carry the metric name and type so replay can rebuild catalog state. The WAL fsync policy is configurable: segment by default, or always if you want every append synced.

Page-based storage in internal/engine/pages.go organizes samples into variable-length compressed page frames. Each frame starts with an 18-byte header (StartTime, EndTime, NumRecords), then an S2-compressed payload containing metric ids, timestamps, and raw values, followed by a CRC32 of the compressed bytes. Go’s encoding/binary, byte slices, and os.File APIs are a good fit for this sort of layout: explicit enough to reason about, without needing a separate serialization framework.

The stats files — internal/engine/datstats.go, internal/engine/walstats.go, and internal/engine/enginestats.go — track the storage layers directly: page frame counts and min/max timestamps for data files, append/flush/fsync counters for the WAL, and aggregate engine stats. Those are useful signals when you’re running on small machines where disk behaviour matters.

Rollup aggregations

When you’re working with sensor data, you almost never want every raw data point across a long time range. You want averages, minimums, maximums over intervals. NanotDB handles this with rollups, implemented across internal/engine/rollup.go and internal/engine/rollup_aggregates.go.

Pre-computing aggregations is a classic time-series optimization. Instead of scanning millions of raw points to compute an hourly average, you store the computed result in a rollup database and query that coarser series later. You pay a bit more CPU and disk when the rollup job runs. You get much cheaper reads for dashboards and long time ranges.

If you’ve worked with time-series tools before, this is similar to what Prometheus does with recording rules, but baked into the storage engine itself rather than layered on top.

Configuration with TOML

The file internal/engine/default_engine.toml provides the default engine configuration. Go doesn’t ship a TOML parser, so NanotDB uses BurntSushi/toml, with the default config embedded and written to <root>/engine.toml on first start.

TOML makes sense here. More readable than JSON for configuration, less footgun-prone than YAML. The engine config controls the HTTP listen address, WAL segment size and fsync policy, durability profile, stats interval, startup databases, and the manifest defaults copied into newly created databases. Per-database manifest.toml files then hold retention, partitioning, page limits, WAL settings, and rollup jobs.

The CLI: nanocli

The CLI in cmd/nanocli/ is broken into focused files, one per command:

• cmd/nanocli/query.go — querying data
• cmd/nanocli/export.go and cmd/nanocli/import.go — data portability
• cmd/nanocli/inspect_dat.go, cmd/nanocli/inspect_db.go, cmd/nanocli/inspect_wal.go — inspecting internal storage structures
• cmd/nanocli/context.go — managing connection context
• cmd/nanocli/output.go — formatting output

The inspect commands are the ones I’d reach for first. Being able to poke at WAL files, data files, and the database catalog directly is great for debugging and for actually understanding what the storage engine is doing under the hood. If you’re building CLI tools in Go, this file-per-command structure is a clean pattern worth borrowing.

There’s also cmd/enginetester/main.go, which looks like a standalone binary for exercising the engine directly. I like this pattern for any Go project with complex internals. A dedicated test harness binary, separate from your main application and your unit tests, gives you a fast way to run integration-level checks without spinning up the whole system.

Why Go fits this problem

Go is a natural fit for something like NanotDB.

Go’s encoding/binary and raw byte slices give you direct control over byte layout without unsafe hacks. The code also uses sync.Pool around encoding buffers in the WAL and page paths, which is a practical way to reduce allocation pressure in a write-heavy system. If you’re targeting a Raspberry Pi or embedded Linux, Go’s GOOS/GOARCH flags let you cross-compile for ARM from your laptop. The project includes .github/workflows/release.yml, which builds release binaries for Raspberry Pi armv6, armv7, and arm64 targets. And of course: single binary deployment. No runtime dependencies. Copy it to your device and run it.

These properties matter when your target is a sensor node with limited RAM and storage. The interesting Go lesson here is not just that the language is fast enough. It’s that the standard library gives you the boring, explicit building blocks — files, byte order, checksums, time, and small concurrency primitives — that make a storage engine like this readable.

A good codebase to read on a Saturday afternoon

NanotDB doesn’t try to be a general-purpose database. It solves one problem — storing and querying time-series sensor data on small machines — and the Go implementation reflects that discipline. The internal/engine package keeps the storage engine private, the append-only model with WAL provides durability without complexity, and rollup aggregations make reads fast over long time ranges.

If you want to understand how databases work at a low level, the NanotDB source is a manageable place to start. It’s small enough to read in an afternoon, and the file organization maps directly to database concepts you can carry to larger systems. For more Go projects worth reading, check out our post on AList’s file server architecture.