Wide-Column Stores & Bigtable Architecture

A Bigtable-style database is a distributed, sparse, sorted map. It is designed for enormous keyed datasets and predictable single-row or contiguous-row-range access—not for relational joins or arbitrary ad hoc predicates.

“Wide-column” describes its logical grouping, not an analytic columnar file layout. Rows may have millions of possible qualifiers while storing only the cells that exist. Column families define operational policy; qualifiers inside a family are data. The row key orders and partitions the entire table.

The logical model #

The original paper defines a cell by

$$(row\ key,\ column\ family{:}qualifier,\ timestamp) \rightarrow bytes.$$

Rows are stored in lexicographic row-key order. A row can contain different qualifiers from its neighbors, and one cell can retain multiple timestamped versions. Missing cells consume no logical row slot, which makes the model sparse.

Column families should be few and stable because they carry configuration such as retention, access, or storage policy. Qualifiers can be numerous and dynamic. In a time-series row, metrics:cpu and metrics:rss are two qualifiers in one family; creating a family per metric confuses schema policy with application data.

Mutations within one row are atomic. That makes the row a transaction and contention boundary. Operations spanning rows do not inherit general ACID transactions, so a schema that requires cross-row atomicity is fighting the model.

The row key is the query plan #

Because data is ordered by row key, efficient reads are:

• an exact row-key lookup;
• a contiguous row-key range scan;
• a prefix scan, which is a convenient way to specify a contiguous range.

Filtering after a broad scan does not repair a bad key. The system must still read or examine the range that the filter rejects. A useful design exercise is to write every latency-critical query first, then construct keys that place each query's required rows together.

Suppose telemetry is most often read for one device over a time interval. A key shaped like

<device-id>#<reverse-time-or-time-bucket>

keeps the device's interval contiguous. Putting raw timestamp first spreads adjacent device events through a global time range and makes a device query scatter. Putting a monotonically increasing timestamp last can concentrate all current writes at one end of a single device's range. A bucket, hash prefix, or reversed timestamp can distribute writes, but each additional shard creates another range that the reader must query and merge.

This is a deliberate trade:

$$write\ distribution \longleftrightarrow read\ fanout.$$

There is no universally balanced row key. The right key distributes the actual write workload while keeping the actual read workload bounded.

Tablets turn the sorted table into movable ranges #

The ordered keyspace is divided into contiguous tablets. Serving nodes own tablet assignments rather than fixed hash buckets. A tablet can split as it grows, and ownership can move for balancing or recovery without changing the logical row key.

The 2006 design used a three-level metadata lookup and stored tablet data in GFS. Current Cloud Bigtable retains the range-tablet and SSTable concepts but uses current Google infrastructure: frontends route requests, serving nodes process tablet traffic, and durable data is stored separately in Colossus. That separation lets a serving node be replaced without moving an owned copy of every durable byte first.

Clients still experience a routing cache. A split or reassignment can make a cached location stale; the client refreshes metadata and retries. This is why tablet movement should be treated as ordinary control-plane behavior rather than an application-visible schema change.

From a write to immutable sorted files #

A write is recorded in a durable log and applied to an in-memory ordered structure, commonly called a memtable. When the memtable reaches a threshold, it is frozen and flushed as an immutable sorted table file (SSTable). Later compaction reads overlapping SSTables and writes replacement files.

Immutability simplifies crash recovery and concurrent reads: published files do not change in place. The costs move elsewhere:

• reads may merge versions from the memtable and several SSTables;
• overwrites and deletes create newer records rather than immediately erasing older bytes;
• compaction consumes read bandwidth, write bandwidth, CPU, and temporary space;
• the log must be replayed after a crash to recover changes not yet flushed.

This is the same broad write-optimized family as an LSM tree. Bigtable adds a distributed range-partitioning and serving layer around it.

Reads, versions, and deletion #

A read resolves the newest visible value by merging current memory with the relevant immutable files. Bloom filters and per-file indexes can avoid reading files that cannot contain a requested row/column, but they cannot eliminate work caused by severe overlap, excessive versions, or a broad range query.

Deletes are tombstones. Version-count and age policies make older values eligible for garbage collection. Physical deletion is asynchronous and occurs during compaction, so eligible data can remain present—and may remain readable unless separately filtered—for a period after the policy threshold.

Keeping many timestamp versions is useful for audit, time travel, and resolving late writes, but it multiplies read and compaction work. Retain versions because a query needs them, not merely because the model permits them.

Replication and consistency #

A single cluster provides a strongly consistent view of row mutations. Multi-cluster replication changes the boundary: writes are copied between clusters asynchronously, so default multi-cluster routing can return an older replica immediately after a write. Routing an application's related reads and writes to one cluster can preserve read-your-writes within that route, at the cost of automatic cross-cluster failover for those requests.

Replication therefore requires an application decision:

• choose automatic multi-cluster routing when availability and geographic proximity outweigh immediate cross-cluster visibility;
• choose single-cluster routing when the workflow requires a strong local session boundary;
• design conflict behavior explicitly if multiple clusters accept writes to the same row.

Replication protects availability. It is not a substitute for schema-level atomicity, backups, or a recovery plan for accidental logical deletion.

Diagnosing behavior by layer #

Start with the requested key range, not CPU. A healthy cluster-wide average can coexist with one saturated tablet, and adding nodes may not help until the hot range splits or the schema distributes it.

Fit and limits #

The model fits workloads with huge sparse tables, high sustained write rates, known access paths, per-entity atomicity, and natural key-range scans: telemetry, time series, personalization state, indexes, counters with carefully controlled contention, and feature serving are common examples.

It is a poor fit when the application needs relational joins, many unrelated secondary access paths, cross-row transactions, arbitrary predicates, or analytics that scan a few columns across most rows. Those are not missing syntax features; they require different data placement and coordination. A Bigtable schema succeeds when the row key makes the important query local and the write load makes that locality sustainable.