The Shard Key Trap

Sharding distributes data across multiple servers. The shard key determines which server holds each document. A good shard key distributes data evenly and routes queries to a single shard. A bad shard key creates hot spots, forces scatter-gather queries, and cannot be changed without resharding the entire collection.

The shard key is the most consequential design decision in a sharded MongoDB deployment. It cannot be changed after the collection is sharded (before MongoDB 5.0). Even with resharding (5.0+), the operation takes hours to days for large collections.

Three Properties of a Good Shard Key

1. High Cardinality. The shard key must have enough distinct values to spread data across all shards. A boolean field has 2 values: data can only go to 2 chunks. The sensorId field with 10,000 distinct values can spread across 10,000 chunks.

2. Low Frequency. No single shard key value should dominate. If 50% of documents have sensorId: "sensor-00001", that sensor’s data cannot be split across shards. All 50% lives on one shard.

3. Non-Monotonic. Monotonically increasing values (timestamps, auto-incrementing IDs, ObjectIds) direct all new inserts to the shard that owns the current maximum range. This creates a write hot spot on a single shard.

The Telemetry Platform’s Shard Key Decision

The collection must be sharded because it exceeds 2 TB and a single replica set cannot serve the 5,000 reads/sec and 2,000 writes/sec with acceptable latency.

Candidate 1: {sensorId: 1}

Cardinality: 10,000 distinct values. Frequency: roughly equal (each sensor produces the same volume). Monotonicity: not monotonic. Seems perfect.

Problem: time-range queries. The dashboard query “readings from sensor-00042, last hour” targets a single shard. Good. The analytics query “all readings from the last hour across all sensors” must scatter to every shard because data for different sensors is distributed across all shards. With 8 shards, every analytics query requires 8 sub-queries and a merge.

Candidate 2: {ts: 1} (timestamp)

Time-range queries target a contiguous range of shards. Analytics queries on the last hour hit only the shard with the most recent data.

Problem: monotonic. All new writes go to the shard owning the current time range. One shard handles 100% of writes. This creates a write hot shard that is saturated while 7 others are idle.

Candidate 3: {sensorId: 1, ts: 1} (compound)

Combines the benefits of both. Single-sensor time-range queries target one shard. Cross-sensor analytics queries scatter, but the compound key distributes writes across all shards because the first component (sensorId) is not monotonic.

// Shard the collection with compound key
database.runCommand(new Document("shardCollection", "telemetry.readings")
    .append("key", new Document("sensorId", 1).append("ts", 1)));