Capacity planning: working set, RAM, disk and IOPS

conceptThe concept

The working set is the data and indexes touched by the workload in a typical period: the hot indexes in full, plus the recently accessed documents. When it fits in the WiredTiger cache (plus the OS cache for compressed pages), reads are served from memory; when it does not, latency becomes disk latency.

• Data size = documents x average BSON size (use $bsonSize or stats().avgObjSize), on disk divided by the compression ratio (often 2-4x with snappy/zstd, measure yours).
• Index size = number of entries x key size (multikey: per element). Hot indexes should fit in cache entirely.
• IOPS: writes x (1 + number of indexes) for random IO in the worst case, journal writes, checkpoint bursts, and cache misses on reads.
• Headroom: plan to run below 60-70% of CPU, disk throughput and cache at peak, and keep disk usage below ~70% so compaction, resync and growth have space.

Sizing calculator

Open the interactive lesson to use this diagram.

whyWhy it matters

Under-sizing shows up as cache pressure and latency; over-sizing wastes money for years. Explicit assumptions (growth per month, hot fraction, peak multiplier) make the estimate reviewable and easy to update when reality differs.

Use caseReal-world scenario

An order system expects 2 million orders a month at 1.5 KB each, retained for 3 years, with 6 indexes. The last 90 days are hot. Hot data: 6 M x 1.5 KB = about 8.4 GB plus hot index ranges; total indexes 72 M x 6 x ~40 B = about 16 GB, of which the recent, right-leaning portions are hot. A 64 GB node (about 31 GB cache) fits the hot set with headroom; total data after 3 years is about 100 GB uncompressed, about 40 GB on disk at 2.5x compression. Sharding is not needed for capacity; it might be needed later for write throughput.

Try itExamples

Run and edit these in the interactive lesson.

["orders", "customers", "products", "reviews", "events"].map(c => {
  const s = db.getCollection(c).stats()
  return { coll: c, count: s.count, avgObjSize: s.avgObjSize, dataMB: +(s.size / 1048576).toFixed(2), indexMB: +(s.totalIndexSize / 1048576).toFixed(2), indexes: s.nindexes }
})

const ordersPerMonth = 2e6, avgBytes = 1500, months = 36, hotMonths = 3, compression = 2.5
const totalGB = ordersPerMonth * months * avgBytes / 1024 ** 3
const hotGB = ordersPerMonth * hotMonths * avgBytes / 1024 ** 3
printjson({ totalDataGB: +totalGB.toFixed(1), onDiskGB: +(totalGB / compression).toFixed(1), hotDataGB: +hotGB.toFixed(1) })

internalsUnder the hood

Cache misses are not uniform: a B-tree index lookup touches the root and internal pages (almost always cached) and one leaf page; a document fetch touches one data page. Right-leaning indexes (ObjectId, timestamps) keep the hot portion small; random keys (UUIDv4, hashed) spread the hot set across the entire index, which is why random-key indexes need to fit entirely in RAM to perform well.

Avoid / preferDo this, not that

// 2 TB of data, so buy 2 TB of RAM

// RAM ≈ indexes in active use + hot documents (say the last 30 days)
//       + connection and query memory + headroom

db.events.insertOne({ _id: UUID(), type: "page_view" })

db.events.insertOne({ type: "page_view" })   // ObjectId _id

PitfallsCommon mistakes

• Sizing by total data instead of working set.
• Ignoring index size, especially multikey and random-key indexes.
• Benchmarking with a tiny dataset that fits in cache, then going live with 50x the data.
• Running disks at 90% full: no room for compaction, initial sync or emergency growth.

LimitsWhen not to use it

Synthetic benchmarks with uniform random data mislead in both directions. Plan from real peak load (not averages) and for N-1: the cluster must carry peak traffic with one member down.

SecuritySecurity notes

Capacity is availability. Keep headroom for a member failure and for traffic spikes, and cap connections with net.maxIncomingConnections so one runaway client cannot exhaust the server.

ChallengeTry it yourself

const s = db.stats()
printjson({ dataMB: +(s.dataSize / 1048576).toFixed(2), indexMB: +(s.indexSize / 1048576).toFixed(2), avgObjBytes: Math.round(s.avgObjSize) })

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.