WiredTiger: cache, eviction, checkpoints and the journal

conceptThe concept

Open the interactive lesson to use this diagram.

• Cache: by default the larger of 50% of (RAM - 1 GB) or 256 MB. Holds uncompressed B-tree pages of collections and indexes. The OS filesystem cache holds the compressed files as a second layer, which is why you should not give WiredTiger all the RAM.
• Journal (write-ahead log): each write is appended to the journal; group commits flush to disk every 100 ms (commitIntervalMs) or immediately for j: true writes. After a crash, MongoDB recovers from the last checkpoint plus the journal.
• Checkpoints: every 60 seconds WiredTiger writes a consistent snapshot of all dirty data to the data files. Between checkpoints, data on disk is the previous checkpoint plus the journal.
• MVCC: updates create new versions in an update chain on the in-memory page; readers see the version that matches their snapshot. Old versions needed by long-running snapshots move to the history store (WiredTigerHS.wt, 4.4+).
• Compression: snappy for collections by default (zstd and zlib available), prefix compression for indexes.

Write path, step by step

Open the interactive lesson to use this diagram.

whyWhy it matters

Most "MongoDB is suddenly slow" incidents are cache incidents: the working set no longer fits, dirty data accumulates faster than eviction can write it, or a long-running operation pins old versions. When thresholds are crossed, application threads are drafted into eviction, and query latency spikes across the board.

Open the interactive lesson to use this diagram.

Use caseReal-world scenario

A nightly batch updates 40 million documents. Dirty cache climbs to 20%, application threads start evicting, and p99 latency of the checkout API goes from 8 ms to 900 ms. The fix is throttling the batch (smaller batches, sleeps, w: "majority" to apply back-pressure), running it off-peak, and checking disk write throughput.

Try itExamples

Run and edit these in the interactive lesson.

const wt = db.serverStatus().wiredTiger
const c = wt.cache
print("cache max GB :", (c["maximum bytes configured"] / 1024 ** 3).toFixed(1))
print("used %       :", (100 * c["bytes currently in the cache"] / c["maximum bytes configured"]).toFixed(2))
print("dirty %      :", (100 * c["tracked dirty bytes in the cache"] / c["maximum bytes configured"]).toFixed(2))
wt.concurrentTransactions

const s = db.orders.stats()
printjson({ count: s.count, avgObjSize: s.avgObjSize, size: s.size, storageSize: s.storageSize, totalIndexSize: s.totalIndexSize, indexes: s.nindexes })

storage:
  dbPath: /var/lib/mongodb
  journal:
    commitIntervalMs: 100
  wiredTiger:
    engineConfig:
      cacheSizeGB: 24          # leave room for the OS page cache, connections and aggregation memory
    collectionConfig:
      blockCompressor: zstd    # better ratio than snappy, slightly more CPU
    indexConfig:
      prefixCompression: true

internalsUnder the hood

Each collection and index is a separate .wt file with its own B-tree. Pages in cache are larger than on disk because they are uncompressed and carry update chains. Reads that miss the cache cause disk reads ("pages read into cache"). Concurrency is controlled by read and write tickets (128 each by default before 7.0; 7.0+ adjusts them dynamically); when tickets are exhausted, operations queue, which shows up as globalLock.currentQueue or low available tickets.

Avoid / preferDo this, not that

storage:
  wiredTiger:
    engineConfig:
      cacheSizeGB: 28     # on a 32 GB host

# leave the default: 50% of (RAM - 1 GB), about 15.5 GB here

docker run --memory 4g mongo:8.0   # and assume the cache fits

docker run --memory 4g mongo:8.0 --wiredTigerCacheSizeGB 1.5

PitfallsCommon mistakes

• Setting cacheSizeGB to almost all of RAM: no room for the filesystem cache, connections (about 1 MB each), and in-memory sorts.
• Running multiple mongod processes on one host without lowering each cache size.
• Long-running cursors or transactions that pin old snapshots, growing the history store and cache pressure.
• Ignoring dirty cache percentage during bulk loads.

LimitsWhen not to use it

Do not tune eviction triggers, checkpoint intervals or other engine knobs without evidence from metrics. The defaults suit almost every workload, and a mistuned eviction setting causes the stalls it was meant to prevent.

SecuritySecurity notes

Community MongoDB does not encrypt data files at rest. MongoDB Enterprise and Percona Server for MongoDB offer an encrypted storage engine with keys in KMIP or Vault; otherwise rely on disk or volume encryption, and remember that backups need encryption too.

ChallengeTry it yourself

const c = db.serverStatus().wiredTiger.cache
const max = c["maximum bytes configured"]
printjson({
  cacheGB: +(max / 1073741824).toFixed(1),
  usedMB: +(c["bytes currently in the cache"] / 1048576).toFixed(1),
  usedPct: +(100 * c["bytes currently in the cache"] / max).toFixed(3),
  dirtyPct: +(100 * c["tracked dirty bytes in the cache"] / max).toFixed(3)
})

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.