Clustered collections

conceptThe concept

In a normal collection, documents live in a WiredTiger table keyed by an internal RecordId, and the _id index is a separate table that maps _id to RecordId. Every insert writes both, and a range query on _id walks the index and then fetches each document.

A clustered collection (5.3+) uses _id as the table's key: documents are stored in _id order and there is no separate _id index. It is created with db.createCollection(name, { clusteredIndex: { key: { _id: 1 }, unique: true } }); the key must be { _id: 1 } and the option cannot be added to an existing collection.

• Range queries on _id become a CLUSTERED_IXSCAN over the data itself, with no FETCH stage.
• Inserts write one table instead of two, and storage drops by the size of the _id index.
• expireAfterSeconds on the collection deletes documents whose _id is a date older than the cutoff, with no separate TTL index.
• Secondary indexes store the _id value instead of a small RecordId, so a large _id makes every secondary index bigger.

Time series collections already use clustered storage for their buckets; clustered collections expose the same idea for ordinary documents.

whyWhy it matters

Append-heavy collections that are read by time ranges, such as audit trails, event logs and ledgers, pay for an _id index they barely need and for random fetches on every range read. Clustering removes both costs with one option at creation time, which is why it is worth knowing before the collection exists.

Use caseReal-world scenario

A payments service writes an audit record for every state change, with an ObjectId _id (time-ordered), and reads "everything between 10:00 and 10:05" during investigations. As a clustered collection, inserts write one table, the range read scans contiguous data, and expireAfterSeconds is not used because audit data must be kept; retention is handled by archiving.

Try itExamples

Run and edit these in the interactive lesson.

db.lab_audit.drop()
db.createCollection("lab_audit", { clusteredIndex: { key: { _id: 1 }, unique: true } })
db.lab_audit.insertMany([{ action: "login" }, { action: "update" }, { action: "logout" }])
db.getCollectionInfos({ name: "lab_audit" })[0].options
db.lab_audit.getIndexes()

db.lab_bad.drop()
db.createCollection("lab_bad", { clusteredIndex: { key: { createdAt: 1 }, unique: true } })

db.lab_c.drop()
db.lab_n.drop()
db.createCollection("lab_c", { clusteredIndex: { key: { _id: 1 }, unique: true } })
const docs = []
for (let i = 1; i <= 200; i++) docs.push({ _id: i, v: i * 2 })
Object.keys(db.lab_c.insertMany(docs).insertedIds).length
Object.keys(db.lab_n.insertMany(docs).insertedIds).length
const stagesOf = p => { const s = []; for (let n = p; n; n = n.inputStage) s.push(n.stage); return s.join(" > ") }
const clustered = stagesOf(db.lab_c.find({ _id: { $gte: 50, $lt: 60 } }).explain().queryPlanner.winningPlan)
const normal = stagesOf(db.lab_n.find({ _id: { $gte: 50, $lt: 60 } }).explain().queryPlanner.winningPlan)
const result = { clustered: clustered, normal: normal }
result

db.lab_sessions.drop()
db.createCollection("lab_sessions", { clusteredIndex: { key: { _id: 1 }, unique: true }, expireAfterSeconds: 86400 })
db.lab_sessions.insertOne({ _id: new Date(), user: "asha" })
db.getCollectionInfos({ name: "lab_sessions" })[0].options

internalsUnder the hood

The clustered key is the WiredTiger table key, so documents with neighbouring _id values sit in neighbouring pages. Monotonically increasing keys (ObjectId, timestamps) append at the right edge and keep pages full. Random keys such as UUIDv4 insert across the whole table, splitting pages and spreading the working set, which is the same problem a random key causes in any B-tree. Keep _id small: it is copied into every secondary index entry.

Avoid / preferDo this, not that

db.createCollection("events", { clusteredIndex: { key: { _id: 1 }, unique: true } })
db.events.insertOne({ _id: UUID(), ... })   // random v4 key

db.createCollection("events", { clusteredIndex: { key: { _id: 1 }, unique: true } })
db.events.insertOne({ ... })   // default ObjectId, increasing over time

db.audit.createIndex({ at: 1 }, { expireAfterSeconds: 2592000 })   // on a normal collection

db.createCollection("audit", { clusteredIndex: { key: { _id: 1 }, unique: true }, expireAfterSeconds: 2592000 })
// _id holds the event time

PitfallsCommon mistakes

• Clustering on a random UUIDv4 _id, so every insert lands on a different page.
• Using a large _id (long strings, embedded documents) with many secondary indexes; every index entry carries the full _id.
• Planning to convert an existing collection. clusteredIndex can only be set at creation; migrate with $out or $merge into a new collection.
• Expecting clustering to speed up queries on other fields. It only changes how _id is stored.

LimitsWhen not to use it

Skip clustering when the _id is random or large, when most reads use other fields, or for small collections where the _id index costs nothing noticeable. Use a time series collection instead for high-volume measurements with a time field and a source identifier.

SecuritySecurity notes

Clustering does not change access control, but a date or sequential _id reveals when a record was created and how many exist. Do not expose such ids in public URLs if that information is sensitive; use a separate random public identifier.

ChallengeTry it yourself

db.lab_clicks.drop()
db.createCollection("lab_clicks", { clusteredIndex: { key: { _id: 1 }, unique: true } })
const rows = []
for (let i = 1; i <= 50; i++) rows.push({ _id: i, page: "/p/" + i })
Object.keys(db.lab_clicks.insertMany(rows).insertedIds).length
db.lab_clicks.find({ _id: { $gte: 10, $lt: 20 } }).explain().queryPlanner.winningPlan.stage

InterviewQuestions you should be able to answer

QuizCheck your understanding

5 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.