Time series, capped collections, TTL and views

conceptThe concept

• Time series (5.0+): db.createCollection("metrics", { timeseries: { timeField: "ts", metaField: "meta", granularity: "minutes" }, expireAfterSeconds: 2592000 }). The server buckets measurements by meta and time internally, compresses them by column, and creates a clustered index on time. You insert and query individual measurements as usual.
• Capped: fixed size in bytes (and optionally documents), insertion order preserved, oldest documents overwritten. Good for logs and tailable cursors, but cannot be sharded, and deletes are not allowed.
• TTL: an index option that deletes documents after a date field plus N seconds.
• Views: db.createView("v", "source", pipeline); read-only, computed on each query, inherit source indexes for the leading $match. Use them to hide fields or present a reshaped model.

whyWhy it matters

Metrics, IoT and event data are the fastest growing data in most companies. Storing them as one document per measurement in a regular collection wastes space and index RAM; time series collections typically reduce storage by an order of magnitude and speed up range queries over time.

Use caseReal-world scenario

Application latency samples per service per minute go into a time series collection with metaField: "service" and a 30-day expiry. Dashboards query the last hour per service; a nightly $merge rolls up daily percentiles into a regular collection kept for a year.

Try itExamples

Run and edit these in the interactive lesson.

db.lab_metrics.drop()
db.createCollection("lab_metrics", { timeseries: { timeField: "ts", metaField: "meta", granularity: "minutes" }, expireAfterSeconds: 2592000 })
db.events.aggregate([
  { $match: { type: { $in: ["checkout", "search"] } } },
  { $project: { _id: 0, ts: 1, meta: { service: "$type", device: "$device" }, latencyMs: 1 } },
  { $merge: { into: "lab_metrics" } }
])
db.lab_metrics.aggregate([
  { $match: { "meta.service": "checkout" } },
  { $group: { _id: { device: "$meta.device", day: { $dateTrunc: { date: "$ts", unit: "day" } } }, avg: { $avg: "$latencyMs" }, max: { $max: "$latencyMs" }, n: { $sum: 1 } } },
  { $sort: { "_id.day": -1, "_id.device": 1 } },
  { $limit: 4 }
])
db.getCollectionInfos({ name: "lab_metrics" })[0].type

db.lab_customers_public.drop()
db.createView("lab_customers_public", "customers", [{ $project: { name: 1, tier: 1, "address.city": 1 } }])
db.lab_customers_public.find({ tier: "platinum" }).limit(3)

db.lab_customers_public.insertOne({ name: "x" })

internalsUnder the hood

Internally a time series collection is a view over a system.buckets.<name> collection. Each bucket holds measurements for one meta value within a time span determined by granularity (or bucketMaxSpanSeconds), with min/max per field so queries can skip buckets. Choose the metaField to be the identifier you filter by (device id, service); high-cardinality metadata creates many small buckets and loses the benefit.

Avoid / preferDo this, not that

db.createCollection("metrics")
db.metrics.createIndex({ host: 1, ts: 1 })   // one document per reading

db.createCollection("metrics", {
  timeseries: { timeField: "ts", metaField: "host", granularity: "seconds" },
  expireAfterSeconds: 2592000
})

timeseries: { timeField: "ts", metaField: "requestId" }

timeseries: { timeField: "ts", metaField: { host: 1, region: 1 } }   // conceptually: stable labels

PitfallsCommon mistakes

• Using a unique id per measurement (request id) as the metaField: every bucket holds one measurement.
• Updating individual measurements frequently; time series collections are optimized for append, with limited update and delete support.
• Using capped collections for data you might need to delete selectively or shard.
• Treating views as materialized; they rerun the pipeline every time.

LimitsWhen not to use it

Time series collections suit append-mostly measurements. Documents that represent entities and change often belong in normal collections. Capped collections silently overwrite the oldest documents, so never use them for anything you must keep.

SecuritySecurity notes

expireAfterSeconds is a data-retention control: document it in your retention policy so that auditors and engineers agree on how long readings live. Meta fields often hold hostnames and IP addresses, which may themselves be sensitive.

ChallengeTry it yourself

db.lat_lab.drop()
db.createCollection("lat_lab", { timeseries: { timeField: "ts", metaField: "device", granularity: "minutes" } })
db.lat_lab.insertMany(db.events.find({}, { _id: 0, ts: 1, device: 1, latencyMs: 1 }).toArray())
db.lat_lab.aggregate([
  { $group: { _id: "$device", avgMs: { $avg: "$latencyMs" }, readings: { $sum: 1 } } },
  { $project: { avgMs: { $round: ["$avgMs", 1] }, readings: 1 } },
  { $sort: { _id: 1 } }
])

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.