Missing fields, null, $type, $regex and $expr

conceptThe concept

• { phone: null } matches documents where phone is null or missing.
• { phone: { $exists: false } } matches only missing; { phone: { $type: "null" } } only explicit null.
• $regex: { name: /^Ar/ }. A case-sensitive regex anchored with ^ can use an index as a range scan; unanchored or case-insensitive regexes scan every index key or document.
• $expr lets you use aggregation expressions in a filter, for example comparing two fields: { $expr: { $gt: ["$discount", { $multiply: ["$total", 0.15] }] } }.
• $where runs JavaScript per document: slow, disabled in many deployments, avoid it.

whyWhy it matters

Null semantics are a classic source of bugs in reports ("customers without a phone" returns the wrong count). Regex and $expr are powerful but easy to make expensive, so you should know which forms can use indexes.

Use caseReal-world scenario

A data-quality job finds customers with no phone ($exists: false), orders whose discount exceeds 15% of the total ($expr), and emails that do not end with a valid domain ($not: /@example\.com$/).

Try itExamples

Run and edit these in the interactive lesson.

db.customers.countDocuments({ phone: null })
db.customers.countDocuments({ phone: { $exists: false } })
db.products.countDocuments({ "attributes.powerWatts": null })
db.products.countDocuments({ "attributes.powerWatts": { $type: "null" } })

db.customers.find({ name: /^Ar/ }, { name: 1, _id: 0 }).limit(5)
db.customers.countDocuments({ email: { $regex: "reddy", $options: "i" } })
db.products.find({ name: /pro$/i }, { name: 1, _id: 0 }).limit(5)

db.orders.find(
  { $expr: { $gt: ["$discount", { $multiply: ["$total", 0.099] }] } },
  { orderNo: 1, total: 1, discount: 1, _id: 0 }
).limit(3)
db.products.countDocuments({ $expr: { $lt: ["$stock", 5] } })

internalsUnder the hood

An index stores missing fields as null, so { field: null } can use an index, but the server must fetch documents to tell null from missing when it matters. $expr with simple equality on a field can use an index since 5.0; comparisons between two fields cannot, because the bound depends on each document.

Avoid / preferDo this, not that

db.customers.find({ email: /gmail/i })

db.customers.find({ email: /^ahmed/ })
// case-insensitive: a collation index plus collation on the query

db.orders.find({ $where: "this.total > this.items.length * 100" })

db.orders.find({ $expr: {
  $gt: ["$total", { $multiply: [{ $size: "$items" }, 100] }]
} })

PitfallsCommon mistakes

• Using { field: null } when you meant "explicitly null" or "missing" only.
• Unanchored, case-insensitive regex on large collections: a full scan every time. Store a normalized lowercase copy and query it with an anchored regex, or use a text or search index.
• Using $where for logic that $expr can express.

LimitsWhen not to use it

$regex is not full-text search: it cannot rank, stem or tokenize. Use a text index or a search engine for words. $expr comparing two fields of the same document cannot use index bounds, so pair it with a selective indexed condition.

SecuritySecurity notes

A regular expression built from user input can take exponential time (ReDoS). Escape user input before putting it in a pattern, cap its length, and consider security.javascriptEnabled: false to turn off $where, $function and $accumulator entirely.

ChallengeTry it yourself

({
  missing: db.customers.countDocuments({ tags: { $exists: false } }),
  empty: db.customers.countDocuments({ tags: { $size: 0 } })
})

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.