Text indexes, $text and full-text search engines

conceptThe concept

A text index tokenizes string fields, removes stop words and stems words ("running" → "run"), storing one entry per term. Query with { $text: { $search: "..." } }:

• Words are OR-ed: "wireless headphones" matches either word.
• "\"noise cancelling\"" requires the phrase; -refurbished excludes a term.
• Relevance: { score: { $meta: "textScore" } } in projection and sort; weights boost fields such as titles.
• One text index per collection (it can cover several fields, or all string fields with "$**": "text").

Full-text search engines (Lucene-based $search via mongot, Elasticsearch, OpenSearch) add fuzzy matching (typos), autocomplete, synonyms, facets, highlighting, per-field analyzers and much better relevance tuning.

whyWhy it matters

Search boxes are everywhere. A text index is enough for simple keyword search over modest data with no extra infrastructure. Knowing its limits tells you when to bring in a search engine before users complain that "headphnes" returns nothing.

Use caseReal-world scenario

An internal admin tool searches product names and tags with a text index. The public storefront, which needs typo tolerance, autocomplete and facet counts, uses a Lucene-based search index synced from MongoDB.

Try itExamples

Run and edit these in the interactive lesson.

db.products.createIndex({ name: "text", tags: "text", category: "text" }, { weights: { name: 5, tags: 2 }, name: "products_text" })
db.products.find({ $text: { $search: "wireless speaker" } }, { name: 1, score: { $meta: "textScore" } }).sort({ score: { $meta: "textScore" } }).limit(5)

db.products.find({ $text: { $search: "\"desk lamp\"" } }, { name: 1 }).limit(5)
db.products.find({ $text: { $search: "coffee -beans" } }, { name: 1 }).limit(5)

db.products.find({ $text: { $search: "running" } }, { name: 1 }).limit(3)
db.products.find({ $text: { $search: "runs" } }, { name: 1 }).limit(3)

db.products.aggregate([
  { $match: { $text: { $search: "pro" } } },
  { $group: { _id: "$category", matches: { $sum: 1 }, avgPrice: { $avg: "$price" } } },
  { $sort: { matches: -1 } }
])

internalsUnder the hood

Text indexes are multikey-like: one key per (term, document) with a weight. They can be large and slow to build, and every write to an indexed field updates many keys. A $text query must use the text index (you cannot hint another index) and, in a compound text index, equality conditions on prefix fields are required. Lucene-based search runs as a separate process (mongot) that reads change streams to keep its indexes eventually consistent with the collection.

Avoid / preferDo this, not that

db.products.find({ name: { $regex: "speaker", $options: "i" } })

db.products.createIndex({ name: "text", tags: "text" })
db.products.find({ $text: { $search: "speaker" } })

db.products.find({ $text: { $search: "wireless" } })

db.products.find({ $text: { $search: "wireless" } }, { score: { $meta: "textScore" } })
  .sort({ score: { $meta: "textScore" } })

PitfallsCommon mistakes

• Expecting typo tolerance or prefix (autocomplete) matching from a text index.
• Creating a second text index on the same collection (error); put all fields in one.
• Using $regex with leading wildcards for search over large collections.
• Forgetting language settings (default English stemming applied to other languages).

LimitsWhen not to use it

$text has no fuzzy matching, autocomplete, synonyms or facets, and only one text index per collection. For a real product search box use a search engine (Atlas Search, or Elasticsearch / OpenSearch fed by change streams). $text fits simple keyword search over modest data.

SecuritySecurity notes

A search box is untrusted input. $text treats the string as words, not operators, but still cap its length and rate-limit the endpoint. Do not add private fields (emails, notes) to a text index behind a public search API: anything indexed becomes searchable by whoever calls it.

ChallengeTry it yourself

db.search_lab.drop()
db.products.aggregate([{ $project: { name: 1, tags: 1 } }, { $out: "search_lab" }])
db.search_lab.createIndex({ name: "text", tags: "text" })
db.search_lab.find({ $text: { $search: "wireless" } }, { _id: 0, name: 1, score: { $meta: "textScore" } })
  .sort({ score: { $meta: "textScore" } })
  .limit(3)

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.