BM25 keyword search

BM25 is the relevance-ranking algorithm at the heart of Azure DocumentDB full-text search. This article shows how to create a search index covering one text field and run a BM25-scored $search + text query against it. For the differences between preview full-text search and generally available text indexes, see Full-text search and text indexes.

Note

Full-text search is in preview. Before using these examples, enable full-text search on your cluster.

What is BM25?

BM25 (Best Match 25) ranks each document by how well its terms match the query, taking three signals into account:

  • Term frequency. Documents that mention a query term more often score higher, with diminishing returns so that a single repeated keyword can't dominate.
  • Inverse document frequency. Rare terms across the corpus carry more weight than common ones, so matching bracket is worth more than matching the.
  • Document length normalization. Long documents aren't unfairly penalized for repeating terms, and short documents aren't unfairly rewarded.

The algorithm is well-studied, defaults are reasonable, and the score returned by Azure DocumentDB is monotonic (higher means more relevant within a single query). Scores are not directly comparable across different queries.

// ❌ Community MongoDB text-index shape. Not the Azure DocumentDB FTS path.
//    The new $search engine does not consume this form.
db.products.createIndex({ title: "text" });
// ✅ Define a search index with createSearchIndexes.
//    Always set dynamic: false and enumerate fields explicitly.
db.runCommand({
  createSearchIndexes: "products",
  indexes: [
    {
      name: "idx_title_standard",
      definition: {
        mappings: {
          dynamic: false,
          fields: {
            title: { type: "string" }
          }
        }
      }
    }
  ]
});

Name the index after the field and intent so it's easy to reference from $search. The index builds asynchronously. Confirm it's ready with the $listSearchIndexes stage shown below before issuing $search queries against it.

Listing and inspecting search indexes

Search indexes don't appear in the output of db.collection.getIndexes(), which only lists regular indexes. Use the $listSearchIndexes aggregation stage to inspect them:

// ✅ List all search indexes on the collection.
db.products.aggregate([
  { $listSearchIndexes: {} }
]);

// Inspect a specific search index by name.
db.products.aggregate([
  { $listSearchIndexes: { name: "idx_title_standard" } }
]);

Each result includes the index name, its latestDefinition, status, and queryable state. Wait until the index reports status: "READY" and queryable: true before issuing $search queries.

Running a $search + text query

// ❌ Regex substring search on a large collection: COLLSCAN, unranked,
//    case-sensitive without the /i flag.
db.products.find({ title: { $regex: "bracket", $options: "i" } });
// ✅ BM25 keyword search with $search + text.
//    Always pass index: "<name>" and cap with a downstream $limit.
db.products.aggregate([
  {
    $search: {
      index: "idx_title_standard",
      text: {
        query: "bracket",
        path: "title"
      }
    }
  },
  { $limit: 20 },
  {
    $project: {
      _id: 0,
      title: 1,
      score: { $meta: "searchScore" }
    }
  }
]);

Three rules apply to every $search + text query:

  • $search is the first stage of the aggregation pipeline.
  • index: "<name>" is set explicitly so the engine knows which search index to use.
  • The result set is capped with a downstream { $limit: N } stage. There is no count or limit field inside $search.

The text operator accepts a single nonempty query string. Set matchCriteria: "any" to match any analyzed query term or "all" to require all analyzed query terms. The path can be a field name, a named multi analyzer mapping, or an explicit array of either form. Every path must be present in the selected index.

For a complete multi-field example, see Structured and text search. To configure case normalization, prefix matching, or language-specific processing, see Custom analyzers.

Working with scores

Every document returned by $search carries a BM25 score that you read through { $meta: "searchScore" }. Project it whenever ranking matters:

db.products.aggregate([
  { $search: { index: "idx_title_standard", text: { query: "bracket", path: "title" } } },
  { $limit: 50 },
  { $project: { _id: 1, title: 1, score: { $meta: "searchScore" } } },
  { $match: { score: { $gte: 1.0 } } }
]);

$search returns results in BM25-descending order by default, so an explicit $sort is unnecessary unless a downstream stage disturbs that ordering. Use a minimum-score $match to drop low-relevance hits and a fixed $limit to bound payload size. To order by mapped scalar fields inside $search, see Sort and limit search results.

Combining with filters

To combine text matching with exact values or ranges inside $search, map the structured fields in the same search index and use compound. See Structured and text search for a complete index and query.

You can also apply a downstream $match, including when the fields aren't mapped in the search index. Apply the filter before the final $limit so eligible matches aren't discarded by an earlier candidate cutoff:

db.products.aggregate([
  { $search: { index: "idx_title_standard", text: { query: "bracket", path: "title" } } },
  { $match: { inStock: true, price: { $lte: 100 } } },
  { $limit: 20 },
  { $project: { _id: 0, title: 1, price: 1, score: { $meta: "searchScore" } } }
]);