Sharding operations in 7.0 and 8.0

conceptThe concept

The balancer today. Since 6.0.3 the balancer moves data to even out data size per shard, not the number of chunks, and the server no longer splits chunks automatically as they grow. A collection is balanced when the difference between the largest and smallest shard is less than three times the range size (128 MB by default).

AutoMerger (7.0). Migrations and older split behaviour leave many small, contiguous chunks on the same shard. The AutoMerger runs while the balancer is enabled (every hour by default) and merges contiguous chunks on a shard that have not moved recently. Control it with sh.startAutoMerger(), sh.stopAutoMerger(), and per collection with sh.enableAutoMerger(ns) and sh.disableAutoMerger(ns). mergeAllChunksOnShard does the same on demand.

Defragmentation (configureCollectionBalancing with defragmentCollection: true) goes further for a collection with a very high chunk count: it merges, moves small chunks next to their neighbours and merges again. Progress shows in sh.balancerCollectionStatus(ns).

8.0 collection placement. An unsharded collection no longer has to live on the database's primary shard. moveCollection moves it to another shard; unshardCollection turns a sharded collection back into an unsharded one on a single shard. Both use the resharding machinery: copy, catch up from the oplog, then block writes for a short commit (about two seconds).

Config shard (8.0). transitionFromDedicatedConfigServer lets the config server replica set also hold user data as a shard named config. A small cluster saves three servers; transitionToDedicatedConfigServer moves the data off again when the cluster grows.

whyWhy it matters

Many runbooks were written for 4.x clusters: they watch chunk counts, disable autosplit and move chunks by hand. On 7.0 and 8.0 the tools are different, and some old habits now hurt, such as balancing by chunk count or running moveChunk loops while the balancer is also working. The 8.0 commands also remove two classic reasons to rebuild a cluster: a collection stuck on an overloaded primary shard, and a collection that was sharded by mistake.

Use caseReal-world scenario

A 4-shard cluster has 60 unsharded collections, all on shard-a because it is the primary shard of the database. shard-a uses 70 % more disk and CPU than the others. On 8.0 the team runs moveCollection for the 10 busiest unsharded collections, spreading them across shard-b to shard-d during a low-traffic window, one at a time, watching replication lag. A collection called audit_2021, sharded years ago on a poor key and now read once a month, is returned to one shard with unshardCollection.

Try itExamples

Run and edit these in the interactive lesson.

const s = sh.status()
;({ shards: s.shards.map(x => x._id), balancer: s.balancer["Currently enabled"], orders: s.databases[0].collections["shop.orders"].chunkMetadata })

db.lab_chunks.drop()
db.lab_chunks.insertMany([
  { ns: "shop.events", min: 0, max: 100, shard: "shard-a" },
  { ns: "shop.events", min: 100, max: 200, shard: "shard-a" },
  { ns: "shop.events", min: 200, max: 300, shard: "shard-a" },
  { ns: "shop.events", min: 300, max: 400, shard: "shard-b" },
  { ns: "shop.events", min: 400, max: 500, shard: "shard-b" },
  { ns: "shop.events", min: 500, max: 600, shard: "shard-a" }
]).acknowledged
db.lab_chunks.aggregate([
  { $setWindowFields: { partitionBy: "$ns", sortBy: { min: 1 }, output: { prevShard: { $shift: { output: "$shard", by: -1 } } } } },
  { $set: { startsRun: { $ne: ["$shard", "$prevShard"] } } },
  { $setWindowFields: { partitionBy: "$ns", sortBy: { min: 1 }, output: { run: { $sum: { $cond: ["$startsRun", 1, 0] }, window: { documents: ["unbounded", "current"] } } } } },
  { $group: { _id: "$run", shard: { $first: "$shard" }, chunks: { $sum: 1 }, min: { $min: "$min" }, max: { $max: "$max" } } },
  { $match: { chunks: { $gt: 1 } } },
  { $project: { _id: 0, shard: 1, chunks: 1, range: { $concat: [{ $toString: "$min" }, "-", { $toString: "$max" }] } } },
  { $sort: { range: 1 } }
])

db.getCollectionNames()
  .filter(n => !n.startsWith("lab_") && !n.startsWith("system."))
  .map(n => ({ coll: n, docs: db[n].countDocuments(), mb: Math.round(db[n].stats().totalSize / 1048576 * 100) / 100 }))
  .sort((a, b) => b.mb - a.mb)
  .slice(0, 4)

sh.isBalancerRunning()                    // the AutoMerger only runs while the balancer is enabled
sh.stopAutoMerger(); sh.startAutoMerger()
sh.disableAutoMerger("shop.events")       // per collection
db.adminCommand({ mergeAllChunksOnShard: "shop.events", shard: "shard-a" })

// a collection left with tens of thousands of small chunks
db.adminCommand({ configureCollectionBalancing: "shop.events", defragmentCollection: true })
sh.balancerCollectionStatus("shop.events")   // firstComplianceViolation: "defragmentingChunks" while it runs

// per-collection range size instead of the cluster-wide 128 MB
db.adminCommand({ configureCollectionBalancing: "shop.events", chunkSize: 256 })

// move an unsharded collection off the primary shard
db.adminCommand({ moveCollection: "shop.audit_log", toShard: "shard-c" })
db.getSiblingDB("admin").aggregate([{ $currentOp: { allUsers: true, localOps: false } }, { $match: { type: "op", "originatingCommand.reshardCollection": { $exists: true } } }])
db.adminCommand({ abortMoveCollection: "shop.audit_log" })   // if it must stop

// return a sharded collection to a single shard
db.adminCommand({ unshardCollection: "shop.audit_2021", toShard: "shard-d" })

// re-distribute with the same shard key (for example after adding shards)
db.adminCommand({ reshardCollection: "shop.events", key: { customerId: "hashed" }, forceRedistribution: true })

// small cluster: let the config server replica set hold data too
db.adminCommand({ transitionFromDedicatedConfigServer: 1 })
sh.listShards()   // a shard with _id "config" appears

// cluster grew: drain the user data off the config servers again
db.adminCommand({ transitionToDedicatedConfigServer: 1 })   // repeat until state is "completed"

db.getSiblingDB("admin").runCommand({ checkMetadataConsistency: 1 })   // a cursor of inconsistencies; empty is good
db.getSiblingDB("shop").runCommand({ checkMetadataConsistency: "orders" })

db.getSiblingDB("admin").aggregate([
  { $shardedDataDistribution: {} },
  { $match: { ns: "shop.orders" } }
])   // per shard: numOrphanedDocs, numOwnedDocuments, ownedSizeBytes, orphanedSizeBytes

internalsUnder the hood

moveCollection, unshardCollection and reshardCollection all build a full copy of the collection on the recipient shards, apply the oplog to catch up, and commit by briefly blocking writes. Each recipient needs free disk for the whole copy, and the donor's oplog must hold enough history for the copy to finish. Only one such operation runs per collection, and they can be aborted with abortMoveCollection, abortUnshardCollection and abortReshardCollection.

Chunk metadata lives in config.chunks on the config servers and is cached by every mongos and shard. checkMetadataConsistency (7.0) compares that metadata with what the shards actually have and lists problems such as orphaned collections or mismatched indexes.

Avoid / preferDo this, not that

// nightly cron
for (const c of chunksOn("shard-a")) sh.moveChunk("shop.events", c.min, "shard-b")

sh.balancerCollectionStatus("shop.events")
db.adminCommand({ configureCollectionBalancing: "shop.events", defragmentCollection: true })

db.adminCommand({ moveCollection: "shop.events", toShard: "shard-c" })   // 900 GB, 20 % free disk, peak hours

// check free disk on shard-c and the donor oplog window first
db.adminCommand({ moveCollection: "shop.audit_log", toShard: "shard-c" })   // smaller collection, quiet window

PitfallsCommon mistakes

• Monitoring chunk counts per shard on 7.0 and 8.0, where the balancer evens out data size instead.
• Running manual moveChunk or moveRange loops while the balancer is also working.
• Starting moveCollection or reshardCollection without free disk for a full copy and enough oplog window.
• Running several resharding-based operations at once during peak traffic.
• Using a config shard on a cluster that is already busy, so config metadata and user data compete for the same nodes.

LimitsWhen not to use it

Do not shard or move collections to solve a query problem; fix indexes and schema first. Do not defragment a collection whose chunk count causes no measurable problem: it is extra migration work. And do not use a config shard for clusters with strict isolation or heavy load, where dedicated config servers are worth the cost.

SecuritySecurity notes

moveCollection, unshardCollection, reshardCollection and the config shard transitions need cluster administration privileges (clusterManager or a custom role with the matching actions). Grant them to automation accounts used in change windows, not to application users, and audit their use; each one moves entire collections between servers.

ChallengeTry it yourself

db.lab_chunks.drop()
db.lab_chunks.insertMany([
  { ns: "shop.events", min: 0, max: 100, shard: "shard-a" },
  { ns: "shop.events", min: 100, max: 200, shard: "shard-a" },
  { ns: "shop.events", min: 200, max: 300, shard: "shard-a" },
  { ns: "shop.events", min: 300, max: 400, shard: "shard-b" },
  { ns: "shop.events", min: 400, max: 500, shard: "shard-b" },
  { ns: "shop.events", min: 500, max: 600, shard: "shard-a" }
]).acknowledged
db.lab_chunks.aggregate([
  { $group: { _id: "$shard", chunks: { $sum: 1 }, keys: { $sum: { $subtract: ["$max", "$min"] } } } },
  { $project: { _id: 0, shard: "$_id", chunks: 1, keys: 1 } },
  { $sort: { shard: 1 } }
])

InterviewQuestions you should be able to answer

QuizCheck your understanding

5 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.