GridFS and storing large files

conceptThe concept

A BSON document is limited to 16 MB. GridFS is a convention, implemented by drivers and the mongofiles tool, for storing larger files in a bucket (named fs by default) made of two collections:

• fs.files: one document per file with _id, length, chunkSize, uploadDate, filename and optional metadata.
• fs.chunks: one document per piece with files_id, n (0-based sequence number) and data (binary). The default chunk size is 255 KiB, so a 1 MB file has four chunks.

Drivers create a unique index on fs.chunks { files_id: 1, n: 1 } and an index on fs.files { filename: 1, uploadDate: 1 }. Reading a file streams the chunks in n order; a download can start at any byte offset by skipping to the right chunk, which is why GridFS suits video seeking and partial reads.

Files are immutable: "updating" a file means uploading a new one with the same filename. The newest revision is the one with the latest uploadDate.

whyWhy it matters

Teams reach for GridFS when a document would exceed 16 MB or when they want files in the same backup and replication as the data. Both are valid, but every byte of every file then goes through the oplog, the WiredTiger cache, backups and initial sync. Knowing the cost lets you choose deliberately.

Use caseReal-world scenario

An on-premises document management system must keep scanned contracts in the same data centre as the metadata, with no object store available. GridFS keeps files and metadata in one replica set with one backup process. A nightly check finds files whose chunk count does not match their length, which catches interrupted uploads before a user does.

Try itExamples

Run and edit these in the interactive lesson.

db.lab_fs.files.drop()
db.lab_fs.chunks.drop()
db.lab_fs.files.createIndex({ filename: 1, uploadDate: 1 })
db.lab_fs.chunks.createIndex({ files_id: 1, n: 1 }, { unique: true })
db.lab_fs.chunks.getIndexes().map(i => i.name)

db.lab_fs.files.drop()
db.lab_fs.chunks.drop()
const chunkSize = 261120
const length = 614400
db.lab_fs.files.insertOne({ _id: 1, filename: "contract-1001.pdf", length: length, chunkSize: chunkSize, uploadDate: ISODate("2026-10-01T09:00:00Z"), metadata: { customerId: 66, type: "contract" } })
const n = Math.ceil(length / chunkSize)
for (let i = 0; i < n; i++) db.lab_fs.chunks.insertOne({ files_id: 1, n: i, data: "chunk-" + i })
db.lab_fs.chunks.find({ files_id: 1 }, { _id: 0, files_id: 1, n: 1 }).sort({ n: 1 })

db.lab_fs.files.drop()
db.lab_fs.chunks.drop()
db.lab_fs.files.insertMany([
  { _id: 1, filename: "a.pdf", length: 614400, chunkSize: 261120, uploadDate: ISODate("2026-10-01") },
  { _id: 2, filename: "b.pdf", length: 600000, chunkSize: 261120, uploadDate: ISODate("2026-10-02") }
])
db.lab_fs.chunks.insertMany([
  { files_id: 1, n: 0, data: "x" }, { files_id: 1, n: 1, data: "x" }, { files_id: 1, n: 2, data: "x" },
  { files_id: 2, n: 0, data: "x" }, { files_id: 2, n: 1, data: "x" }
])
db.lab_fs.files.aggregate([
  { $lookup: { from: "lab_fs.chunks", localField: "_id", foreignField: "files_id", as: "c" } },
  { $project: { filename: 1, expected: { $ceil: { $divide: ["$length", "$chunkSize"] } }, stored: { $size: "$c" } } },
  { $match: { $expr: { $ne: ["$expected", "$stored"] } } }
])

db.lab_fs.chunks.insertOne({ files_id: 99, n: 0, data: "x" })
db.lab_fs.chunks.aggregate([
  { $group: { _id: "$files_id", chunks: { $sum: 1 } } },
  { $lookup: { from: "lab_fs.files", localField: "_id", foreignField: "_id", as: "f" } },
  { $match: { f: { $size: 0 } } },
  { $project: { _id: 0, files_id: "$_id", chunks: 1 } }
])

db.lab_fs.files.deleteOne({ _id: 3 })
db.lab_fs.files.insertOne({ _id: 3, filename: "a.pdf", length: 1000, chunkSize: 261120, uploadDate: ISODate("2026-10-05") })
db.lab_fs.files.find({ filename: "a.pdf" }, { filename: 1, uploadDate: 1 }).sort({ uploadDate: -1 }).limit(1)

const { MongoClient, GridFSBucket } = require("mongodb");
const fs = require("fs");
const client = await MongoClient.connect(uri);
const bucket = new GridFSBucket(client.db("docs"), { bucketName: "contracts" });

// upload
await new Promise((ok, fail) => fs.createReadStream("contract-1001.pdf")
  .pipe(bucket.openUploadStream("contract-1001.pdf", { metadata: { customerId: 66 } }))
  .on("finish", ok).on("error", fail));

// download bytes 1 MB to 2 MB only
bucket.openDownloadStreamByName("contract-1001.pdf", { start: 1048576, end: 2097152 }).pipe(res);

mongofiles --uri "mongodb://backup@db1.example.com/docs?authSource=admin&tls=true" --prefix contracts put contract-1001.pdf
mongofiles --uri "..." --prefix contracts list
mongofiles --uri "..." --prefix contracts get contract-1001.pdf --local /tmp/contract.pdf

internalsUnder the hood

Each chunk is a normal document, so a 2 GB file is about 8,000 documents and 2 GB of oplog on upload. Chunks of recently read files occupy the WiredTiger cache, competing with the working set. When sharding a large bucket, shard fs.chunks on { files_id: 1, n: 1 } so one file's chunks stay together, and usually leave fs.files unsharded because it is small.

Deleting a file removes the files document and then its chunks. If a process dies in between, or an upload is interrupted, you get orphaned or missing chunks; drivers do not repair them.

Avoid / preferDo this, not that

db.uploads.insertOne({ name: "scan.pdf", bytes: hugeBinary })   // 30 MB

// > 16 MB: GridFS (or object storage) and a reference
bucket.openUploadStream("scan.pdf", { metadata: { ownerId } })

db.fs.files.deleteOne({ _id: id })   // chunks left behind

bucket.delete(id)   // driver removes the files document and its chunks
// plus a periodic orphan check on fs.chunks

PitfallsCommon mistakes

• Storing files in GridFS that are a few KB, where a BinData field in a normal document is simpler and faster.
• Forgetting the unique { files_id, n } index on a hand-built bucket, so reads scan the chunks collection.
• Assuming deletes are atomic. An interrupted delete or upload leaves orphaned or missing chunks.
• Putting terabytes of media in the same replica set as transactional data, so backups, initial sync and the cache are dominated by files.

LimitsWhen not to use it

Do not use GridFS for small binaries (thumbnails, icons, signatures): a BinData field in a normal document is one read instead of two. Do not use it for large media libraries served to the internet: object storage plus a CDN is cheaper and keeps database backups and initial sync small.

SecuritySecurity notes

Files in GridFS inherit the database's access control, so a user with read on the bucket can download every file in it. Use one bucket per sensitivity level, or serve files only through an application that checks ownership in metadata. Scan uploads for malware before storing them, and remember that backups now contain every file.

ChallengeTry it yourself

db.lab_fs.files.drop()
db.lab_fs.chunks.drop()
db.lab_fs.files.insertMany([
  { _id: 1, filename: "a.pdf", length: 614400, chunkSize: 261120, uploadDate: ISODate("2026-10-01") },
  { _id: 2, filename: "b.pdf", length: 600000, chunkSize: 261120, uploadDate: ISODate("2026-10-02") }
])
db.lab_fs.chunks.insertMany([
  { files_id: 1, n: 0, data: "x" }, { files_id: 1, n: 1, data: "x" }, { files_id: 1, n: 2, data: "x" },
  { files_id: 2, n: 0, data: "x" }, { files_id: 2, n: 1, data: "x" }
])
db.lab_fs.files.aggregate([
  { $lookup: { from: "lab_fs.chunks", localField: "_id", foreignField: "files_id", as: "c" } },
  { $match: { $expr: { $ne: [{ $size: "$c" }, { $ceil: { $divide: ["$length", "$chunkSize"] } }] } } },
  { $project: { _id: 0, filename: 1 } }
])

InterviewQuestions you should be able to answer

QuizCheck your understanding

5 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.