Client-side field level encryption and Queryable Encryption

conceptThe concept

With in-use encryption the driver encrypts fields before sending them and decrypts results; the server stores BinData ciphertext. Plaintext never reaches the server, logs, the oplog or backups.

• Customer master key (CMK): lives in a KMS (AWS KMS, Azure Key Vault, Google Cloud KMS, a KMIP server, or a local key for development).
• Data encryption keys (DEKs): generated per field or per tenant, stored encrypted by the CMK in a key vault collection (for example encryption.__keyVault).
• CSFLE (4.2+): deterministic encryption allows equality queries but reveals equal values; random encryption is stronger but not queryable.
• Queryable Encryption (GA in 7.0, range queries GA in 8.0): randomized encryption that still supports equality and range queries using encrypted metadata collections.

Automatic encryption (the driver reads a schema or encryptedFields map and encrypts transparently) requires MongoDB Enterprise or Atlas plus the crypt_shared library or mongocryptd. Explicit encryption, where code calls encrypt() per field, also works with Community Edition.

whyWhy it matters

Regulations and internal policy often require that DBAs, backup operators and anyone with a disk image cannot read card numbers, national IDs or health data. Encryption at rest protects the disk, TLS protects the wire, but only in-use encryption protects data from the database tier itself.

Use caseReal-world scenario

A lending platform encrypts pan and ssn with Queryable Encryption (equality) and income with range support. Support engineers can look up a customer by PAN through the application, while DBAs restoring a backup or reading the oplog see only ciphertext. Losing the CMK would make the data unrecoverable, so it lives in KMS with strict access and backups.

Try itExamples

Run and edit these in the interactive lesson.

const { MongoClient, ClientEncryption } = require("mongodb");
const kmsProviders = { aws: { accessKeyId: process.env.AWS_KEY, secretAccessKey: process.env.AWS_SECRET } };
const client = await MongoClient.connect(uri);
const enc = new ClientEncryption(client, { keyVaultNamespace: "encryption.__keyVault", kmsProviders });

const dekId = await enc.createDataKey("aws", { masterKey: { region: "ap-south-1", key: process.env.CMK_ARN }, keyAltNames: ["pan-key"] });
const pan = await enc.encrypt("ABCDE1234F", { keyId: dekId, algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" });
await client.db("lend").collection("customers").insertOne({ name: "Asha", pan });

// equality lookup works because the deterministic ciphertext is identical
const probe = await enc.encrypt("ABCDE1234F", { keyId: dekId, algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" });
await client.db("lend").collection("customers").findOne({ pan: probe });

const encryptedFields = {
  fields: [
    { path: "pan", bsonType: "string", queries: { queryType: "equality" } },
    { path: "income", bsonType: "int", queries: { queryType: "range", min: 0, max: 100000000 } }
  ]
};
// with automatic encryption enabled on the client:
await enc.createEncryptedCollection(db, "customers", { provider: "aws", createCollectionOptions: { encryptedFields }, masterKey });

db.lab_keyvault.drop()
db.lab_keyvault.createIndex({ keyAltNames: 1 }, { unique: true, partialFilterExpression: { keyAltNames: { $exists: true } } })
db.lab_keyvault.getIndexes().map(i => ({ name: i.name, unique: !!i.unique, partial: i.partialFilterExpression }))

internalsUnder the hood

Encrypted fields are stored as BinData subtype 6. Queryable Encryption creates internal enxcol_ metadata collections (state and compaction) per encrypted collection, which add storage and write amplification; run compactStructuredEncryptionData periodically. Indexes on encrypted fields are managed by the system, sorting and most aggregation on encrypted fields is not possible, and the shard key cannot be encrypted.

Avoid / preferDo this, not that

// master key stored in the application repo
const kmsProviders = { local: { key: fs.readFileSync("./master.key") } }

// customer master key in a KMS; the app only has permission to use it
const kmsProviders = { aws: { accessKeyId, secretAccessKey } }

// deterministic encryption on a low-cardinality field
{ path: "status", algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" }

// randomized for fields you do not query; Queryable Encryption for those you do
{ path: "ssn", queries: { queryType: "equality" } }

PitfallsCommon mistakes

• Losing or deleting the customer master key, which makes every encrypted field unreadable forever.
• Using deterministic encryption on low-cardinality fields (gender, status), which leaks the distribution.
• Assuming automatic encryption works on Community Edition.
• Forgetting that sorts, $regex and most aggregation expressions cannot operate on encrypted fields.

LimitsWhen not to use it

Field-level encryption protects specific sensitive fields from anyone without the keys, including DBAs and backups. It is not a substitute for encryption at rest, TLS or access control, and encrypted fields lose most query abilities (sorting, ranges outside Queryable Encryption's supported types, aggregation on values). Encrypt only fields that need it.

SecuritySecurity notes

The key vault holds data encryption keys wrapped by the master key. Protect it like the data: restrict access, back it up together with the data (a restored database without its key vault is unreadable), and plan key rotation with rewrapManyDataKey.

ChallengeTry it yourself

const kv = db.getSiblingDB("encryption").getCollection("__keyVault")
kv.createIndex({ keyAltNames: 1 }, { unique: true, partialFilterExpression: { keyAltNames: { $exists: true } } })
kv.getIndexes().map(i => i.name)

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.