Encryption at Rest and Its Performance Cost

The Symptom

After enabling LUKS full-disk encryption to meet compliance requirements, the telemetry platform’s write throughput drops from 50,000 ops/s to 38,000 ops/s. Read latency increases by 20%.

The Cause

Encryption at rest adds CPU and I/O overhead. Every block written to disk must be encrypted, and every block read must be decrypted. The cost depends on the encryption method:

LUKS (dm-crypt): The Linux kernel encrypts/decrypts at the block device layer. Every I/O operation passes through the encryption layer. Uses AES-256-XTS by default. Modern CPUs have AES-NI instructions that accelerate this, but the overhead is still measurable.
Cloud-managed (AWS EBS encryption, GCP CMEK): The storage controller handles encryption in hardware. The CPU overhead is near zero, but IOPS may be slightly reduced.
WiredTiger native encryption (MongoDB Enterprise): Encryption happens inside WiredTiger before writing to the filesystem. The data is encrypted in the cache and on disk.

The Benchmark

Encryption method	Write throughput	Read p99	CPU overhead	IOPS impact
None	50,000/s	12ms	Baseline	Baseline
LUKS (AES-256-XTS, AES-NI)	38,000/s	14.5ms	+15% CPU	-5% IOPS
Cloud-managed (EBS encryption)	48,000/s	12.5ms	< 1% CPU	-2% IOPS
WiredTiger native (AES-256-CBC)	42,000/s	13ms	+10% CPU	Baseline IOPS

LUKS has the highest overhead because it encrypts at the block device layer, which means every I/O operation (including filesystem metadata, journal, and oplog) is encrypted. MongoDB’s WiredTiger compression runs before LUKS encryption, so the data is compressed then encrypted.

Cloud-managed encryption has the lowest overhead because the encryption happens in the storage controller hardware, not on the CPU.

The Fix

Option 1: Cloud-managed encryption (recommended for cloud deployments).

# AWS EBS encryption via StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mongodb-encrypted
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  encrypted: "true"
  kmsKeyId: "arn:aws:kms:us-east-1:123456789:key/your-key-id"
  iops: "16000"
  throughput: "250"

No MongoDB configuration changes needed. Encryption is transparent to the application.

Option 2: LUKS with AES-NI optimization (for on-premises deployments).

# Verify AES-NI support
grep -o aes /proc/cpuinfo | head -1
# aes

# Create LUKS volume with optimal cipher
cryptsetup luksFormat --cipher aes-xts-plain64 --key-size 512 \
    --hash sha256 /dev/nvme1n1

cryptsetup luksOpen /dev/nvme1n1 mongodb-data
mkfs.xfs /dev/mapper/mongodb-data
mount -o noatime /dev/mapper/mongodb-data /data/db

Optimize LUKS for MongoDB by enabling write-back caching and increasing the number of encryption threads:

# Enable write-back mode (faster, relies on UPS for crash safety)
cryptsetup refresh --perf-no_read_workqueue --perf-no_write_workqueue \
    --allow-discards mongodb-data

# Check dm-crypt performance flags
dmsetup table mongodb-data
# 0 ... crypt aes-xts-plain64 ... no_read_workqueue no_write_workqueue

Option 3: Tune MongoDB to reduce encryption overhead.

Regardless of encryption method, reduce the number of I/O operations to minimize the encryption tax:

// FAST: Larger batch sizes reduce per-operation encryption overhead
MongoClientSettings settings = MongoClientSettings.builder()
    .applyConnectionString(new ConnectionString(uri))
    .applyToSocketSettings(builder -> builder
        .connectTimeout(10, TimeUnit.SECONDS)
        .readTimeout(30, TimeUnit.SECONDS)
    )
    .build();

// Use larger cursors to reduce round trips (fewer encrypted I/O operations)
collection.find(filter)
    .batchSize(500)   // Default 101. Larger batches = fewer I/O operations
    .into(results);

The Proof

Configuration	Write throughput	CPU overhead	Compliance
No encryption	50,000/s	Baseline	Not compliant
LUKS (default settings)	38,000/s	+15%	Compliant
LUKS (optimized)	44,000/s	+12%	Compliant
Cloud-managed (EBS)	48,000/s	< 1%	Compliant

After LUKS optimization (no_read_workqueue, no_write_workqueue):

Metric	LUKS default	LUKS optimized
Write throughput	38,000/s	44,000/s
Read p99	14.5ms	13ms
Checkpoint duration	8.5s	6.2s

The Trade-off

Cloud-managed encryption is the cheapest in terms of performance but ties the deployment to a specific cloud provider. LUKS is portable (works on any Linux server) but has higher CPU overhead. WiredTiger native encryption requires MongoDB Enterprise, which has licensing costs.

The no_read_workqueue and no_write_workqueue flags in LUKS skip the dm-crypt work queues and process encryption inline. This reduces latency but increases CPU usage on the calling thread. For MongoDB, where I/O threads are already pinned to cores, inline processing is faster because it avoids the context switch to the dm-crypt work queue.

The allow-discards flag passes TRIM commands through the encryption layer. This improves SSD performance (the SSD can reclaim blocks) but leaks information about which blocks are in use. For most compliance requirements, this is acceptable because the attacker still cannot read the block contents.

For the telemetry platform, cloud-managed encryption is the choice because the deployment is on AWS. The 4% throughput reduction (50,000 to 48,000) is negligible compared to the operational simplicity of not managing encryption keys and LUKS volumes.