Encrypt App Data with KMS

Your Python app is storing secrets — database credentials, API keys, customer PII — and if that data lands on disk unencrypted, a single leaked snapshot or stolen backup becomes a headline breach. AWS Key Management Service (KMS) gives you managed, auditable encryption keys that turn plaintext into ciphertext with a few API calls, but knowing when and how to use KMS versus other encryption tools is what separates a secure app from a false sense of security. In this lesson you'll learn how to use KMS to encrypt Python app data at rest — from the core concepts to a hands-on walkthrough that you can adapt to your own projects.

The problem this lesson solves

Imagine you're building a Django app that stores customer payment tokens. You read the docs, you hash passwords, you set DEBUG = False. Then your ops team finds a database dump sitting in an unencrypted S3 bucket. Or a developer commits a .env file with production secrets to GitHub. You have a data-at-rest problem: the data on disk, in backups, or in logs is readable by anyone who gets their hands on it.

AWS KMS solves this by centralizing key management. Instead of scattering AES keys inside your application code (where they're hard to rotate and easy to leak), you create a Customer Master Key (CMK) in KMS, and your Python app uses that key to encrypt and decrypt data. KMS handles the hardware security modules (HSMs), key rotation, and CloudTrail auditing. You get encryption that's compliant (HIPAA, SOC, PCI) and operationally simple, without rolling your own crypto — which, as the saying goes, is a one-way ticket to a breach.

If you're already sending data to S3 or RDS, you might think "doesn't S3 encryption handle this?" S3's default SSE-S3 encrypts objects with keys AWS manages for you, but you can't control rotation or restrict who can decrypt. KMS gives you that control, plus an audit trail of every encryption operation. For data that's not in a managed service — say, a local file, a field in a database, or a message in a queue — KMS is the right tool.

Core concept / mental model

Think of KMS as a digital safe deposit box for your encryption keys.

• You have a customer master key (CMK) — the master key stored inside KMS's tamper-proof HSMs. This is the master key that never leaves KMS.
• When you want to encrypt data, you call encrypt with the CMK. KMS returns a blob of ciphertext.
• When you want to decrypt, you call decrypt with the same CMK (or permission to use it). KMS returns the plaintext.

But here's the catch: KMS encrypt/decrypt has a 4 KB payload limit. You can't encrypt a 1 GB file directly. That's where envelope encryption comes in — the most important concept in this lesson.

Envelope encryption works like this:

1. Your app asks KMS for a data key — a plaintext key and an encrypted copy of that key.
2. You use the plaintext data key to encrypt your data locally, using a standard algorithm like AES-256.
3. You discard the plaintext data key immediately after encryption, keeping only the encrypted data key.
4. To decrypt, you send the encrypted data key back to KMS, which returns the plaintext data key, and then you decrypt your data.

The benefit? You only talk to KMS once per file, not per block. And the data key itself is protected by KMS, so you never store a usable key on disk.

In code, the flow looks like this:

# Pseudocode — the real thing in hands-on
import boto3
kms = boto3.client('kms', region_name='us-east-1')

# Generate a data key
response = kms.generate_data_key(
    KeyId='alias/my-key',
    KeySpec='AES_256'
)
plaintext_data_key = response['Plaintext']  # use this to encrypt locally
encrypted_data_key = response['CiphertextBlob']  # store this with your data

KMS itself never sees your data — it only sees your keys. That's a mental model shift: KMS is not a data encryption service, it's a key encryption service.

How it works step by step

Here's the step-by-step workflow for encrypting application data at rest with KMS:

1. Create a KMS key (CMK) — In the console or via CLI, create a symmetric CMK (or use an AWS-managed key but you lose rotation control). Give it an alias like alias/my-app-key.
2. Set IAM permissions — The IAM role your Python app runs under needs kms:Encrypt, kms:Decrypt, and kms:GenerateDataKey permissions on that key. Use the key policy to restrict which principals can use it.
3. Write your encryption logic — In your Python app, use boto3 to generate a data key, encrypt your data locally, and store the encrypted data key alongside the ciphertext.
4. Decrypt when needed — To read the data, retrieve the encrypted data key, call kms.decrypt, get the plaintext data key, and decrypt your data locally.
5. Handle errors and rotation — If KMS is unavailable or permissions change, your app should fail gracefully. When you rotate the CMK, KMS keeps decrypting with old versions automatically, so your data isn't orphaned.

This pattern applies whether you're encrypting a single field in a database, a file in S3 (using SSE-KMS), or a whole volume.

Hands-on walkthrough

Let's put this into practice. You'll need:

• Python 3.10+ and boto3 installed (pip install boto3)
• AWS credentials configured (via aws configure or environment variables)
• A KMS key created. Use the console, or create one with the CLI:

aws kms create-key --description "My app key"
aws kms create-alias --alias-name alias/my-app-key --target-key-id <key-id>

Now write a Python script that encrypts a file and then decrypts it:

# encrypt_file.py
import boto3
from cryptography.fernet import Fernet
import base64

kms = boto3.client('kms', region_name='us-east-1')
KEY_ID = 'alias/my-app-key'

# Step 1: Generate a data key from KMS
resp = kms.generate_data_key(KeyId=KEY_ID, KeySpec='AES_256')
plaintext_key = resp['Plaintext']
encrypted_key = resp['CiphertextBlob']

# Step 2: Encrypt your data locally using Fernet (AES-128 under the hood, but you can
# use any symmetric algorithm; here we adapt Fernet)
# Fernet needs a URL-safe base64-encoded 32-byte key, so we'll adapt.
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
import os

iv = os.urandom(16)
cipher = Cipher(algorithms.AES(plaintext_key), modes.CBC(iv))
encryptor = cipher.encryptor()
# Pad data to block size
from cryptography.hazmat.primitives import padding
padder = padding.PKCS7(128).padder()
plaintext = b"Secret customer data"
padded = padder.update(plaintext) + padder.finalize()
ct = encryptor.update(padded) + encryptor.finalize()

# Step 3: Store the encrypted key, IV, and ciphertext together
with open('encrypted.bin', 'wb') as f:
    f.write(len(encrypted_key).to_bytes(4, 'big'))
    f.write(encrypted_key)
    f.write(iv)
    f.write(ct)

print("Encrypted data written to encrypted.bin")

# decrypt_file.py
import boto3
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives import padding

kms = boto3.client('kms', region_name='us-east-1')

with open('encrypted.bin', 'rb') as f:
    key_len = int.from_bytes(f.read(4), 'big')
    encrypted_key = f.read(key_len)
    iv = f.read(16)
    ct = f.read()

# Step 4: Ask KMS to decrypt the data key
resp = kms.decrypt(CiphertextBlob=encrypted_key)
plaintext_key = resp['Plaintext']

# Step 5: Decrypt your data locally
cipher = Cipher(algorithms.AES(plaintext_key), modes.CBC(iv))
decryptor = cipher.decryptor()
padded = decryptor.update(ct) + decryptor.finalize()
unpadder = padding.PKCS7(128).unpadder()
plaintext = unpadder.update(padded) + unpadder.finalize()

print(f"Decrypted: {plaintext.decode()}")

Run the encrypt script, then the decrypt script, and you should see Decrypted: Secret customer data.

Pro tip: In production, you'd use a higher-level library like aws-encryption-sdk-python which handles envelope encryption, data key caching, and proper key management automatically. The low-level calls above teach you the mechanics, but the SDK is what you'd deploy.

Compare options / when to choose what

You have several ways to encrypt data at rest on AWS. Here's a quick comparison:

When to choose what: - If you're storing files exclusively in S3 and need encryption (and you don't need granular IAM restrictions), use SSE-S3 or SSE-KMS. SSE-KMS gives you audit and control. - If you're storing secrets in DynamoDB, or encrypting fields before saving to a database, use KMS envelope encryption in your Python code — that's the pattern from the walkthrough. - If you need to encrypt within a VPC without internet access, you can use VPC endpoints for KMS to keep traffic private.

Troubleshooting & edge cases

• AccessDeniedException when calling GenerateDataKey — The IAM role doesn't have permission. Ensure your role has a policy like: json { "Effect": "Allow", "Action": ["kms:GenerateDataKey", "kms:Decrypt"], "Resource": "arn:aws:kms:us-east-1:123456789012:key/<key-id>" } Also check the key policy grants access to your account root, and that you haven't accidentally set a restrictive grant.

• InvalidCiphertext — You tried to decrypt with a different key than the one used to encrypt, or the ciphertext was corrupted. When using envelope encryption, make sure you store the encrypted data key with the data, not separately.

• KeyUnavailable — The key was disabled or scheduled for deletion. Go to KMS console and re-enable it. Your app should catch this exception and retry or alert.

• Different region — If you encrypt in us-east-1 and try to decrypt in eu-west-1, you'll get an error. KMS keys are regional; use the same region for encrypt and decrypt operations.

• Exceeding the 4 KB limit — Trying to encrypt a payload larger than 4 KB directly will fail. Use generate_data_key and encrypt locally, as shown above.

• Rate limiting — KMS has default limits (5,500 requests per second per key for symmetric keys). If you're encrypting thousands of records per second, enable data key caching with the AWS Encryption SDK to reduce KMS calls.

What you learned & what's next

You've now got a mental model of KMS as a key management service, not a data encryption service. You can explain how envelope encryption works, and you've written Python code that uses KMS to encrypt and decrypt data at rest — using generate_data_key, local encryption, and decrypt. You also know when to use KMS over SSE-S3 or client-side encryption, and you're aware of the common pitfalls like permissions, region, and the 4 KB limit.

In the next lesson, you'll likely move to secrets management with AWS Secrets Manager — a natural companion to KMS that stores your secrets (like database passwords) encrypted with KMS, and gives you automatic rotation. Or you might continue to encrypting data in transit with TLS/ACM. Whatever comes next, you now have a solid foundation to build on.

For a deeper dive, check out the AWS Encryption SDK for Python, which simplifies envelope encryption and data key caching. And remember: encryption at rest is a baseline — combine it with IAM policies, VPC security, and audit logging for a defense-in-depth approach.