Use DynamoDB for NoSQL storage

You’ve built the API, containerized it, and pushed it to the cloud — but now the database is the bottleneck. Your relational database is struggling to keep up, joins are slow, and scaling writes means painful sharding. That’s the exact moment you need to use DynamoDB for high-scale NoSQL storage. DynamoDB is a fully managed, serverless key-value and document database from AWS that delivers single-digit millisecond performance at any scale, without you managing a single server. In this lesson, you’ll learn how to model, query, and operate DynamoDB for production workloads, with hands-on Python examples you can run today.

The problem this lesson solves

Traditional relational databases (RDS, PostgreSQL, MySQL) are powerful, but they hit a wall at high scale:

• Write bottlenecks — single-primary replication limits write throughput.
• Join complexity — normalized schemas slow down as data grows.
• Operational overhead — you must manage backups, failover, and scaling.
• Cost spikes — provisioning for peak load wastes money.

When your app needs predictable single-digit millisecond latency, unlimited storage, and zero server management, you need a different approach. DynamoDB is built for exactly this: it scales horizontally by design, replicates data across three availability zones, and charges you only for what you use. In this lesson, you’ll stop fighting your database and start building at scale.

Core concept / mental model

Think of DynamoDB as a giant, distributed hash table — but with superpowers. Instead of rows and columns, you store items (like JSON documents) in tables. Each item has a primary key that must be unique, and you query items by that key with blistering speed.

A DynamoDB table is like a warehouse with a million bins. Each bin has a unique label (the partition key), and items are placed into bins based on that label. To find something, you go straight to the bin — no searching the whole warehouse. That’s why queries are so fast.

Here are the core building blocks you’ll use every day:

• Table — a collection of items; think of it as a file folder.
• Item — a single record; like a row, but schemaless.
• Attribute — a key-value pair within an item; like a field.
• Primary key — uniquely identifies each item. Two types:
• Partition key (simple) — one attribute; distributes data across partitions.
• Composite key (partition + sort key) — two attributes; allows queries with a range on the sort key.
• Secondary index — lets you query on non-key attributes (think of it as a second view of your data).

Pro tip: In DynamoDB, you denormalize data — you don’t join tables. Instead, you store related data together in a single item or use secondary indexes to support different access patterns. The faster you accept this, the smoother your DynamoDB journey.

How it works step by step

Here is the logical flow from zero to a working DynamoDB table:

1. Create a table — give it a name, define its primary key, and choose a billing mode.
2. Choose capacity mode — on-demand or provisioned (we’ll cover both later).
3. Insert/update items — use PutItem or UpdateItem; DynamoDB stores your JSON-like data.
4. Query items — use GetItem for a single item by key, or Query for multiple items with the same partition key.
5. Scan (careful!) — read every item in the table; expensive and slow, so use it sparingly.
6. Scale automatically — DynamoDB handles partitions and throughput behind the scenes.

Let’s see this in action with the AWS SDK for Python (boto3).

Hands-on walkthrough

Prerequisites

Make sure you have:

• An AWS account and credentials configured (aws configure)
• Python 3.8+ installed
• boto3 library (pip install boto3)
• Basic IAM permissions for DynamoDB (e.g., AmazonDynamoDBFullAccess)

1. Create a table

We’ll create a simple Users table with a partition key user_id (string). Use on-demand capacity to start.

import boto3

dynamodb = boto3.resource('dynamodb', region_name='us-east-1')

table = dynamodb.create_table(
    TableName='Users',
    KeySchema=[
        {'AttributeName': 'user_id', 'KeyType': 'HASH'}
    ],
    AttributeDefinitions=[
        {'AttributeName': 'user_id', 'AttributeType': 'S'}
    ],
    BillingMode='PAY_PER_REQUEST'
)

# Wait for table to be active
table.wait_until_exists()
print("Table status:", table.table_status)

Expected output: Table status: ACTIVE

2. Insert an item

Now let’s add a few users. Notice the attribute types — strings, numbers, booleans, lists, maps — all supported natively.

table.put_item(
    Item={
        'user_id': 'u123',
        'email': 'alice@example.com',
        'age': 30,
        'active': True,
        'tags': ['premium', 'beta']
    }
)

# Add a second item
table.put_item(
    Item={
        'user_id': 'u456',
        'email': 'bob@example.com',
        'age': 25,
        'active': False,
        'tags': ['free']
    }
)
print("Items inserted.")

3. Get an item by key

Retrieving a single item is fast and simple:

response = table.get_item(
    Key={'user_id': 'u123'}
)
if 'Item' in response:
    print("User found:", response['Item'])
else:
    print("User not found")

Expected output:

User found: {'user_id': 'u123', 'email': 'alice@example.com', 'age': 30, 'active': True, 'tags': ['premium', 'beta']}

4. Query with a sort key

For more advanced queries, use a composite key. Let’s create an Orders table with customer_id (partition key) and order_date (sort key). This lets you query all orders for a customer within a date range.

orders_table = dynamodb.create_table(
    TableName='Orders',
    KeySchema=[
        {'AttributeName': 'customer_id', 'KeyType': 'HASH'},
        {'AttributeName': 'order_date', 'KeyType': 'RANGE'}
    ],
    AttributeDefinitions=[
        {'AttributeName': 'customer_id', 'AttributeType': 'S'},
        {'AttributeName': 'order_date', 'AttributeType': 'S'}
    ],
    BillingMode='PAY_PER_REQUEST'
)
orders_table.wait_until_exists()

# Insert a couple of orders
orders_table.put_item(Item={'customer_id': 'c1', 'order_date': '2025-01-15', 'amount': 99})
orders_table.put_item(Item={'customer_id': 'c1', 'order_date': '2025-02-01', 'amount': 150})

# Query all orders for c1 in January and February
response = orders_table.query(
    KeyConditionExpression='customer_id = :cid AND order_date BETWEEN :start AND :end',
    ExpressionAttributeValues={
        ':cid': 'c1',
        ':start': '2025-01-01',
        ':end': '2025-02-28'
    }
)
for item in response['Items']:
    print(item)

Expected output:

{'customer_id': 'c1', 'order_date': '2025-01-15', 'amount': 99}
{'customer_id': 'c1', 'order_date': '2025-02-01', 'amount': 150}

Pro tip: Always design your table around your query patterns, not your entities. That means you might have multiple tables for different access patterns, or duplicate data within an item to avoid extra reads.

Compare options / when to choose what

You don’t have to choose DynamoDB for everything. Here’s a quick comparison with other AWS storage options:

When to choose DynamoDB over RDS:

• Your access patterns are well-defined (you know how you’ll query the data).
• You need high write throughput without sharding headaches.
• You want a fully managed service with built-in backups and encryption.
• You prefer a NoSQL model with flexible schemas.

When to avoid DynamoDB:

• You need complex multi-table joins and transactional integrity across many entities.
• You rely heavily on ad-hoc queries and dynamic filters (SQL is better).
• Your workload is read-heavy with tiny write volume — RDS might be cheaper.

Capacity modes — another important choice: - On-demand (PAY_PER_REQUEST): pay per read/write request; perfect for unpredictable traffic. No capacity planning. - Provisioned: set a fixed read/write capacity; cheaper for steady traffic. You can enable auto-scaling to adjust.

Variations: - Use DynamoDB Accelerator (DAX) for microsecond read latency — it’s an in-memory cache in front of DynamoDB. - Use Global Tables for multi-region writes with conflict resolution. - If you need SQL-on-DynamoDB, try PartiQL (a SQL-compatible query language built into DynamoDB).

Troubleshooting & edge cases

Even with DynamoDB, things can go wrong. Here are common issues and how to fix them:

1. ProvisionedThroughputExceededException

Symptom: You get ...exceeded the maximum allowed provisioned throughput...

Cause: You exceeded your provisioned read/write capacity, or you hit the partition’s hot key limit.

Fix: - Switch to on-demand capacity for spiky traffic. - If provisioned, increase your capacity or enable auto-scaling. - Distribute your data better — avoid “hot partitions” where one partition key dominates.

# Example: handling retries with exponential backoff
import time
import random

def put_with_retry(table, item, retries=5):
    for attempt in range(retries):
        try:
            table.put_item(Item=item)
            return True
        except Exception as e:
            if 'ProvisionedThroughputExceeded' in str(e):
                wait = (2 ** attempt) + random.uniform(0, 1)
                time.sleep(wait)
            else:
                raise
    return False

2. Hot partitions

Symptom: High latency for a specific partition key, while others are fine.

Cause: Your partition key has low cardinality (e.g., a status field with only two values). All traffic hits a few partitions.

Fix: Choose a high-cardinality key like a user ID or a composite key with a random suffix. For write-heavy workloads, use write sharding — append a random number to the partition key and store the mapping elsewhere.

3. Scan is too slow/expensive

Symptom: Your Scan operation returns data slowly or costs too much.

Cause: Scan reads every item, consuming RCUs (read capacity units) linearly.

Fix: - Use Query instead, filtering by partition key. - Add a GSI (Global Secondary Index) for unpopular queries. - Limit the result with Limit parameter and use pagination.

# Use a GSI to query by email (non-key attribute)
gsi_name = 'EmailIndex'
response = table.query(
    IndexName=gsi_name,
    KeyConditionExpression='email = :email',
    ExpressionAttributeValues={':email': 'alice@example.com'}
)

4. Item too large

Symptom: You get an Item size to large error.

Cause: DynamoDB limits items to 400 KB.

Fix: Store large blobs in S3 and keep a URL reference in DynamoDB. Break the item into smaller sub-items if possible.

5. Eventual consistency surprise

Symptom: After a write, a read doesn’t see the change immediately.

Cause: DynamoDB replicates data across AZs; default reads are eventually consistent.

Fix: If you need immediate consistency, set ConsistentRead=True in your GetItem call (costs twice as many RCUs).

response = table.get_item(
    Key={'user_id': 'u123'},
    ConsistentRead=True
)

6. Missing attributes in queries

Symptom: Your Query doesn’t return items.

Cause: The query condition requires a sort key expression, and your data doesn’t match.

Fix: Check your KeyConditionExpression syntax. Use > for string comparisons, and BETWEEN for ranges. Also, remember that Query always requires an exact partition key.

What you learned & what's next

You’ve learned how to use DynamoDB for high-scale NoSQL storage — the core concept, how to create tables, insert and query items, and when to choose DynamoDB over other AWS databases. You now know how to avoid hot partitions, handle throttling, and use GSIs to support multiple access patterns. These skills are critical for any production backend that needs to scale without breaking a sweat.

Key takeaways: - DynamoDB is a fully managed NoSQL database with single-digit-millisecond latency and automatic scaling. - Design your table schema around your query patterns — not the other way around. - Use on-demand capacity for unpredictable traffic; provisioned with auto-scaling for steady load. - Avoid Scan; use Query with partition key and optional sort key. - Watch for hot partitions — choose high-cardinality keys. - Use ConsistentRead=True only when needed; it costs double.

Next step: In the next lesson, you’ll build on this foundation and learn how to stream DynamoDB changes to other AWS services using DynamoDB Streams and AWS Lambda. That’s a powerful pattern for event-driven architectures.

Now, go ahead and practice — create a new table, insert some data, and try a few queries. You’re on your way to scaling your backend like a pro.