Manage EBS Volumes on EC2

You've launched EC2 instances, locked down security groups, and automated deployments with Python — but at some point, your application needs more storage. Maybe your database is outgrowing its root volume, or you need a dedicated disk for logs and backups. Without a plan for attaching and managing EBS volumes, you'll be stuck rebooting instances, juggling data, or overpaying for unused capacity. This lesson shows you exactly how to attach, manage, and scale EBS volumes on EC2 — using the AWS CLI and boto3 — so you can handle storage like a seasoned DevOps engineer.

The problem this lesson solves

When you launch an EC2 instance, it comes with a root volume — usually an EBS volume that holds the operating system. That's great until you run out of space or need a separate disk for application data. Many developers try to cram everything onto the root volume, which leads to:

• Disk-full alerts when logs or temporary files fill up the drive.
• Risk of data loss if the instance is terminated (depending on the root volume's delete-on-termination setting).
• Poor performance when I/O-heavy workloads share the same disk as the OS.
• Complex backups because you can't snapshot just the application data.

The solution is to attach additional EBS volumes to your instances. EBS (Elastic Block Store) provides durable, block-level storage that you can attach to a running instance — no downtime required. With Python and the AWS SDK (boto3), you can automate the entire lifecycle: create, attach, format, mount, resize, and snapshot volumes.

By the end of this lesson, you'll be able to:

• Explain what EBS volumes are and how they relate to EC2 instances.
• Attach an EBS volume to an EC2 instance using both the AWS CLI and boto3.
• Format and mount the volume so your application can use it.
• Resize volumes and extend file systems without rebooting.
• Troubleshoot common issues like volume attachment failures and missing devices.

Core concept / mental model

Think of an EC2 instance as a laptop and an EBS volume as an external hard drive. The laptop has a built-in hard drive (the root volume), but when you need more space, you plug in an external drive via USB. The external drive is independent — you can unplug it and plug it into another laptop, keep it for one laptop only, or even share it between two laptops (with caveats).

In AWS terms:

• EBS volume: A block-level storage device that you create independently of any EC2 instance.
• Availability Zone (AZ): Where the volume lives. An EBS volume is tied to a specific AZ, so it can only be attached to instances in the same AZ.
• Instance store vs. EBS: Instance store volumes are ephemeral — they're lost when the instance stops. EBS volumes are persistent — they survive instance stops and terminations (unless you explicitly delete them).

Here's a simple diagram in words:

EC2 Instance (in us-east-1a)
    |
    | connected via network
    |
EBS Volume (in us-east-1a)  <-- same AZ required

Key characteristics of EBS volumes:

• Persistence: Data survives instance reboots and stops.
• Snapshots: You can take point-in-time backups to S3.
• Scalability: Increase size and performance types while attached.
• Cost: You pay for provisioned storage, even if unused. Detach unused volumes to save money.

Pro tip: Always match the volume's AZ to the instance's AZ. If they mismatch, you'll get an attachment error — a classic gotcha for beginners.

How it works step by step

Attaching and managing an EBS volume follows a predictable lifecycle. Here's the full sequence:

1. Create an EBS volume in a specific AZ (or use an existing snapshot).
2. Attach the volume to a running (or stopped) instance in the same AZ.
3. Format the volume with a file system (e.g., ext4) if it's new.
4. Mount the volume to a directory on the instance.
5. Configure auto-mount so the volume is available after reboots (via /etc/fstab).
6. Resize or snapshot as needed.

Step 1: Create the volume

You can create a volume via the AWS Management Console, the AWS CLI, or boto3. For automation, boto3 is your best friend. The volume type (e.g., gp3, io2) and size (in GiB) must be specified. The volume is created in a specific AZ — remember that.

Step 2: Attach the volume

Attaching is like plugging in the external drive. You specify the volume ID, instance ID, and a device name (e.g., /dev/sdf). The attachment happens instantly, but the operating system may need a few seconds to recognize the new device. No reboot is required.

Step 3: Format and mount

The volume appears as a raw block device (e.g., /dev/xvdf). You need to create a file system with mkfs and then mount it. On modern Linux, run lsblk to see the new disk. In Python, you can use subprocess or fabric to run these shell commands, or you can use the AWS Systems Manager Run Command.

Step 4: Persist the mount

A mount is temporary — it's gone after a reboot. To make it permanent, add an entry to /etc/fstab. Use the device UUID instead of the device name because device names can change on reboot.

Step 5: Resize or snapshot

You can modify the volume size or performance type with modify_volume. After resizing, you must extend the file system to use the extra space. Snapshots are point-in-time backups; they're incremental, so the first snapshot copies all data, and subsequent snapshots only save changes.

Hands-on walkthrough

Let's get your hands dirty. We'll create, attach, format, and mount an EBS volume using the AWS CLI and boto3.

Prerequisites

• An EC2 instance running Amazon Linux 2 or Ubuntu in us-east-1a.
• AWS CLI installed and configured with credentials (e.g., aws configure).
• Python 3.10+ and boto3 installed (pip install boto3).

Example 1: Create and attach with AWS CLI

First, get your instance ID and its AZ:

$ aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" --query "Reservations[].Instances[].{ID:InstanceId,AZ:Placement.AvailabilityZone}" --output table

Now create a 10 GiB gp3 volume in the same AZ:

$ aws ec2 create-volume --availability-zone us-east-1a --size 10 --volume-type gp3

Note the VolumeId from the output (e.g., vol-1234567890abcdef0). Then attach it:

$ aws ec2 attach-volume --volume-id vol-1234567890abcdef0 --instance-id i-0abcdef1234567890 --device /dev/sdf

Example 2: Automate with Python and boto3

The Python way is cleaner for automation. Here's a complete script that creates, attaches, and waits for the volume to be available:

import boto3
from botocore.exceptions import ClientError

# Initialize clients
ec2 = boto3.client('ec2')

# Configuration
INSTANCE_ID = 'i-0abcdef1234567890'
AZ = 'us-east-1a'
VOLUME_SIZE = 10  # GiB
DEVICE = '/dev/sdf'

# Create the volume
try:
    response = ec2.create_volume(
        AvailabilityZone=AZ,
        Size=VOLUME_SIZE,
        VolumeType='gp3',
        TagSpecifications=[
            {
                'ResourceType': 'volume',
                'Tags': [{'Key': 'Name', 'Value': 'data-volume'}]
            }
        ]
    )
    volume_id = response['VolumeId']
    print(f"Created volume {volume_id}")

    # Wait until the volume is available
    waiter = ec2.get_waiter('volume_available')
    waiter.wait(VolumeIds=[volume_id])
    print("Volume is available")

    # Attach the volume
    ec2.attach_volume(VolumeId=volume_id, InstanceId=INSTANCE_ID, Device=DEVICE)
    print(f"Attached {volume_id} to {INSTANCE_ID} at {DEVICE}")

except ClientError as e:
    print(f"Error: {e}")

Expected output:

Created volume vol-1234567890abcdef0
Volume is available
Attached vol-1234567890abcdef0 to i-0abcdef1234567890 at /dev/sdf

Example 3: Format and mount from the instance

Now SSH into your instance and verify the new device:

$ ssh ec2-user@your-instance-ip
$ lsblk

You should see a device like /dev/xvdf. Format it and mount it:

$ sudo mkfs -t ext4 /dev/xvdf
$ sudo mkdir /data
$ sudo mount /dev/xvdf /data
$ df -h

The /data directory now has 10 GiB of storage. To make the mount persistent, add this line to /etc/fstab (use the UUID from blkid):

$ sudo blkid /dev/xvdf

Then append to /etc/fstab:

UUID=your-uuid-here  /data  ext4  defaults,nofail  0  2

Pro tip: Use the nofail option in fstab. If the volume isn't attached at boot, the instance won't hang on a missing device.

Example 4: Resize the volume with Python

When you need more space, resize the volume and then extend the file system. Here's the boto3 part:

import boto3

ec2 = boto3.client('ec2')

# Resize from 10 GiB to 20 GiB
response = ec2.modify_volume(
    VolumeId='vol-1234567890abcdef0',
    Size=20
)
print(response['VolumeModification'])

After the modification reaches the optimizing or completed state (use describe_volumes_modifications to check), you must extend the file system on the instance:

$ sudo growpart /dev/xvdf 1   # for partitioned volumes
$ sudo resize2fs /dev/xvdf    # for ext4; use xfs_growfs for XFS

Compare options / when to choose what

EBS volumes come in several types, and you can also use instance store or Amazon EFS. Here's a comparison:

When to choose what:

• EBS gp3 — default for most workloads: web servers, app data, small databases.
• EBS io2 — mission-critical databases, high-performance applications.
• Instance Store — temporary data, caches, or when you don't need durability.
• EFS — multiple instances need shared access to the same files.

As for the attachment method:

• AWS Console — good for one-off manual tasks.
• AWS CLI — good for scripts in a terminal.
• boto3/Python — best for automation, integration with other Python scripts, and building DevOps tools.

Troubleshooting & edge cases

Here are common problems and how to fix them:

• "Volume is not in the same availability zone" error — Ensure the volume and instance are in the same AZ. You can't attach cross-AZ; copy the volume to the target AZ using a snapshot.
• Device name not showing up after attach — Wait a few seconds, then run lsblk or sudo udevadm settle. Sometimes the device appears as /dev/xvdf instead of /dev/sdf.
• Mount fails after reboot — Check /etc/fstab for the correct UUID and use nofail. If the instance boots but the mount is missing, verify the volume is attached and the entry is correct.
• Volume remains "attached" after instance termination — If you didn't set DeleteOnTermination to true for the volume (when attaching), it might remain as an orphaned volume. Check the AWS console for unattached volumes and delete or reuse them.
• Disk full but volume seems large — You may have resized the volume but not extended the file system. Run lsblk to see block size vs. file system size; use growpart and resize2fs.
• Snapshot stuck in "pending" — Snapshotting a volume with heavy writes can take a long time. Use --no-reboot when creating snapshots from instances, and consider automating snapshots off-peak.

What you learned & what's next

You now have a solid understanding of how to attach and manage EBS volumes on EC2. You can create, attach, format, mount, resize, and snapshot volumes — all from Python with boto3. You also know how to troubleshoot typical issues like AZ mismatches and fstab misconfigurations.

To cement the learning objective, retry the hands-on examples with a new volume size (e.g., 15 GiB) and practice resizing. Notice how the file system extension works.

What's next? In the next lesson, you'll learn how to back up and restore EC2 instances using EBS snapshots — a critical skill for disaster recovery and data migration. You'll automate snapshot creation with Python, rotate them to manage costs, and restore an instance from a snapshot. Each new skill builds on the previous one, so keep practicing!