How to mock S3 remote backend for Terraform in Python

Simulate a Terraform S3 remote backend using moto to write and read state files, enabling local testing without real AWS.

Medium Python 3.9+ Aug 9, 2026 Production deployment patterns 14 views 0 copies

boto3 moto terraform s3 testing
Requires third-party packages — install first
pip install boto3 moto

Python code

43 lines
Python 3.9+ 
import boto3
from moto import mock_aws
import json
from pathlib import Path

@mock_aws
def demo_s3_remote_backend():
    s3 = boto3.client("s3", region_name="us-east-1")
    bucket = "terraform-state-bucket"
    key = "env/prod/terraform.tfstate"
    
    s3.create_bucket(Bucket=bucket)
    
    # Simulate Terraform writing state to S3
    state_data = json.dumps({
        "version": 4,
        "terraform_version": "1.9.0",
        "resources": [{"type": "aws_instance", "name": "web", "instances": [{"attributes": {"id": "i-12345"}}]}]
    })
    
    s3.put_object(Bucket=bucket, Key=key, Body=state_data)
    
    # Simulate Terraform reading state from S3
    response = s3.get_object(Bucket=bucket, Key=key)
    loaded_state = json.loads(response["Body"].read().decode())
    
    # Check for state locking (versioning)
    s3.put_bucket_versioning(
        Bucket=bucket,
        VersioningConfiguration={"Status": "Enabled"}
    )
    versions = s3.list_object_versions(Bucket=bucket, Prefix=key)
    
    return {
        "bucket": bucket,
        "key": key,
        "resource_id": loaded_state["resources"][0]["instances"][0]["attributes"]["id"],
        "version_count": len(versions["Versions"])
    }

if __name__ == "__main__":
    result = demo_s3_remote_backend()
    print(json.dumps(result, indent=2))

Output

stdout 
{
  "bucket": "terraform-state-bucket",
  "key": "env/prod/terraform.tfstate",
  "resource_id": "i-12345",
  "version_count": 1
}

How it works

The @mock_aws decorator intercepts all boto3 calls and runs them against an in-memory mock instead of real AWS. We create a bucket and then put_object writes a JSON state file, simulating Terraform's backend write. Reading back with get_object and decoding the body reproduces what Terraform does when it loads remote state. Enabling versioning on the bucket and calling list_object_versions demonstrates how state locking and history are tracked, with the version count returned as 1 for the initial write. This pattern is ideal for test suites that need to validate Terraform state operations without incurring AWS costs.

Common mistakes

  • Forgetting to install moto with `pip install moto[s3]` to get S3 mocking support.
  • Not placing the decorator on the function that uses boto3, so the mock never activates.
  • Assuming `list_object_versions` returns versions before enabling bucket versioning — it returns an empty list otherwise.
  • Hardcoding the region or bucket name when tests need to run in isolation with unique names.

Variations

  1. Use `terraform-aws` state backend simulation with `s3cmd`-style CLI instead of boto3 for lower-level control.
  2. Create a reusable fixture in pytest that yields the bucket and state client for multiple tests.

Real-world use cases

  • Testing Terraform state migration tools locally against a mocked S3 bucket, without needing AWS credentials.
  • Verifying that a CI pipeline correctly reads and writes state files before pushing to a real S3 remote backend.
  • Validating state locking logic in a deployment automation script that coordinates concurrent Terraform runs.

Sponsored

Run locally

This sample needs third-party packages, so it cannot run in the browser IDE. Copy the code above, install the packages shown at the top, then run it in your own Python environment.

More from Production deployment patterns

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes