Automate Remediation with SSM Documents

You’ve locked down IAM policies, encrypted data at rest, and tightened your container supply chain. But what happens when a critical security finding shows up at 3 AM, and your team is still asleep? Manually remediating every misconfiguration is slow, error-prone, and leaves your cloud account exposed. This is where automate remediation with SSM documents changes the game — you can write reusable, auditable workflows that fix security issues automatically the moment they’re detected.

The problem this lesson solves

Cloud security isn’t just about preventing incidents; it’s about responding to them fast. Imagine your security scanning tool flags an EC2 instance with a public SSH port open. You get the alert, create a ticket, and a teammate eventually logs in to fix it. That process can take hours or days — time attackers love. Even worse, manual fixes are inconsistent: one engineer might close the port, another might change the security group, and a third might reboot the instance unnecessarily.

The core problem: remediation is reactive, slow, and human-dependent. You need a way to execute security fixes automatically, with full audit trails, and without writing custom scripts that rot over time. AWS Systems Manager (SSM) Documents solve exactly that — they let you encode remediation steps as versioned, repeatable runbooks that trigger automatically when a security finding occurs.

Core concept / mental model

Think of an SSM Document as a runbook — a set of written instructions that AWS can execute automatically. Just like a fire drill tells you exactly what to do when the alarm sounds, an SSM Document tells AWS exactly what to do when a security rule is violated. It’s a JSON or YAML file that defines steps (like using a shell script or a Python snippet) that run on your EC2 instances or in your account.

The mental model is simple:

• Trigger: Something happens — a Config rule marks a resource non-compliant, a CloudWatch alarm fires, or a Security Hub finding appears.
• Action: Amazon EventBridge (or another service) invokes an SSM Document.
• Execution: The document runs on the target resource (e.g., an EC2 instance) and performs the fix — closing a port, updating a package, attaching a tag, or stopping an instance.
• Record: Every execution is logged in AWS CloudTrail, giving you a full audit trail.

You can think of it as a security autopilot: you define the policy, and AWS flies the plane.

Key terms to remember:

• SSM Document: A JSON/YAML runbook with steps.
• Command Document: Runs commands on EC2 instances (e.g., shell scripts).
• Automation Document: Runs scripted workflows in your AWS account (no instances needed) — perfect for modifying security groups or IAM policies.
• Simple Execution: A one-step runbook; Multi-Plan: multiple steps with branching.

How it works step by step

Here’s the logical sequence of how an SSM Document remediates a security issue:

1. Detect: AWS Config (or another service) detects a non-compliant resource. For example, a security group rule allows SSH from anywhere (0.0.0.0/0).
2. Trigger: EventBridge picks up the Config rule violation and triggers a rule that runs an SSM Automation Document.
3. Execute: The Automation Document uses AWS APIs to remove the offending rule. For EC2 instance-level issues (like a malware process), you might use a Command Document that runs a script on the instance.
4. Verify: The document checks whether the remediation succeeded — e.g., it re-scans the security group to confirm the rule is gone.
5. Log: All steps are recorded in CloudTrail, and you can see execution details in the SSM console.

The key is that this entire pipeline runs without human intervention, within seconds of detection.

Hands-on walkthrough

Let’s build a real-world automation: automatically stop an EC2 instance if its CPU is being used by an unauthorized miner (a common security finding). We’ll use an Automation Document to stop the instance and tag it as compromised.

Step 1: Create the SSM Automation Document

Go to the AWS SSM console → Documents → Create document → Automation. Paste this YAML (or JSON):

---
schemaVersion: "0.3"
description: "Stop a compromised EC2 instance"
assumeRole: "arn:aws:iam::123456789012:role/SSMRemediationRole"
parameters:
  InstanceId:
    type: String
    description: "Instance to stop"
mainSteps:
  - name: StopInstance
    action: "aws:changeInstanceState"
    inputs:
      InstanceIds:
        - "{{ InstanceId }}"
      DesiredState: "stopped"
  - name: TagInstance
    action: "aws:executeAutomation"
    inputs:
      DocumentName: "AWS-CreateTags"
      RuntimeParameters:
        ResourceIds:
          - "{{ InstanceId }}"
        Tags:
          - Key: "Compromised"
            Value: "true"

This document takes an instance ID and stops it, then tags it with Compromised=true.

Step 2: Set up the trigger

Create an Amazon EventBridge rule that matches Security Hub findings with a specific type (e.g., "EC2.1" for a suspicious crypto miner). The rule’s target is the SSM Automation document, passing the EC2 instance ID as a parameter.

Step 3: Test it

You can manually test the document from the SSM console: run it, provide an instance ID, and watch it stop the instance.

Step 4: Verify with CLI

Use the AWS CLI to list executions:

aws ssm list-command-executions --filters key=DocumentName,value=StopCompromisedInstance

Expected output shows the execution still in progress or successful.

You now have a fully automated remediation workflow!

Compare options / when to choose what

You have a few ways to automate remediation. Here’s how they compare:

When to choose SSM Documents: You need a runbook with steps, approvals, or auditing. If you just need to apply a security group change, a simple Lambda may be easier. But for consistent, auditable, and repeatable remediations, SSM Documents are the standard.

Troubleshooting & edge cases

Common failures and how to fix them:

• SSM Agent not installed: Command Documents fail on instances without the agent. Solution: Top up your EC2 AMIs with the SSM agent, or use AWS Systems Manager Agent (SSMA) bootstrap.
• IAM permissions missing: Automation Documents need a role with ssm:StartAutomationExecution and permission for the actions you call (e.g., ec2:StopInstances). Check CloudTrail for AccessDenied errors.
• Document version drift: If you update a document, EventBridge might still reference an older version. Always specify DocumentVersion in your EventBridge rule.
• Non-Compliant resource not found: Sometimes Config reports a resource ID that doesn’t exist anymore. Use a aws:executeAutomation that first checks existence.
• Execution timeout: Long-running scripts hit SSM’s 48-hour limit? Unlikely, but use timeoutSeconds in steps to fail fast.

What you learned & what's next

You now know how to automate remediation with SSM documents — turning security findings into automatic, auditable fixes. You can explain the core mechanism, create an Automation Document, and wire it to EventBridge for hands-off response. This lesson is lesson 28 in the Cloud security essentials path. Next up, we’ll explore automated compliance enforcement with AWS Config rules, where you’ll learn to prevent misconfigurations before they ever happen. Keep going — your cloud is getting more secure by the lesson!