Trigger Lambda with S3 Events

Ever stared at your S3 bucket waiting for a new file to appear, then manually kicked off a script to process it? That's exactly the kind of friction that eats your day. In this lesson, you'll eliminate that manual step by wiring S3 to automatically fire a Lambda function the instant an object lands — the event-driven backbone of countless serverless Python backends.

The Problem This Lesson Solves

Think about any workflow that depends on files: uploading CSVs to a data lake, pushing images to a media pipeline, or dumping logs for analysis. Without automation, you have two choices: a developer babysits the bucket and manually triggers processing, or you write a polling script that checks S3 every few seconds and burns compute (and your patience) for nothing.

Both approaches fail at scale. Manual work doesn't ship. Polling adds latency and needless cost. What you really want is the event to be the trigger — the moment an object appears, the work starts. That's exactly what S3 event notifications to Lambda give you.

By the end of this lesson, you'll not only understand the architecture, but you'll have a working, testable setup that you can drop into any Python backend. We'll build on the S3 and Lambda concepts from earlier lessons in this track, so if you haven't created a bucket or a function yet, go back and do that first — this lesson assumes you can.

Core Concept / Mental Model

Let's build a picture you can carry in your head.

The Event-Driven Iceberg

Imagine an iceberg: the only part you see is the trigger — the S3 upload. But underneath, a whole chain reacts: S3 emits a notification, a notification destination (like Lambda) receives it, the function runs your Python code, and the result is your processed output.

Here are the three essential pieces:

• Event source: The S3 bucket. When a PUT request succeeds and an object is created, S3 can emit an event.
• Event notification: A configuration on the bucket that says, "When this happens, send a message to that destination." For Lambda, this is a special resource-based policy.
• Event handler: Your Lambda function. It receives a JSON payload describing the event and runs your code.

Pro tip: The single most important mental shift is to stop thinking "poll" and start thinking "react." Your code only runs when something meaningful happens — zero waste.

The Invocation Model

When an object is uploaded, S3 sends a test event (for console setups) or a real event. Lambda synchronously or asynchronously invokes your function, passing a payload like this:

{
  "Records": [
    {
      "eventName": "ObjectCreated:Put",
      "s3": {
        "bucket": {
          "name": "my-bucket"
        },
        "object": {
          "key": "uploads/report.csv"
        }
      }
    }
  ]
}

Your function reads event['Records'][0]['s3']['object']['key'] to know which file triggered it. That's the whole contract.

How It Works Step by Step

Let's walk the exact sequence from upload to function execution — know this cold and every config will make sense.

1. An object is created — A PUT, POST, or multipart upload completes in your S3 bucket.
2. S3 evaluates its notification configuration — The bucket has a list of event types (e.g., s3:ObjectCreated:*) and destinations.
3. S3 publishes the event — It writes an event message and delivers it to the configured destination. For Lambda, this is done via an internal topic that Lambda subscribes to.
4. Lambda invokes your function — The service sees the new message, matches it to your function, and runs it. The function receives the event JSON in the event parameter.
5. Your code executes — You process the object, maybe download it with boto3, transform, and store results elsewhere.
6. You can log and monitor — CloudWatch Logs captures your function's output; metrics show invocations and errors.

The cause-and-effect chain is direct: upload triggers event triggers invocation triggers code. There's no queue, no scheduled job, no hidden timer.

Pro tip: S3 event notifications are at-least-once — your function might be invoked more than once for the same object. Make your code idempotent (e.g., check if a processed file already exists).

Hands-On Walkthrough

Time to build it. We'll create a bucket, a Lambda function, a trigger, and test the whole loop. You'll need AWS CLI and Python 3.10+ (or the Lambda console — I'll show both).

1. Create a Bucket

aws s3 mb s3://my-event-demo-bucket-$RANDOM

Note the bucket name — S3 bucket names are globally unique. I'll use BUCKET_NAME as a placeholder.

2. Write Your Lambda Function

Create a file lambda_function.py with a simple handler that logs the event and, if the object is a .csv, prints a message:

import json
import urllib.parse

def lambda_handler(event, context):
    print("Received event: " + json.dumps(event, indent=2))

    # Get the bucket and object key from the event
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')

    if key.endswith('.csv'):
        print(f"CSV detected: s3://{bucket}/{key}")
    else:
        print(f"Non-CSV file: {key} — ignoring")

    return {
        'statusCode': 200,
        'body': json.dumps(f'Processed {key}')
    }

3. Create the Lambda Function via CLI

Package your code and create the function:

zip function.zip lambda_function.py

aws lambda create-function \
    --function-name s3-trigger-demo \
    --runtime python3.11 \
    --role arn:aws:iam::YOUR_ACCOUNT_ID:role/lambda-s3-role \
    --handler lambda_function.lambda_handler \
    --zip-file fileb://function.zip

You'll need an IAM role with basic Lambda execution permissions (CloudWatch Logs) — see earlier lessons.

4. Add the Trigger

Now wire S3 to Lambda. Grant S3 permission to invoke your function, then add the notification config:

aws lambda add-permission \
    --function-name s3-trigger-demo \
    --statement-id s3invoke \
    --action "lambda:InvokeFunction" \
    --principal s3.amazonaws.com \
    --source-arn arn:aws:s3:::BUCKET_NAME

aws s3api put-bucket-notification-configuration \
    --bucket BUCKET_NAME \
    --notification-configuration '{
      "LambdaFunctionConfigurations": [
        {
          "LambdaFunctionArn": "arn:aws:lambda:REGION:ACCOUNT_ID:function:s3-trigger-demo",
          "Events": ["s3:ObjectCreated:*"]
        }
      ]
    }'

Pro tip: If you use the AWS console, the UI does steps 4 and 5 for you in the bucket's Properties → Event notifications panel. But knowing the CLI makes it scriptable.

5. Test the End-to-End Flow

Upload a file and watch the logs:

aws s3 cp test.csv s3://BUCKET_NAME/

aws logs tail /aws/lambda/s3-trigger-demo --follow

Expected output (CloudWatch Logs):

Received event: {"Records": [{"eventName": "ObjectCreated:Put", ...}]}
CSV detected: s3://BUCKET_NAME/test.csv

You just triggered a Lambda with an S3 event. 🎉

Compare Options / When to Choose What

Not every automation needs S3 → Lambda. Here's how it stacks up against alternatives:

When to choose what: - Use S3 → Lambda directly when you have a single consumer workflow and want minimal moving parts. - Use S3 → SNS when the same event must trigger multiple actions (e.g., log, alert, process). - Use S3 → SQS when you need a buffer before processing, or if your function might fail and you want to retry later without losing messages.

For this lesson, stick with the direct trigger — it's the cleanest.

Troubleshooting & Edge Cases

Even simple setups break. Here's the fix-cheat-sheet.

"Access Denied" when S3 tries to invoke Lambda

• Symptom: In the console, the trigger shows "not configured" or invocations fail with AccessDeniedException.
• Fix: You forgot the resource-based policy (add-permission). Double-check the source-arn matches your bucket ARN exactly.

"The provided execution role does not have permissions to call the put-object" — wait, you don't use that

Remember your function needs to read from S3 to process the file. Even if the trigger works, downloading the object fails if the role lacks s3:GetObject.

• Fix: Attach a policy allowing s3:GetObject on that bucket.

Pro tip: Test your function with a manual payload (from the console's Test tab) before uploading a real file — it isolates trigger issues from code bugs.

Lambda never fires

1. Verify the event type matches. If you upload via multipart upload, the event is ObjectCreated:CompleteMultipartUpload — a prefix filter like ObjectCreated:* catches it.
2. Check the bucket's notification configuration is attached to the right bucket (easy to mix up if you copied a name).
3. Inspect CloudWatch Logs — if the function isn't invoked, the trigger config is the problem; if there's an error, fix your code.

Duplicate executions

S3 is at-least-once, so retries can double-invoke. Design your handler to be idempotent — e.g., check a database or S3 location for existing processed output before doing work.

What You Learned & What's Next

You've mastered the fundamental pattern: how to trigger a Lambda function from an S3 event, step by step. You can now:

• Explain the event-driven model and the payload contract
• Configure a bucket notification and attach a Lambda trigger via CLI or console
• Write a Python handler that reads Records and processes object keys
• Troubleshoot the most common issues (permissions, event types, idempotency)

This is the core of every serverless pipeline. Next up in the AWS Tutorial track, you'll extend this pattern to fan out with SNS or SQS — imagine your S3 upload triggering an email report to 50 users plus a database update. That's where the real world beckons.

Go build something that reacts, not polls.