Secrets Outside Source Code

You've just finished a feature, you're about to commit, and then you see it: a hardcoded API_KEY = 'sk-...' sitting in the code like a ticking time bomb. It might work locally, but the moment that code hits a shared repo, every collaborator — and any attacker with access to the repo's history — now holds the keys to your kingdom. Committing secrets to source code is one of the most common and damaging security mistakes in modern development. This lesson teaches you how to manage secrets outside source code, a foundational practice that keeps credentials safe, reduces risk, and aligns with industry best practices.

The problem this lesson solves

When secrets such as API keys, database passwords, and private tokens are embedded directly in code, they become part of the codebase's history. Even if you delete them later, they remain in the commit history, accessible to anyone with repository access. This creates a severe security vulnerability: a single hardcoded secret can lead to unauthorized access, data breaches, and financial loss.

Consider the following scenario:

# danger_zone.py
DATABASE_PASSWORD = "Tr0ub4dor!"  # Never do this!
STRIPE_API_KEY = "sk_live_9f8a7d6c"  # Never do this!

Anyone who can see the code (including developers with read access, CI runners, and attackers if the repo is public or leaked) can extract these credentials. Even in private repos, insider threats and accidental leaks are real risks.

Moreover, hardcoding secrets makes rotation and management difficult. Changing a secret requires code changes, redeployment, and potential downtime — a painful process that encourages shortcuts.

The solution is to separate secrets from source code and load them at runtime from safe, external sources. This approach keeps secrets out of version control and enables secure, centralized management.

Core concept / mental model

Think of secrets as runtime configuration, not code. Just as you don't hardcode server addresses or feature flags, secrets should be injected from the environment.

Mental model: Your application is a stage, and secrets are the backstage passes. The audience (source code) should never see them. Instead, a secure manager (like a secret manager or environment variable) hands them to the performer (your app) only when needed.

• Environment variables: Simple, OS-level key-value pairs available to processes.
• Secret managers: Specialized tools (like HashiCorp Vault, AWS Secrets Manager, or Doppler) that store, encrypt, and control access to secrets.
• Configuration files (excluded from version control): Local config.ini or .env files that are gitignored.

All three approaches keep secrets out of source code, but they differ in complexity and security strength.

How it works step by step

Let's walk through the general workflow of managing secrets outside source code:

1. Identify all secrets in your project: API keys, database passwords, tokens, private keys, etc.
2. Remove them from your source code and replace them with references (e.g., os.environ"DB_PASSWORD"`).
3. Choose a secrets management solution based on your environment (dev vs. production) and requirements.
4. Store secrets securely in the chosen solution (environment variable, secret manager, etc.).
5. Load secrets at runtime in your application code.
6. Exclude secret-containing files from version control (e.g., .env files) using .gitignore.
7. Rotate and revoke secrets regularly, and audit access.

Step-by-step detail

• Environment variables: Set them on the server or in your shell. In Python, use os.getenv('SECRET_KEY') or os.environ['SECRET_KEY'].
• .env files: Store secrets locally in a file like .env (gitignored) and load them with a library like python-dotenv.
• Secret managers: Use vendor-specific SDKs or REST APIs to fetch secrets at runtime.

Best practice for git

• Add .env and other secret files to your .gitignore immediately.
• Never commit *.env or config.ini with secrets.
• Use pre-commit hooks to scan for secrets and block commits.

Hands-on walkthrough

Let's put theory into practice with two examples: using environment variables and using a .env file. First, ensure you have Python 3.10+ and pip installed.

Example 1: Environment variables (OS-level)

Set an environment variable in your terminal:

export MY_API_KEY="super-secret-value"

Then use it in Python:

import os

api_key = os.getenv("MY_API_KEY")
if not api_key:
    raise ValueError("MY_API_KEY environment variable is not set")

print(f"Using API key: {api_key}")
# Output: Using API key: super-secret-value

Example 2: Using a .env file with python-dotenv

This approach is ideal for local development. Create a .env file (gitignored):

DB_HOST=localhost
DB_PORT=5432
DB_USER=admin
DB_PASSWORD=Tr0ub4dor!

Install the library and load it:

pip install python-dotenv

Then in your app:

from dotenv import load_dotenv
import os

load_dotenv()  # Loads variables from .env into environment

db_host = os.getenv("DB_HOST", "localhost")
db_port = int(os.getenv("DB_PORT", 5432))
db_user = os.getenv("DB_USER")
db_pass = os.getenv("DB_PASSWORD")

print(f"Connecting to {db_user}@{db_host}:{db_port}...")
# Output: Connecting to admin@localhost:5432...

Important: Never commit the .env file. Add it to .gitignore:

.env

Example 3: Integrating with a secret manager (concept)

For production, a secret manager is recommended. Here's a pseudo-code example using AWS Secrets Manager (conceptual):

import boto3
from botocore.exceptions import ClientError

def get_secret(secret_name):
    client = boto3.client("secretsmanager")
    try:
        response = client.get_secret_value(SecretId=secret_name)
        return response["SecretString"]
    except ClientError as e:
        raise SystemExit(f"Unable to retrieve secret: {e}") from e

secret = get_secret("prod/db_password")
print(f"Retrieved secret length: {len(secret)}")

Compare options / when to choose what

When to choose what

• Prototyping or learning: Use .env files locally.
• Production: Use a dedicated secret manager.
• CI/CD pipelines: Inject secrets via environment variables in the pipeline configuration, not in the repo.
• Microservices / Kubernetes: Use Kubernetes Secrets and integrate with a secret manager for advanced needs.

Troubleshooting & edge cases

• Secret not found: Ensure the environment variable is set before the Python process starts. Double-check spelling and casing.
• .env file not loaded: Place load_dotenv() at the top of your script, and confirm the .env file path is correct if not in the same directory.
• git still tracks .env: If you've already committed it, remove it with git rm --cached .env, add to .gitignore, and purge history if necessary (use git filter-repo or tools like BFG).
• Secret exposure in logs: Never log secrets. Use logging filters to redact sensitive fields.
• Secret rotation: Use a secret manager to automate rotation. If using .env, manually rotate and update.

Pro tip: Use a tool like git-secrets or trufflehog to scan for accidentally committed secrets. Integrate it into your CI pipeline to block commits that contain patterns like -----BEGIN PRIVATE KEY-----.

What you learned & what's next

You now understand how to manage secrets outside source code: the risks of hardcoding, the mental model of secrets as runtime config, and the practical steps to use environment variables or .env files. You've also compared secret managers and learned troubleshooting tips to avoid common pitfalls.

In the next lesson, we'll build on this by exploring how to securely handle secrets in CI/CD pipelines and automate rotation — ensure your deployment processes don't reintroduce the same vulnerabilities.

Now, apply what you've learned: refactor a small project to move secrets out of code, and verify that no secret ever appears in your git history.