Don't Store Secrets in Code: A Practical Guide to Python Configuration Management

Every Python project starts simple. You have a few constants—a database URL, an API key, maybe a log level. So you toss them in a config.py file. Six months later, that file has grown into a tangled mess of environment variables, JSON files, and half-baked if-else chains for development versus production.

I've been there. It's a rite of passage. But configuration management doesn't have to be painful. Let's look at how to handle it properly in Python projects, from small scripts to sprawling applications.

The Three Enemies of Good Config

Before we dive into solutions, here's what we're fighting against:

1. Hardcoded secrets – That password = "letmein" sitting in your repository
2. Config sprawl – Settings scattered across 14 different files with no clear pattern
3. Environment mismatch – "But it works on my machine" syndrome

The goal is a configuration system that's predictable, secure, and dead simple to use.

Environment Variables: The Foundation

Environment variables should be your default choice for anything sensitive or environment-specific. Python's os.environ is built-in, but raw usage is ugly:

import os

db_url = os.environ.get("DATABASE_URL", "postgres://localhost:5432/mydb")

This works, but it's verbose and error-prone. Better alternatives:

python-dotenv for Development

# .env file (never commit this!)
DATABASE_URL=postgres://prod:secret@aws.com/db
API_KEY=sk-abc123

# config.py
from dotenv import load_dotenv
import os

load_dotenv()  # Loads .env file if present

DATABASE_URL = os.getenv("DATABASE_URL")
API_KEY = os.getenv("API_KEY")

The beauty of python-dotenv is that it only loads the .env file locally. In production, you'd set real environment variables through your orchestrator (Docker, Kubernetes, Heroku, etc.).

Pydantic Settings: The Modern Way

For projects that need validation, type coercion, or hierarchical configs, Pydantic's BaseSettings is a game-changer:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    app_name: str = "My App"
    debug: bool = False
    database_url: str
    redis_url: str = "redis://localhost:6379"
    allowed_hosts: list[str] = ["localhost"]

settings = Settings()

# Usage elsewhere
def create_app():
    app = FastAPI()
    if settings.debug:
        print("DEBUG mode enabled")
    return app

Pydantic Settings automatically reads from environment variables (with case-insensitive matching) and provides: - Type validation (catches debug="yes" before it reaches your code) - Default values - .env file support - Nested configuration models for complex setups

Configuration File Patterns

Sometimes environment variables aren't enough—especially when you have complex, structured config like feature flags or nested service configurations.

YAML Config with Dynaconf

Dynaconf is like a Swiss Army knife for configuration. It handles multiple environments, secrets vaults, and file formats:

# settings.yaml
default:
  app:
    name: MyApp
    port: 8080

  database:
    host: localhost
    port: 5432

development:
  debug: true

production:
  debug: false

from dynaconf import Dynaconf

settings = Dynaconf(
    settings_files=['settings.yaml'],
    environments=True,
    load_dotenv=True
)

print(settings.database.host)  # "localhost"
print(settings.current_env)    # "development" (default)

The Twelve-Factor App Approach

If you're building web applications, follow the Twelve-Factor App methodology: Store config in the environment. This means:

• No config files in your repository
• Everything comes from environment variables
• Environment-specific defaults live in your deployment system, not your code

# config.py
import os
import json

class Config:
    @property
    def database_url(self):
        return os.environ["DATABASE_URL"]

    @property
    def redis_cluster(self):
        nodes = os.environ.get("REDIS_NODES", "[]")
        return json.loads(nodes)

config = Config()

Handling Secrets Safely

Never, ever commit secrets. Here's a practical checklist:

• Use .env files (add .env to .gitignore)
• For team secrets, use a vault service like HashiCorp Vault, AWS Secrets Manager, or Infisical
• Generate example config files with placeholder values: config.example.yaml
• Pre-commit hooks can catch accidental secret commits using tools like detect-secrets or git-secrets

Real-World Example: A Service with Multiple Environments

Here's a complete pattern I use for production services:

# config.py
from pydantic_settings import BaseSettings
from functools import lru_cache

class DatabaseSettings(BaseSettings):
    url: str
    pool_size: int = 10
    timeout: int = 30

class ServiceSettings(BaseSettings):
    name: str = "payment-gateway"
    port: int = 8000
    login_rate_limit: int = 5  # requests per minute

class Settings(BaseSettings):
    environment: str = "development"
    debug: bool = False
    database: DatabaseSettings = DatabaseSettings()
    services: ServiceSettings = ServiceSettings()

    class Config:
        env_nested_delimiter = "__"

@lru_cache
def get_settings() -> Settings:
    return Settings()

Usage across your codebase:

from config import get_settings

settings = get_settings()
print(settings.database.url)
print(settings.services.login_rate_limit)

The @lru_cache ensures the config is loaded once and reused—avoids parsing .env files on every import.

The One Rule You Can't Break

Whatever approach you choose, make your config explicit and inspectable. When something breaks at 3 AM, you should be able to:

1. Print all current configuration values (except secrets)
2. Know exactly where each value came from (env var? config file? default?)
3. Change it without redeploying (environment variable changes take effect on process restart)

Good configuration management doesn't just prevent bugs—it prevents the midnight panic of wondering which server has which settings.