Mount Secrets as Files

You've containerized your Python app, wired up ConfigMaps, and even created a Secret. But how does your Python code actually read that secret? Environment variables are the obvious answer, but they carry hidden risks: they show up in kubectl exec output, they can get logged accidentally, and they're stored in the pod's spec in plain sight. If your savvy attacker is staring at os.environ, your database password is already compromised. This lesson shows you a more robust pattern: mounting secrets as files — the standard way to deliver sensitive values to containers in production. You'll learn why it's safer, how to set it up, and how to read those files from Python. By the end, you'll be able to ship Python services that read credentials securely from the filesystem, just like the pros who run real workloads on Kubernetes.

The Problem This Lesson Solves

Imagine this: you have a Python FastAPI app that talks to PostgreSQL. The connection string lives in a Secret. The obvious move? Pass the Secret as an environment variable.

# deployment.yaml (risky approach)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment
spec:
  template:
    spec:
      containers:
        - name: api
          image: my-api:latest
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: db-secret
                  key: database-url

This works, but it bleeds secrets into the pod's environment. Now try kubectl exec into the pod and runenv— there's your password. If your app crashes and the logs dumpos.environ, you've just leaked the credential to your log aggregator. Worse, if someone gets read access to the pod spec (say, via a CI pipeline or a README), they see the key reference, and while the actual value isn't in the spec, the environment itself becomes a hunting ground for anyone who canexec`.

The pain points: - Secrets in env are visible to anyone who can exec into the pod. - Logs and error reporting tools can silently capture environment variables. - Debugging sessions (print(os.environ)) leak secrets into terminals and log files. - Many libraries (e.g., Django, SQLAlchemy) prefer reading a file path, not an env var.

Mounting secrets as files solves all of this by putting the secret value into a read-only file inside your container. Your Python process can access it without it ever touching the environment.

Core Concept / Mental Model

Think of Kubernetes Secrets like a safe deposit box. The Secret object stores a small piece of sensitive data — a password, a token, a private key — in etcd, encrypted if you've enabled encryption at rest. The pod is your app's home. To open the safe deposit box, you don't dump all the contents into your living room (that's env vars). Instead, you take out one item and place it on the coffee table for the app to read when needed. That coffee table is the mounted file.

A volume mount does exactly that: Kubernetes takes the Secret's data and materializes it as files on disk inside the container. Each key in the Secret becomes a file; the value is the file's content. For example, a Secret with correctly decoded bytes for api-key becomes a file at /etc/secrets/api-key containing the actual key. Your Python code can then read that file with normal open() and os.read(), nothing magical.

The beauty: the file is read-only (unless you explicitly make it writable), and it's isolated from the environment. Even if someone runs env in the pod, they see nothing. They'd have to know the exact file path and have filesystem access — which is far more restricted in any sane security model.

How It Works Step by Step

1. Define your Secret — Create a Secret object (via YAML manifest, kubectl create secret, or the Python Kubernetes client). The Secret's data field holds base64-encoded values. Optionally, you can use stringData for plain text and let Kubernetes encode it.

2. Reference the Secret in a volume — In your Pod or Deployment spec, add a volumes entry with secret specifying the secret name. This tells Kubernetes: "I want a volume backed by this Secret."

3. Mount the volume into the container — In the container's volumeMounts, choose a mount path like /etc/secrets. For each key in the Secret, a file appears at that path. The default permission is 0444 (read-only for all), but you can override with defaultMode.

4. Read the file in Python — In your app, open the file, read the bytes, and decode to a string. Wrap it in a try/except to fail gracefully if the file is missing.

5. Consume the secret — Use the string to build connection strings, sign requests, or initialize clients.

Let's see this in action.

Hands-On Walkthrough

Create a Secret and a Deployment that mounts it as a file, then run a Python script to read it.

Step 1: Create a Secret

Save this as db-secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  # base64 of 'postgresql://user:pass@localhost:5432/appdb'
  database-url: cG9zdGdyZXNxbDovL3VzZXI6cGFzc0Bsb2NhbGhvc3Q6NTQzMi9hcHBkYg==

Apply it:

kubectl apply -f db-secret.yaml

Step 2: Deploy a Python app that reads the secret file

Create a small Flask or plain Python script that reads the file at startup. Save this as app.py:

# app.py
import os
import sys

def load_secret_from_file(path):
    """Read and return the secret from a mounted file."""
    try:
        with open(path, 'r') as f:
            return f.read().strip()
    except FileNotFoundError:
        print(f"FATAL: Secret file {path} not found", file=sys.stderr)
        raise

def main():
    db_url = load_secret_from_file('/etc/secrets/database-url')
    print("Database URL loaded successfully (not shown for security)")
    print(f"URL starts with: {db_url[:10]}...")

if __name__ == "__main__":
    main()

Now create the Deployment YAML that mounts the secret:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: secret-reader-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: secret-reader
  template:
    metadata:
      labels:
        app: secret-reader
    spec:
      containers:
        - name: secret-reader
          image: python:3.10-slim
          command: ["python", "-c", "exec(open('app.py').read())"]
          volumeMounts:
            - name: secrets
              mountPath: /etc/secrets
              readOnly: true
      volumes:
        - name: secrets
          secret:
            secretName: db-secret

Apply the deployment:

kubectl apply -f deployment.yaml

Check the pod's logs:

kubectl logs deployment/secret-reader-deployment

Expected output:

Database URL loaded successfully (not shown for security)
URL starts with: postgresql:

Your Python process now reads the secret from a file — no environment variables involved.

Step 3: Verify what's in the file system

You can exec into the pod to see the file:

kubectl exec -it deployment/secret-reader-deployment -- ls -l /etc/secrets

You'll see something like:

-r--r--r-- 1 root root 45 Jun  1 12:34 database-url

Then try cat /etc/secrets/database-url to see the actual value (for debugging only — never do this in production logs).

Alternatively, use the awk command to read the file with output redirection to prove it's not in the environment:

kubectl exec -it deployment/secret-reader-deployment -- sh -c 'env' | grep database

The grep finds nothing — the secret isn't in env.

Compare Options / When to Choose What

Environment variables vs. mounted files — here's a head-to-head:

When to choose files: - Whenever the secret is sensitive (password, token, key). - When your app or library expects a file path (e.g., GOOGLE_APPLICATION_CREDENTIALS pointing to a JSON file). - When you need to update secrets without restarting the pod (though note: with subPath the file won't update, so prefer full volume mount for live updates).

When env vars are fine: - Non-secret configuration (e.g., feature flags that can be public). - very small strings that won't cause operational risk if accidentally logged.

Variation: subPath mounting — You can mount a single file instead of a whole directory using subPath. This is handy when you have a config directory with many files and you only want one secret file. But be warned: subPath changes the update behavior — the file will NOT be updated automatically when the Secret changes; you'd need to restart the pod.

Another variation: using the secret volume in an init container. You can clone the secrets to a shared emptyDir volume, then mount that into your app container. This lets you transform the secret (e.g., decode JSON) before the main app starts.

Troubleshooting & Edge Cases

1. Secret "db-secret" not found — The secret doesn't exist in the namespace, or it's in a different namespace. Check with kubectl get secrets -n your-namespace and ensure your pod is in the same namespace.

2. The file permission is too restrictive — By default, files get mode 0444, which is world-readable. For sensitive keys, you might want 0400 (owner-only read). Use defaultMode in the volume spec.

volumes:
  - name: secrets
    secret:
      secretName: db-secret
      defaultMode: 0400  # owner read only

3. The secret value has a trailing newline — When you create a secret with echo -n, you avoid the newline. But if you use stringData with a newline, the file content will have a trailing \n. Your Python read() will include it, so .strip() is your friend.

4. The file doesn't update when the Secret changes — With a full volume mount (no subPath), the kubelet periodically syncs changes (usually within a minute). But if you used subPath, the file is statically bound and won't update. For automatic updates, avoid subPath for secrets.

5. Permission denied when reading the file — If your container runs as non-root (best practice), ensure the file is readable. The default 0444 is readable by all users, but if you set defaultMode to 0400, your non-root user can't read it. Check your security context and adjust defaultMode accordingly.

6. kubectl exec complains about the secret file being empty — Check the data field encoding. Secrets are base64-encoded. If you accidentally put plain text in data, the file will contain garbage (because Kubernetes decodes the base64). Use stringData in YAML to avoid manual encoding.

apiVersion: v1
kind: Secret
metadata:
  name: db-secret
stringData:
  database-url: postgresql://user:pass@localhost:5432/appdb

7. Your Python app crashes with 'invalid file descriptor' — This is typical if you try to read a file after the pod restarts and the file is gone. Always wrap file reads in try/except with a clear error. Consider using an entrypoint script that waits for the file to appear if your init order is uncertain.

8. The pod fails with The volume "secrets" is invalid — Double-check that your volume name matches between volumes and volumeMounts. A typo in name will trigger this.

What You Learned & What's Next

You've mastered a core security best practice: mounting secrets as files in your Python containers. You learned why environment variables are too risky for sensitive data, how Kubernetes turns a Secret into a read-only file inside your pod, and how to read that file from Python with a simple open() call. You also compared this approach with env vars, learned about subPath and permission modes, and debugged common issues like missing secrets and stale files.

You now know how to protect database credentials, API keys, and tokens in your Python services. This is a fundamental skill for any production Kubernetes deployment.

Next up in the track: The next lesson dives into Persistent Volumes and Persistent Volume Claims. You'll learn how to give your Python apps durable storage that survives pod restarts — critical for stateful services like databases, file uploads, or any app that needs to retain data across failures. Check out the next lesson in the Kubernetes for Python Developers series.

Happy Kubernetes-ing, and keep those secrets safe!