Stop Hardcoding Secrets: A Practical Guide to Vault on Kubernetes

You know the drill. Somewhere in your codebase, there's a hardcoded API key. Or a database password in a.env file that's been committed to Git three times. We've all been there. But in 2024, if you're still treating secrets like configuration values, you're fighting an uphill battle against every compliance framework, security audit, and production outage waiting to happen.

Enter HashiCorp Vault. It's not new, but it's matured into the de facto standard for secrets management—especially when paired with Kubernetes and cloud native infrastructure. Let's strip away the buzzwords and look at how this actually works in practice.

Why Vault Beats Environment Variables

Environment variables are leaky by design. They show up in process dumps, debug logs, and accidentally get printed to stdout during a CI run. Kubernetes Secrets are slightly better (base64 is not encryption), but they still sit in etcd, accessible to anyone with cluster admin rights.

Vault gives you: - Dynamic secrets — credentials that expire after use - Audit logging — every access is recorded - Encryption at rest and in transit — no plaintext anywhere - Fine-grained access control — not just "can read" vs "can't read"

The real killer feature? Dynamic secrets. Instead of a static database password that lives for years, Vault generates a temporary credential on demand. If it leaks, it's useless in 30 minutes.

The Kubernetes Integration: Sidecar or SDK?

You have two main paths to connect Kubernetes pods to Vault.

Sidecar Injection

The Vault Agent Injector runs as a mutating admission webhook. When you deploy a pod with specific annotations, it automatically injects a sidecar container that fetches secrets and writes them to a shared volume.

annotations:
  vault.hashicorp.com/agent-inject: "true"
  vault.hashicorp.com/role: "app-role"
  vault.hashicorp.com/agent-inject-secret-db-creds: "database/creds/app"

The application reads from /vault/secrets/db-creds as if it were a config file. Zero changes to your application code. This works brilliantly for legacy apps or teams that don't want to modify their container entrypoints.

SDK-Based Integration (The Better Way for New Apps)

For greenfield projects, use the Vault API directly in your app. Libraries exist for Go, Python, Node.js, and Java. The pod authenticates via its Kubernetes service account token and requests secrets on startup (or renews them periodically).

import hvac

client = hvac.Client()
client.auth.kubernetes(
    role='app-role',
    jwt=open('/var/run/secrets/kubernetes.io/serviceaccount/token').read()
)
secret = client.secrets.kv.v2.read_secret_version(
    path='myapp/production/db'
)

This approach gives you more control over rotation, caching, and error handling. But it means your application is now Vault-aware.

Cloud Platform-Specific Patterns

Each cloud provider has its own managed secrets service (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault). Should you use those instead of Vault?

The honest answer: Yes, if you're 100% single-cloud and never planning to migrate. But Vault gives you a consistent API across clouds and on-prem. It's the abstraction layer your security team will thank you for later.

AWS + Vault

The classic pattern uses Vault's AWS secrets engine to generate IAM credentials dynamically:

vault write aws/roles/deployer \
    credential_type=iam_user \
    policy_arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Your application never touches an IAM user permanently. Vault creates one, the app uses it for 15 minutes, then it's revoked.

GCP + Vault

GCP service accounts can be generated on-demand via Vault's GCP secrets engine. This is particularly useful for BigQuery or Cloud Storage access from Kubernetes pods that shouldn't have a permanent GCP identity.

The Practical Workflow

Here's what a production pipeline looks like:

1. Write secrets to Vault — engineers never touch them directly. Terraform or a CI pipeline manages the initial seed values.
2. Pods authenticate via Kubernetes service account — Vault validates the JWT and knows which cluster, namespace, and service account requested access.
3. Secrets arrive as files or API responses — never as environment variables.
4. Secrets are refreshed automatically — the sidecar or app code handles rotation without restarting the pod.
5. Audit trail exists for every secret access — "Who read the production database password at 3am? Oh, it was the CI pipeline running your deployment."

What Breaks If You Get This Wrong

• Overly permissive policies — if your role allows reading any secret in any path, you've just recreated the old environment variable problem.
• Missing rotation — dynamic secrets that don't renew break your app at the worst possible moment (3am on a Saturday).
• Vault cluster goes down — yes, it's a single point of failure. Run it on at least three nodes, back up the unseal keys, and test recovery procedures.

The Bottom Line

Secrets management isn't about finding the perfect tool. It's about building a system where credentials are ephemeral, access is auditable, and no human ever types a production password into a terminal.

Vault on Kubernetes with cloud provider integration isn't just a best practice—it's table stakes for any team shipping software that handles other people's data. Start with a single secret, automate its rotation, and watch your security posture improve with zero code changes.