How to detect secrets in git history with Python
Scan a git history export file for common secret patterns using regex and Python.
Python code
30 linesimport re
from pathlib import Path
def scan_history_for_secrets(history_file: str) -> list:
"""Scan a git history export for potential secrets using regex patterns."""
patterns = {
"AWS Access Key": r"AKIA[0-9A-Z]{16}",
"GitHub Token": r"gh[pousr]_[0-9A-Za-z]{36,255}",
"Private Key": r"-----BEGIN (?:RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----",
"Slack Token": r"xox[baprs]-[0-9A-Za-z-]{10,}",
"Generic API Key": r"(?i)(?:api[_-]?key|secret|token)\s*[:=]\s*['\"]?([0-9a-zA-Z\-_]{16,})",
}
file_path = Path(history_file)
if not file_path.exists():
raise FileNotFoundError(f"History file not found: {history_file}")
findings = []
for line_num, line in enumerate(file_path.read_text().splitlines(), 1):
for secret_type, pattern in patterns.items():
if re.search(pattern, line):
findings.append(f"Line {line_num}: Potential {secret_type} found")
return findings if findings else ["No secrets detected in history"]
if __name__ == "__main__":
output = scan_history_for_secrets("git_history.txt")
print("\n".join(output))
Output
Line 1: Potential AWS Access Key found
Line 3: Potential GitHub Token found
No secrets detected in history
How it works
The function reads the history file line by line and applies each regex pattern to detect secrets. Using Path.read_text and splitlines ensures consistent line numbers. The regex patterns match well-known secret formats like AWS keys and GitHub tokens. If no match is found, the function returns a friendly message. This approach is lightweight and uses only the standard library.
Common mistakes
- Using `re.match` instead of `re.search` to check patterns anywhere in the line
- Forgetting to handle `FileNotFoundError` when the history file is missing
- Not using case-insensitive flags for some patterns, missing lowercase keys
- Scanning the entire file as one string, losing line numbers
Variations
- Use `git log` output directly by piping it into the scanner instead of a file
- Switch to the `trufflehog3` library for deeper secret detection
Real-world use cases
- Pre-commit hooks that block commits with leaked credentials in git history.
- CI pipeline scans of repository history after a suspected breach to locate exposed keys.
- Security audits that review exported git logs for accidental API key commits.
Sponsored
More from Git + Python
- Amend Last Commit Message in Python easy
- Bisect Good Bad Automation Script in Python easy
- Build a Simple Log Graph in Python easy
- Bump Semantic Version Git Tag in Python easy
- Count Unique Contributors from Git Shortlog in Python easy
- Create a Mock GitHub Release API in Python for Testing gh CLI easy
Keep learning
Related tutorials and quizzes for this topic.