Secret Scanning and Preventing Credentials Leaks in GitHub

In 2023, GitGuardian detected 10 million secrets leaked on public GitHub repositories — a 67% increase from the previous year. API keys, database passwords, JWT secrets, and cloud credentials ending up in git history is one of the most common and most preventable security incidents in software development. I have found committed secrets twice in codebases I inherited for maintenance — both times by running gitleaks against the full git history. In one case, a live production database password had been in git history for 14 months. The credential was still active. This post covers how to prevent it from happening and what to do when it already has.

Why Secrets End Up in Git

Developers commit secrets not because they are careless but because the path of least resistance leads there. Hardcoded credentials in config files that get committed. Environment variables stored in .env files that are not in .gitignore. API keys pasted into code 'temporarily.' Secrets in Kubernetes YAML files committed to infrastructure repos. Credentials in CI/CD pipeline configuration files. The root cause is that secrets management requires extra steps — reading from environment, setting up a secrets manager, configuring CI secrets — and under time pressure, developers cut those steps. Prevention requires making the secure path easier than the insecure path.

GitHub Secret Scanning

GitHub Secret Scanning automatically scans public repositories for known secret patterns (AWS keys, GitHub tokens, Stripe keys, 200+ patterns) and notifies repository owners when found. For private repositories, Secret Scanning is available on GitHub Advanced Security (included in GitHub Enterprise and GitHub Team). When GitHub detects a secret, it sends an alert in the Security tab and notifies the credential issuer (AWS, Stripe, etc.) who may automatically revoke the credential. This is a great last line of defense, but it only catches secrets after they are pushed — prevention is better.

# Install gitleaks
brew install gitleaks
# or: go install github.com/gitleaks/gitleaks/v8@latest

# Scan full git history (run once on every repo)
gitleaks detect --source=. --log-opts="--all" --verbose

# Scan only staged files (for pre-commit hook use)
gitleaks protect --staged

# Configure with pre-commit framework
# .pre-commit-config.yaml:
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.21.2
    hooks:
      - id: gitleaks

# Install pre-commit on developer machines
pip install pre-commit
pre-commit install  # installs hooks in .git/hooks/

# Custom gitleaks config to allowlist false positives
# .gitleaks.toml:
[allowlist]
description = "Test credentials and placeholders"
regexes = [
  '''YOUR_API_KEY_HERE''',
  '''test_key_[a-z0-9]{8}''',
  '''REPLACE_WITH_ACTUAL_KEY''',
]
paths = [
  '''tests/fixtures/''',
  '''docs/examples/''',
]

# GitHub Actions: block PRs with secrets
# .github/workflows/security.yml:
name: Secret Scanning
on: [push, pull_request]
jobs:
  gitleaks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # full history needed
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Pre-Commit Secret Scanning with gitleaks

Gitleaks is an open-source tool that scans git repositories for secrets using regex patterns. Install as a pre-commit hook to block commits containing secrets. The tool catches AWS access keys, private RSA keys, GitHub tokens, database connection strings, JWT secrets, and 100+ other patterns. False positives are low for real-world projects — the patterns are specific enough to avoid flagging placeholder values like 'YOUR_API_KEY_HERE'.

Gitleaks Configuration

Configure gitleaks to scan your entire commit diff before committing. Add it to your pre-commit framework config. For repositories with legitimate test credentials (test API keys that are intentionally in fixtures), use the allowlist configuration to whitelist specific strings or file patterns. The gitleaks.toml config file lets you add custom patterns for company-specific secret formats (internal API key patterns, custom token formats) not covered by the default rule set.

Infrastructure and CI/CD Secret Management

For CI/CD pipelines: never hardcode secrets in workflow files. Use GitHub Actions secrets (encrypted, not visible in logs), or a proper secrets manager. For Kubernetes: use Kubernetes Secrets (base64 encoded, not encrypted by default — enable Secrets Encryption at Rest in etcd), or use External Secrets Operator with Vault, AWS Secrets Manager, or GCP Secret Manager as the backend. For application secrets in production: use environment variables injected by your deployment platform, not committed config files. This applies even to 'non-sensitive' config — keeping the pattern consistent prevents the 'oh this one is fine to commit' exception that becomes a habit.

# Remove a secret from git history using git-filter-repo
# Install: pip install git-filter-repo

# STEP 1: Rotate the credential FIRST — assume it's compromised
# (AWS, Stripe, GitHub, database — all of them)

# STEP 2: Remove the file from history
git filter-repo --path .env --invert-paths
# Or remove a specific string pattern:
git filter-repo --replace-text <(echo 'sk-abc123==>REDACTED')

# STEP 3: Force push rewritten history (coordinate with team first!)
git push origin --force --all
git push origin --force --tags

# STEP 4: Notify all team members to re-clone
# Old clones have the secret in their local history

# STEP 5: Verify on GitHub
# Check Security → Secret Scanning alerts — confirm no active alerts

# STEP 6: Check access logs for the leaked credential
# AWS: CloudTrail → filter by AccessKeyId
# Stripe: Dashboard → Developers → Logs → filter by API key
# Database: PostgreSQL log_connections + pg_audit for credential use

Scanning Existing Repository History

Run gitleaks on the full history of every existing repository as a one-time audit: `gitleaks detect --source=. --log-opts='--all'`. This scans all commits, not just recent ones. Run it on every repo before adding the pre-commit hook — you need to know your baseline. For large repositories with thousands of commits, the scan may take a few minutes. Treat findings as incidents: rotate every credential found, regardless of how old the commit is. Old API keys are often still valid — companies rarely rotate credentials without a reason.

Defense-in-Depth for Secrets

My layered secrets security: (1) Developer workstations: pre-commit gitleaks hook catches secrets before commit. (2) CI/CD pipeline: gitleaks runs as a PR check in GitHub Actions, blocking merges if secrets are detected. (3) Repository level: GitHub Secret Scanning enabled on all repos, alerts sent to security Slack channel. (4) Infrastructure: secrets stored in Vault or cloud secrets manager, injected as environment variables at runtime. (5) Rotation policy: all secrets have a maximum 90-day rotation policy enforced by automation. (6) Audit logging: all secret access logged in the secrets manager. This defense-in-depth means a secret must bypass 6 layers before it creates a lasting exposure.

Related Articles

• OWASP Top 10 Developer Checklist
• Docker Container Security Scanning