Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Credential Providers

Horizon Epoch supports multiple methods for providing credentials to storage backends. This guide covers when to use each provider and how to configure them.

Overview

Credential providers abstract how secrets are retrieved:

  • Static - Hardcoded credentials (dev/testing only)
  • Environment - From environment variables
  • File - From files on disk
  • Vault - From HashiCorp Vault
  • AWS - From AWS Secrets Manager or IAM

Static Credentials

Use for development and testing only.

from horizon_epoch import Config

config = Config(
    metadata_url="postgresql://localhost/horizon_epoch"
).add_postgres(
    name="mydb",
    url="postgresql://user:password@localhost/db"
)

# epoch.toml
[storage.postgres.mydb]
url = "postgresql://user:password@localhost/db"

Environment Variables

Recommended for containers and CI/CD.

export EPOCH_POSTGRES_MYDB_URL="postgresql://user:pass@host/db"
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."

config.add_postgres(
    name="mydb",
    url="${EPOCH_POSTGRES_MYDB_URL}"  # Variable substitution
)

# epoch.toml - variables are substituted at runtime
[storage.postgres.mydb]
url = "${EPOCH_POSTGRES_MYDB_URL}"

File-Based Credentials

For credentials managed by external tools (e.g., Kubernetes secrets).

config.add_postgres(
    name="mydb",
    credentials_file="/run/secrets/db-password"
)

# epoch.toml
[storage.postgres.mydb]
host = "db.example.com"
database = "production"
username = "app_user"
password_file = "/run/secrets/db-password"

File format (JSON):

{
    "password": "secret-value"
}

Or plain text:

secret-value

Credential Caching

Credentials are cached to reduce external calls:

config = Config(
    credential_cache_ttl=300,  # 5 minutes
    credential_cache_max_size=100
)

For frequently rotating credentials:

config = Config(
    credential_cache_ttl=60,  # 1 minute
    credential_refresh_before_expiry=30  # Refresh 30s before expiry
)

Credential Refresh

Some providers support automatic refresh:

# AWS credentials auto-refresh when using IAM roles
config.add_s3(
    name="datalake",
    bucket="my-bucket",
    use_instance_credentials=True  # Auto-refresh enabled
)

Provider Priority

When multiple credential sources are available:

  1. Explicit configuration (URL, password)
  2. Environment variables
  3. Credential files
  4. External providers (Vault, AWS)
  5. Instance credentials (IAM roles)

Security Best Practices

1. Never Commit Credentials

# .gitignore
epoch.toml
.env
*.pem
*.key

2. Use Separate Credentials Per Environment

# epoch.dev.toml
[storage.postgres.mydb]
url = "${DEV_DATABASE_URL}"

# epoch.prod.toml
[storage.postgres.mydb]
url = "${PROD_DATABASE_URL}"

3. Rotate Credentials Regularly

Use providers that support rotation:

  • AWS Secrets Manager with rotation
  • Vault dynamic database credentials
  • Short-lived IAM credentials

4. Audit Credential Access

Enable logging for credential operations:

config = Config(
    log_credential_access=True,  # Log when credentials are accessed
    log_level="info"
)

5. Use Least Privilege

Create dedicated credentials for Horizon Epoch with only required permissions.

Troubleshooting

Credential Not Found

Error: Credential 'EPOCH_POSTGRES_MYDB_URL' not found
  • Check environment variable is set
  • Verify file path exists and is readable
  • Check Vault/AWS connectivity

Credential Expired

Error: Credential expired and refresh failed
  • Check IAM role permissions
  • Verify Vault token is valid
  • Check system clock sync (for time-based auth)

Permission Denied

Error: Permission denied reading credential file
  • Check file permissions: chmod 600 /path/to/credential
  • Verify process user has read access

Next Steps