AWS Secrets Integration

Integrate Horizon Epoch with AWS Secrets Manager, IAM authentication, and other AWS credential services.

AWS Secrets Manager

Basic Configuration

from horizon_epoch import Config

config = Config(
    aws_region="us-east-1"
).add_postgres(
    name="production",
    host="mydb.cluster-xxx.us-east-1.rds.amazonaws.com",
    database="production",
    aws_secret_id="horizon-epoch/production/postgres"
)

Secret Format

Store credentials in Secrets Manager:

{
    "username": "epoch_user",
    "password": "secret-password",
    "host": "mydb.cluster-xxx.us-east-1.rds.amazonaws.com",
    "port": 5432,
    "database": "production"
}

aws secretsmanager create-secret \
    --name horizon-epoch/production/postgres \
    --secret-string '{"username":"epoch_user","password":"secret-password"}'

Secret Rotation

Enable automatic rotation:

aws secretsmanager rotate-secret \
    --secret-id horizon-epoch/production/postgres \
    --rotation-lambda-arn arn:aws:lambda:us-east-1:xxx:function:SecretsRotation \
    --rotation-rules AutomaticallyAfterDays=30

Horizon Epoch automatically uses refreshed credentials.

RDS IAM Authentication

Connect to RDS without passwords using IAM:

Prerequisites

1. Enable IAM authentication on RDS:

aws rds modify-db-instance \
    --db-instance-identifier mydb \
    --enable-iam-database-authentication

2. Create IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "rds-db:connect",
            "Resource": "arn:aws:rds-db:us-east-1:123456789012:dbuser:mydb/epoch_user"
        }
    ]
}

3. Create database user:

CREATE USER epoch_user WITH LOGIN;
GRANT rds_iam TO epoch_user;

Configuration

config.add_postgres(
    name="production",
    host="mydb.cluster-xxx.us-east-1.rds.amazonaws.com",
    database="production",
    username="epoch_user",
    use_iam_auth=True
)

IAM Roles for S3

Instance Credentials (EC2/ECS/Lambda)

config.add_s3(
    name="datalake",
    bucket="my-data-bucket",
    use_instance_credentials=True  # Uses instance profile
)

Assume Role

For cross-account access:

config.add_s3(
    name="partner-data",
    bucket="partner-bucket",
    assume_role_arn="arn:aws:iam::999888777666:role/HorizonEpochAccess",
    external_id="partner-external-id"  # If required
)

Role Chaining

For complex permission structures:

config.add_s3(
    name="restricted",
    bucket="restricted-bucket",
    assume_roles=[
        "arn:aws:iam::111222333444:role/JumpRole",
        "arn:aws:iam::555666777888:role/TargetRole"
    ]
)

AWS SSO / Identity Center

For development with SSO:

# Configure SSO profile
aws configure sso

# Set profile
export AWS_PROFILE=my-sso-profile

# Or in config
config = Config(
    aws_profile="my-sso-profile"
)

Configuration File

# epoch.toml
[aws]
region = "us-east-1"
profile = "production"  # Optional: use specific profile

[storage.postgres.production]
host = "mydb.cluster-xxx.us-east-1.rds.amazonaws.com"
database = "production"
aws_secret_id = "horizon-epoch/production/postgres"

[storage.postgres.analytics]
host = "analytics.cluster-xxx.us-east-1.rds.amazonaws.com"
database = "analytics"
username = "epoch_user"
use_iam_auth = true

[storage.s3.datalake]
bucket = "company-datalake"
assume_role_arn = "arn:aws:iam::123456789012:role/DataLakeAccess"

Required IAM Permissions

Secrets Manager Access

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue",
                "secretsmanager:DescribeSecret"
            ],
            "Resource": "arn:aws:secretsmanager:us-east-1:*:secret:horizon-epoch/*"
        }
    ]
}

S3 Access

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

Assume Role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::*:role/HorizonEpoch*"
        }
    ]
}

Troubleshooting

Access Denied (Secrets Manager)

Error: AccessDeniedException when calling GetSecretValue

• Check IAM policy allows secretsmanager:GetSecretValue
• Verify secret ARN matches policy
• Check if secret has resource-based policy

IAM Auth Token Failed

Error: PAM authentication failed for user

• Verify IAM authentication is enabled on RDS
• Check user has rds_iam role granted
• Verify IAM policy has correct resource ARN

Credentials Expired

Error: ExpiredTokenException

• Instance credentials refresh automatically
• For assumed roles, check session duration
• For SSO, run aws sso login

Best Practices

1. Use IAM authentication over passwords when possible
2. Enable secret rotation in Secrets Manager
3. Use instance roles instead of access keys
4. Scope permissions narrowly
5. Enable CloudTrail for audit logging

Next Steps

• Vault Integration - Alternative secrets management
• Multi-Backend Setup - Multiple AWS accounts
• Configuration Reference - Full options