Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Connect Azure Blob Storage

Connect Horizon Epoch to Azure Blob Storage for Delta Lake tables.

CLI Configuration

# Add Azure storage backend
epoch storage add datalake \
    --type azure \
    --account "mystorageaccount" \
    --container "data"

Authentication Methods

Account Key

epoch storage add datalake \
    --type azure \
    --account "mystorageaccount" \
    --container "data" \
    --account-key "your-account-key"

Or via environment variable:

export AZURE_STORAGE_ACCOUNT_KEY="your-account-key"

epoch storage add datalake \
    --type azure \
    --account "mystorageaccount" \
    --container "data"

SAS Token

epoch storage add datalake \
    --type azure \
    --account "mystorageaccount" \
    --container "data" \
    --sas-token "?sv=2021-06-08&ss=b&srt=sco..."

Service Principal

export AZURE_CLIENT_ID="your-client-id"
export AZURE_CLIENT_SECRET="your-client-secret"
export AZURE_TENANT_ID="your-tenant-id"

epoch storage add datalake \
    --type azure \
    --account "mystorageaccount" \
    --container "data" \
    --auth-method service_principal

Managed Identity

For Azure VMs, AKS, or App Service with managed identity:

epoch storage add datalake \
    --type azure \
    --account "mystorageaccount" \
    --container "data" \
    --auth-method managed_identity

Register Delta Tables

epoch table add events \
    --location "azure://datalake/delta/events"

Table Storage Structure

Tables are stored as Delta Lake format:

container/
  delta/
    events/
      _delta_log/
      part-00000.parquet
      part-00001.parquet

Configuration File

# ~/.epoch/config.toml
[storage.azure.datalake]
account = "mystorageaccount"
container = "data"
prefix = "horizon-epoch/"
auth_method = "account_key"

Required Permissions

Minimum permissions for the storage account:

  • Storage Blob Data Reader (read operations)
  • Storage Blob Data Contributor (read/write operations)

Troubleshooting

AuthorizationFailure

  • Verify account name and credentials
  • Check container exists
  • Ensure identity has proper RBAC roles

Container Not Found

  • Verify container name is correct
  • Check storage account name

Slow Operations

  • Use regional endpoints when possible
  • Consider Azure Private Link for private network access