Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Horizon Epoch

Git-like version control for data across heterogeneous storage systems.

Horizon Epoch brings the power of Git-style version control to your data infrastructure. Whether your data lives in relational databases, cloud object storage, or local filesystems, Horizon Epoch provides unified branching, merging, and history tracking.

Supported Storage Backends

  • Relational Databases - PostgreSQL, MySQL, SQL Server, SQLite
  • Cloud Object Storage - AWS S3, Azure Blob Storage, Google Cloud Storage
  • Local Storage - Local filesystem with Delta Lake format

Key Features

  • Multi-Storage Support - Work with data across 8 different storage backends
  • Zero-Copy Branching - Create instant branches without duplicating data
  • True Three-Way Merge - Smart conflict detection and resolution at the field level
  • Copy-on-Write Semantics - Efficient storage with minimal overhead
  • Git-Like Commands - Familiar CLI interface for data engineers

Who Is This For?

Horizon Epoch is designed for:

  • Data Engineers who need to safely test ETL changes before production
  • Analytics Teams who want to experiment with transformations in isolation
  • DevOps Engineers managing data across multiple environments
  • Anyone who has wished for Git-like workflows for their data

Quick Example

# Initialize a repository
epoch init my-data-repo --metadata-url "postgresql://localhost/horizon_epoch"

# Register a table to track
epoch table add users --location "postgresql://localhost/mydb/public.users"

# Create a feature branch
epoch branch create feature/new-schema

# Make changes, then commit
epoch commit -m "Add user preferences column"

# Merge back to main
epoch merge feature/new-schema

How It Works

Horizon Epoch tracks data versions through a metadata layer that sits alongside your existing storage:

  1. Metadata Database - Stores commit history, branch pointers, and version indices
  2. Storage Adapters - Connect to your data where it lives (PostgreSQL, MySQL, SQL Server, SQLite, S3, Azure, GCS, local filesystem)
  3. Copy-on-Write - Only modified records are stored separately on branches

This architecture means:

  • No data migration required
  • Works with your existing infrastructure
  • Minimal storage overhead for branching

Getting Started

Ready to try Horizon Epoch? Start with the Installation Guide and then follow the Quickstart tutorial.

Documentation Structure

This documentation is organized into several sections: