Geo-Distributed Deployment

Geo-distributed deployment runs one Lasso node per region, with each node measuring latency to upstream providers from its geographic location. Applications connect to their nearest node for region-optimized routing.

Architecture

Deployment Model

Deploy one Lasso node per region and route application traffic to the nearest node:

┌─────────────────────────────────────────────────────────────┐
│ US-East Region                                              │
│ ├─> Applications connect to us-east-1.rpc.example.com      │
│ ├─> Lasso Node (LASSO_NODE_ID=us-east-1)                   │
│ │   ├─> Measures latency from US-East                      │
│ │   ├─> Routes to fastest providers for this region        │
│ │   └─> Independent circuit breakers                       │
│ └─> Providers: Alchemy, Infura, LlamaRPC, etc.             │
├─────────────────────────────────────────────────────────────┤
│ EU-West Region                                              │
│ ├─> Applications connect to eu-west-1.rpc.example.com      │
│ ├─> Lasso Node (LASSO_NODE_ID=eu-west-1)                   │
│ │   ├─> Measures latency from EU-West                      │
│ │   ├─> Routes to fastest providers for this region        │
│ │   └─> Independent circuit breakers                       │
│ └─> Providers: Alchemy, Infura, LlamaRPC, etc.             │
├─────────────────────────────────────────────────────────────┤
│ AP-Southeast Region                                         │
│ ├─> Applications connect to ap-southeast-1.rpc.example.com │
│ ├─> Lasso Node (LASSO_NODE_ID=ap-southeast-1)              │
│ │   ├─> Measures latency from AP-Southeast                 │
│ │   ├─> Routes to fastest providers for this region        │
│ │   └─> Independent circuit breakers                       │
│ └─> Providers: Alchemy, Infura, LlamaRPC, etc.             │
└─────────────────────────────────────────────────────────────┘

Optional: Cluster Aggregation

Connect nodes via clustering for unified observability:

┌─────────────────────────────────────────────────────────────┐
│ Cluster Aggregation (optional)                              │
│ ├─> Dashboard shows metrics from all regions               │
│ ├─> Regional drill-down for comparison                     │
│ ├─> Topology monitoring (node health)                      │
│ └─> No impact on routing hot path                          │
└─────────────────────────────────────────────────────────────┘

Key Principles

Independent Node Operation

Each node runs a complete, isolated supervision tree
Routing decisions based on local latency measurements only
No cluster consensus or coordination in request hot path
A single node works standalone without clustering

Regional Latency Awareness

Passive benchmarking reveals which providers are fastest from each region
Provider performance varies significantly by geography
Example: Alchemy may be fastest in US-East but slower in EU-West
Each node discovers the optimal provider for its region

Observability-First Clustering

Clustering aggregates metrics for dashboards only
View cluster as unified whole or drill into regions
Identify providers struggling in specific regions
No routing impact: clustering is purely for observability

Configuration

Region-Specific Node IDs

Set LASSO_NODE_ID to match the deployment region:

# US-East node
export LASSO_NODE_ID="us-east-1"

# EU-West node
export LASSO_NODE_ID="eu-west-1"

# AP-Southeast node
export LASSO_NODE_ID="ap-southeast-1"

Importance:

Metrics are partitioned by {provider_id, node_id}
Dashboard groups performance by region
Circuit breaker states are per-node
Enables regional comparison in observability

Enable Clustering (Optional)

To aggregate metrics across regions, enable clustering on all nodes:

# Same on all nodes
export CLUSTER_DNS_QUERY="lasso.internal"
export CLUSTER_NODE_BASENAME="lasso"

# Unique per node
export LASSO_NODE_ID="us-east-1"  # or eu-west-1, ap-southeast-1, etc.

See Clustering for DNS setup details.

Traffic Routing

Route application traffic to the nearest Lasso node using:

GeoDNS

Use geographic DNS routing to direct clients to the nearest node:

us-east-1.rpc.example.com    → 203.0.113.10 (US-East IP)
eu-west-1.rpc.example.com    → 203.0.113.20 (EU-West IP)
ap-southeast-1.rpc.example.com → 203.0.113.30 (AP-Southeast IP)

# Geo-routed domain
rpc.example.com → resolves to nearest regional endpoint

Providers:

AWS Route 53 (geolocation routing)
Cloudflare (geo-routing)
Google Cloud DNS (geolocation routing)
Azure Traffic Manager

Anycast

Announce the same IP from multiple regions, with BGP routing traffic to the nearest location:

rpc.example.com → 203.0.113.1 (announced from all regions)

Characteristics:

Automatic nearest-region routing via BGP
No DNS-based routing required
Requires anycast-capable infrastructure (Cloudflare, cloud providers)

Load Balancer Geographic Routing

Use cloud load balancers with geographic routing:

AWS Global Accelerator: Anycast + health checks
GCP Global Load Balancer: Cross-region load balancing
Azure Front Door: Global HTTP load balancer
Cloudflare Load Balancer: Geo-steering

Deployment Examples

Kubernetes Multi-Region

Deploy to each region

# us-east-1 cluster
apiVersion: apps/v1
kind: Deployment
metadata:
  name: lasso
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: lasso
  template:
    metadata:
      labels:
        app: lasso
    spec:
      containers:
        - name: lasso
          image: myregistry.com/lasso-rpc:latest
          env:
            - name: SECRET_KEY_BASE
              valueFrom:
                secretKeyRef:
                  name: lasso-secrets
                  key: secret-key-base
            - name: PHX_HOST
              value: "us-east-1.rpc.example.com"
            - name: LASSO_NODE_ID
              value: "us-east-1"
            - name: CLUSTER_DNS_QUERY
              value: "lasso.production.svc.cluster.local"
            - name: CLUSTER_NODE_BASENAME
              value: "lasso"
          ports:
            - containerPort: 4000

Repeat for eu-west-1 and ap-southeast-1 clusters with region-specific LASSO_NODE_ID and PHX_HOST.

Configure cross-region service discovery

For cross-region clustering, use a global DNS service that resolves to all regional cluster IPs:

# Global DNS entry (managed by cloud provider)
lasso.internal.example.com:
  - us-east-1: 10.0.1.10, 10.0.1.11, 10.0.1.12
  - eu-west-1: 10.0.2.10, 10.0.2.11, 10.0.2.12
  - ap-southeast-1: 10.0.3.10, 10.0.3.11, 10.0.3.12

Set up GeoDNS for traffic routing

# Route 53 example
rpc.example.com:
  - Type: A
    Routing: Geolocation
    Records:
      - Location: North America → us-east-1-lb.example.com
      - Location: Europe → eu-west-1-lb.example.com
      - Location: Asia → ap-southeast-1-lb.example.com
      - Default → us-east-1-lb.example.com

Docker Swarm Multi-Region

# Deploy stack to each region
version: '3.8'

services:
  lasso:
    image: myregistry.com/lasso-rpc:latest
    environment:
      SECRET_KEY_BASE: ${SECRET_KEY_BASE}
      PHX_HOST: us-east-1.rpc.example.com  # Change per region
      PHX_SERVER: "true"
      LASSO_NODE_ID: us-east-1  # Change per region
      CLUSTER_DNS_QUERY: lasso.global.internal
      CLUSTER_NODE_BASENAME: lasso
    deploy:
      replicas: 3
      placement:
        constraints:
          - node.labels.region == us-east-1
    networks:
      - lasso-overlay

networks:
  lasso-overlay:
    driver: overlay
    attachable: true

VM/Bare Metal Multi-Region

Deploy to each region

# US-East node
export SECRET_KEY_BASE="your-secret"
export PHX_HOST="us-east-1.rpc.example.com"
export PHX_SERVER="true"
export LASSO_NODE_ID="us-east-1"
export CLUSTER_DNS_QUERY="lasso.internal.example.com"
export CLUSTER_NODE_BASENAME="lasso"

./bin/lasso start

Repeat for EU-West and AP-Southeast with region-specific values.

Configure service discovery

Use Consul, etcd, or cloud DNS to maintain lasso.internal.example.com resolving to all node IPs.

Set up load balancer

Configure a load balancer (HAProxy, nginx, cloud LB) with geographic routing to regional nodes.

Regional Performance Comparison

The dashboard provides regional drill-down when clustering is enabled:

Aggregate View

View performance across all regions:

Provider        | Avg Latency | Success Rate | Requests
----------------|-------------|--------------|----------
Alchemy         | 245ms       | 99.8%        | 150,000
Infura          | 289ms       | 99.5%        | 120,000
LlamaRPC        | 312ms       | 98.9%        | 80,000

Regional Drill-Down

Compare performance by region:

Provider: Alchemy

Region          | Avg Latency | Success Rate | Requests
----------------|-------------|--------------|----------
us-east-1       | 120ms       | 99.9%        | 60,000
eu-west-1       | 280ms       | 99.8%        | 50,000
ap-southeast-1  | 350ms       | 99.5%        | 40,000

This reveals:

Alchemy is fastest in US-East
Higher latency in EU-West and AP-Southeast
Consider regional provider alternatives (e.g., EU-specific providers)

Circuit Breaker Isolation

Circuit breakers are per-node, providing regional fault isolation: Scenario: Alchemy has an outage in EU-West but is healthy in other regions.

Region          | Circuit State | Reason
----------------|---------------|---------------------------
us-east-1       | :closed       | Healthy
eu-west-1       | :open         | Consecutive timeouts
ap-southeast-1  | :closed       | Healthy

Result:

EU-West node fails over to Infura or LlamaRPC
US-East and AP-Southeast continue using Alchemy
No cross-region coordination needed

Scaling Patterns

Horizontal Scaling Within Region

Run multiple nodes per region behind a load balancer:

US-East Region:
  Load Balancer (us-east-1.rpc.example.com)
    ├─> Lasso Node 1 (LASSO_NODE_ID=us-east-1)
    ├─> Lasso Node 2 (LASSO_NODE_ID=us-east-1)
    └─> Lasso Node 3 (LASSO_NODE_ID=us-east-1)

Characteristics:

Same LASSO_NODE_ID for all nodes in the region
Metrics aggregate at the region level
Stateless request handling (no sticky sessions needed)

Vertical Scaling

Increase resources per node:

BEAM scheduler cores: Set --erl "+S 8:8" for 8 schedulers
Memory: Increase container/VM memory for larger metric stores
Connection pools: Tune Finch pool sizes in config/runtime.exs

Monitoring and Observability

Health Checks Per Region

Configure health checks in each region’s load balancer:

# AWS ALB target group health check
HealthCheckPath: /api/health
HealthCheckIntervalSeconds: 30
HealthCheckTimeoutSeconds: 10
HealthyThresholdCount: 2
UnhealthyThresholdCount: 3

Regional Metrics

Key metrics to monitor per region:

Request rate: Requests/second per region
Latency: P50, P95, P99 per provider per region
Circuit breaker state: Open breakers indicate regional issues
Provider availability: Percentage of healthy providers
Cluster connectivity: Node states (:responding, :unresponsive)

Unified Dashboard

When clustering is enabled, access the dashboard from any node to see cluster-wide metrics:

http://us-east-1.rpc.example.com/dashboard  → See all regions
http://eu-west-1.rpc.example.com/dashboard  → See all regions
http://ap-southeast-1.rpc.example.com/dashboard → See all regions

Disaster Recovery

Regional Failover

If an entire region goes down:

GeoDNS/Load balancer detects unhealthy region
Traffic routes to next-nearest region
Clients reconnect to fallback region
No manual intervention required

Degraded Mode

If clustering fails but nodes are healthy:

Each node continues routing independently
Dashboard shows local metrics only
Applications unaffected (routing unchanged)
Fix cluster networking to restore aggregation

Best Practices

Use stable node IDs: Match cloud region codes (us-east-1, not node-1)
Deploy at least 3 regions: Provides global coverage and redundancy
Monitor regional performance: Identify geographic provider issues
Test cross-region failover: Simulate region outages
Plan network topology: Ensure low-latency cluster communication
Use regional provider endpoints: Some providers have regional URLs

Cost Optimization

Provider Selection by Region

Some providers offer better pricing or performance in specific regions:

# config/profiles/production.yml
chains:
  ethereum:
    providers:
      # Use regional endpoints where available
      - id: "alchemy_us"
        url: "https://eth-mainnet.g.alchemy.com/v2/${ALCHEMY_API_KEY}"
        regions: ["us-east-1", "us-west-2"]
      
      - id: "alchemy_eu"
        url: "https://eth-mainnet.eu.g.alchemy.com/v2/${ALCHEMY_API_KEY}"
        regions: ["eu-west-1", "eu-central-1"]

Request Volume Balancing

Distribute traffic based on regional request volumes:

Deploy more nodes in high-traffic regions
Use cheaper providers in low-traffic regions
Monitor cost per region in cloud billing

Next Steps

Clustering - Configure cross-region clustering
Production Checklist - Pre-launch verification
Architecture - Understand geo-distributed design

Get Started

Core Concepts

Configuration

Deployment

Observability

Advanced

​Architecture

​Deployment Model

​Optional: Cluster Aggregation

​Key Principles

​Independent Node Operation

​Regional Latency Awareness

​Observability-First Clustering

​Configuration

​Region-Specific Node IDs

​Enable Clustering (Optional)

​Traffic Routing

​GeoDNS

​Anycast

​Load Balancer Geographic Routing

​Deployment Examples

​Kubernetes Multi-Region

​Docker Swarm Multi-Region

​VM/Bare Metal Multi-Region

​Regional Performance Comparison

​Aggregate View

​Regional Drill-Down

​Circuit Breaker Isolation

​Scaling Patterns

​Horizontal Scaling Within Region

​Vertical Scaling

​Monitoring and Observability

​Health Checks Per Region

​Regional Metrics

​Unified Dashboard

​Disaster Recovery

​Regional Failover

​Degraded Mode

​Best Practices

​Cost Optimization

​Provider Selection by Region

​Request Volume Balancing

​Next Steps

Architecture

Deployment Model

Optional: Cluster Aggregation

Key Principles

Independent Node Operation

Regional Latency Awareness

Observability-First Clustering

Configuration

Region-Specific Node IDs

Enable Clustering (Optional)

Traffic Routing

GeoDNS

Anycast

Load Balancer Geographic Routing

Deployment Examples

Kubernetes Multi-Region

Docker Swarm Multi-Region

VM/Bare Metal Multi-Region

Regional Performance Comparison

Aggregate View

Regional Drill-Down

Circuit Breaker Isolation

Scaling Patterns

Horizontal Scaling Within Region

Vertical Scaling

Monitoring and Observability

Health Checks Per Region

Regional Metrics

Unified Dashboard

Disaster Recovery

Regional Failover

Degraded Mode

Best Practices

Cost Optimization

Provider Selection by Region

Request Volume Balancing

Next Steps