Geo-Distributed Deployment

Geo-distributed deployment runs one Lasso node per region, with each node measuring latency to upstream providers from its geographic location. Applications connect to their nearest node for region-optimized routing.

Architecture

Deployment Model

Deploy one Lasso node per region and route application traffic to the nearest node:

┌─────────────────────────────────────────────────────────────┐
│ US-East Region                                              │
│ ├─> Applications connect to us-east-1.rpc.example.com      │
│ ├─> Lasso Node (LASSO_NODE_ID=us-east-1)                   │
│ │   ├─> Measures latency from US-East                      │
│ │   ├─> Routes to fastest providers for this region        │
│ │   └─> Independent circuit breakers                       │
│ └─> Providers: Alchemy, Infura, LlamaRPC, etc.             │
├─────────────────────────────────────────────────────────────┤
│ EU-West Region                                              │
│ ├─> Applications connect to eu-west-1.rpc.example.com      │
│ ├─> Lasso Node (LASSO_NODE_ID=eu-west-1)                   │
│ │   ├─> Measures latency from EU-West                      │
│ │   ├─> Routes to fastest providers for this region        │
│ │   └─> Independent circuit breakers                       │
│ └─> Providers: Alchemy, Infura, LlamaRPC, etc.             │
├─────────────────────────────────────────────────────────────┤
│ AP-Southeast Region                                         │
│ ├─> Applications connect to ap-southeast-1.rpc.example.com │
│ ├─> Lasso Node (LASSO_NODE_ID=ap-southeast-1)              │
│ │   ├─> Measures latency from AP-Southeast                 │
│ │   ├─> Routes to fastest providers for this region        │
│ │   └─> Independent circuit breakers                       │
│ └─> Providers: Alchemy, Infura, LlamaRPC, etc.             │
└─────────────────────────────────────────────────────────────┘

Optional: Cluster Aggregation

Connect nodes via clustering for unified observability:

┌─────────────────────────────────────────────────────────────┐
│ Cluster Aggregation (optional)                              │
│ ├─> Dashboard shows metrics from all regions               │
│ ├─> Regional drill-down for comparison                     │
│ ├─> Topology monitoring (node health)                      │
│ └─> No impact on routing hot path                          │
└─────────────────────────────────────────────────────────────┘

Key Principles

Independent Node Operation

Each node runs a complete, isolated supervision tree
Routing decisions based on local latency measurements only
No cluster consensus or coordination in request hot path
A single node works standalone without clustering

Regional Latency Awareness

Passive benchmarking reveals which providers are fastest from each region
Provider performance varies significantly by geography
Example: Alchemy may be fastest in US-East but slower in EU-West
Each node discovers the optimal provider for its region

Observability-First Clustering

Clustering aggregates metrics for dashboards only
View cluster as unified whole or drill into regions
Identify providers struggling in specific regions
No routing impact: clustering is purely for observability

Configuration

Region-Specific Node IDs

Set LASSO_NODE_ID to match the deployment region:

# US-East node
export LASSO_NODE_ID="us-east-1"

# EU-West node
export LASSO_NODE_ID="eu-west-1"

# AP-Southeast node
export LASSO_NODE_ID="ap-southeast-1"

Importance:

Metrics are partitioned by {provider_id, node_id}
Dashboard groups performance by region
Circuit breaker states are per-node
Enables regional comparison in observability

Enable Clustering (Optional)

To aggregate metrics across regions, enable clustering on all nodes:

# Same on all nodes
export CLUSTER_DNS_QUERY="lasso.internal"
export CLUSTER_NODE_BASENAME="lasso"

# Unique per node
export LASSO_NODE_ID="us-east-1"  # or eu-west-1, ap-southeast-1, etc.

See Clustering for DNS setup details.

Traffic Routing

Route application traffic to the nearest Lasso node using:

GeoDNS

Use geographic DNS routing to direct clients to the nearest node:

us-east-1.rpc.example.com    → 203.0.113.10 (US-East IP)
eu-west-1.rpc.example.com    → 203.0.113.20 (EU-West IP)
ap-southeast-1.rpc.example.com → 203.0.113.30 (AP-Southeast IP)

# Geo-routed domain
rpc.example.com → resolves to nearest regional endpoint

Providers:

AWS Route 53 (geolocation routing)
Cloudflare (geo-routing)
Google Cloud DNS (geolocation routing)
Azure Traffic Manager

Anycast

Announce the same IP from multiple regions, with BGP routing traffic to the nearest location:

rpc.example.com → 203.0.113.1 (announced from all regions)

Characteristics:

Automatic nearest-region routing via BGP
No DNS-based routing required
Requires anycast-capable infrastructure (Cloudflare, cloud providers)

Load Balancer Geographic Routing

Use cloud load balancers with geographic routing:

AWS Global Accelerator: Anycast + health checks
GCP Global Load Balancer: Cross-region load balancing
Azure Front Door: Global HTTP load balancer
Cloudflare Load Balancer: Geo-steering

Deployment Examples

Kubernetes Multi-Region

Deploy to each region

# us-east-1 cluster
apiVersion: apps/v1
kind: Deployment
metadata:
  name: lasso
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: lasso
  template:
    metadata:
      labels:
        app: lasso
    spec:
      containers:
        - name: lasso
          image: myregistry.com/lasso-rpc:latest
          env:
            - name: SECRET_KEY_BASE
              valueFrom:
                secretKeyRef:
                  name: lasso-secrets
                  key: secret-key-base
            - name: PHX_HOST
              value: "us-east-1.rpc.example.com"
            - name: LASSO_NODE_ID
              value: "us-east-1"
            - name: CLUSTER_DNS_QUERY
              value: "lasso.production.svc.cluster.local"
            - name: CLUSTER_NODE_BASENAME
              value: "lasso"
          ports:
            - containerPort: 4000

Repeat for eu-west-1 and ap-southeast-1 clusters with region-specific LASSO_NODE_ID and PHX_HOST.

Configure cross-region service discovery

For cross-region clustering, use a global DNS service that resolves to all regional cluster IPs:

# Global DNS entry (managed by cloud provider)
lasso.internal.example.com:
  - us-east-1: 10.0.1.10, 10.0.1.11, 10.0.1.12
  - eu-west-1: 10.0.2.10, 10.0.2.11, 10.0.2.12
  - ap-southeast-1: 10.0.3.10, 10.0.3.11, 10.0.3.12

Set up GeoDNS for traffic routing

# Route 53 example
rpc.example.com:
  - Type: A
    Routing: Geolocation
    Records:
      - Location: North America → us-east-1-lb.example.com
      - Location: Europe → eu-west-1-lb.example.com
      - Location: Asia → ap-southeast-1-lb.example.com
      - Default → us-east-1-lb.example.com

Docker Swarm Multi-Region

# Deploy stack to each region
version: '3.8'

services:
  lasso:
    image: myregistry.com/lasso-rpc:latest
    environment:
      SECRET_KEY_BASE: ${SECRET_KEY_BASE}
      PHX_HOST: us-east-1.rpc.example.com  # Change per region
      PHX_SERVER: "true"
      LASSO_NODE_ID: us-east-1  # Change per region
      CLUSTER_DNS_QUERY: lasso.global.internal
      CLUSTER_NODE_BASENAME: lasso
    deploy:
      replicas: 3
      placement:
        constraints:
          - node.labels.region == us-east-1
    networks:
      - lasso-overlay

networks:
  lasso-overlay:
    driver: overlay
    attachable: true

VM/Bare Metal Multi-Region

Deploy to each region

# US-East node
export SECRET_KEY_BASE="your-secret"
export PHX_HOST="us-east-1.rpc.example.com"
export PHX_SERVER="true"
export LASSO_NODE_ID="us-east-1"
export CLUSTER_DNS_QUERY="lasso.internal.example.com"
export CLUSTER_NODE_BASENAME="lasso"

./bin/lasso start

Repeat for EU-West and AP-Southeast with region-specific values.

Configure service discovery

Use Consul, etcd, or cloud DNS to maintain lasso.internal.example.com resolving to all node IPs.

Set up load balancer

Configure a load balancer (HAProxy, nginx, cloud LB) with geographic routing to regional nodes.

Regional Performance Comparison

The dashboard provides regional drill-down when clustering is enabled:

Aggregate View

View performance across all regions:

Provider        | Avg Latency | Success Rate | Requests
----------------|-------------|--------------|----------
Alchemy         | 245ms       | 99.8%        | 150,000
Infura          | 289ms       | 99.5%        | 120,000
LlamaRPC        | 312ms       | 98.9%        | 80,000

Regional Drill-Down

Compare performance by region:

Provider: Alchemy

Region          | Avg Latency | Success Rate | Requests
----------------|-------------|--------------|----------
us-east-1       | 120ms       | 99.9%        | 60,000
eu-west-1       | 280ms       | 99.8%        | 50,000
ap-southeast-1  | 350ms       | 99.5%        | 40,000

This reveals:

Alchemy is fastest in US-East
Higher latency in EU-West and AP-Southeast
Consider regional provider alternatives (e.g., EU-specific providers)

Circuit Breaker Isolation

Circuit breakers are per-node, providing regional fault isolation: Scenario: Alchemy has an outage in EU-West but is healthy in other regions.

Region          | Circuit State | Reason
----------------|---------------|---------------------------
us-east-1       | :closed       | Healthy
eu-west-1       | :open         | Consecutive timeouts
ap-southeast-1  | :closed       | Healthy

Result:

EU-West node fails over to Infura or LlamaRPC
US-East and AP-Southeast continue using Alchemy
No cross-region coordination needed

Scaling Patterns

Horizontal Scaling Within Region

Run multiple nodes per region behind a load balancer:

US-East Region:
  Load Balancer (us-east-1.rpc.example.com)
    ├─> Lasso Node 1 (LASSO_NODE_ID=us-east-1)
    ├─> Lasso Node 2 (LASSO_NODE_ID=us-east-1)
    └─> Lasso Node 3 (LASSO_NODE_ID=us-east-1)

Characteristics:

Same LASSO_NODE_ID for all nodes in the region
Metrics aggregate at the region level
Stateless request handling (no sticky sessions needed)

Vertical Scaling

Increase resources per node:

BEAM scheduler cores: Set --erl "+S 8:8" for 8 schedulers
Memory: Increase container/VM memory for larger metric stores
Connection pools: Tune Finch pool sizes in config/runtime.exs

Monitoring and Observability

Health Checks Per Region

Configure health checks in each region’s load balancer:

# AWS ALB target group health check
HealthCheckPath: /api/health
HealthCheckIntervalSeconds: 30
HealthCheckTimeoutSeconds: 10
HealthyThresholdCount: 2
UnhealthyThresholdCount: 3

Regional Metrics

Key metrics to monitor per region:

Request rate: Requests/second per region
Latency: P50, P95, P99 per provider per region
Circuit breaker state: Open breakers indicate regional issues
Provider availability: Percentage of healthy providers
Cluster connectivity: Node states (:responding, :unresponsive)

Unified Dashboard

When clustering is enabled, access the dashboard from any node to see cluster-wide metrics:

http://us-east-1.rpc.example.com/dashboard  → See all regions
http://eu-west-1.rpc.example.com/dashboard  → See all regions
http://ap-southeast-1.rpc.example.com/dashboard → See all regions

Disaster Recovery

Regional Failover

If an entire region goes down:

GeoDNS/Load balancer detects unhealthy region
Traffic routes to next-nearest region
Clients reconnect to fallback region
No manual intervention required

Degraded Mode

If clustering fails but nodes are healthy:

Each node continues routing independently
Dashboard shows local metrics only
Applications unaffected (routing unchanged)
Fix cluster networking to restore aggregation

Best Practices

Use stable node IDs: Match cloud region codes (us-east-1, not node-1)
Deploy at least 3 regions: Provides global coverage and redundancy
Monitor regional performance: Identify geographic provider issues
Test cross-region failover: Simulate region outages
Plan network topology: Ensure low-latency cluster communication
Use regional provider endpoints: Some providers have regional URLs

Cost Optimization

Provider Selection by Region

Some providers offer better pricing or performance in specific regions:

# config/profiles/production.yml
chains:
  ethereum:
    providers:
      # Use regional endpoints where available
      - id: "alchemy_us"
        url: "https://eth-mainnet.g.alchemy.com/v2/${ALCHEMY_API_KEY}"
        regions: ["us-east-1", "us-west-2"]
      
      - id: "alchemy_eu"
        url: "https://eth-mainnet.eu.g.alchemy.com/v2/${ALCHEMY_API_KEY}"
        regions: ["eu-west-1", "eu-central-1"]

Request Volume Balancing

Distribute traffic based on regional request volumes:

Deploy more nodes in high-traffic regions
Use cheaper providers in low-traffic regions
Monitor cost per region in cloud billing

Next Steps

Clustering - Configure cross-region clustering
Production Checklist - Pre-launch verification
Architecture - Understand geo-distributed design

​Architecture

​Deployment Model

​Optional: Cluster Aggregation

​Key Principles

​Independent Node Operation

​Regional Latency Awareness

​Observability-First Clustering

​Configuration

​Region-Specific Node IDs

​Enable Clustering (Optional)

​Traffic Routing

​GeoDNS

​Anycast

​Load Balancer Geographic Routing

​Deployment Examples

​Kubernetes Multi-Region

​Docker Swarm Multi-Region

​VM/Bare Metal Multi-Region

​Regional Performance Comparison

​Aggregate View

​Regional Drill-Down

​Circuit Breaker Isolation

​Scaling Patterns

​Horizontal Scaling Within Region

​Vertical Scaling

​Monitoring and Observability

​Health Checks Per Region

​Regional Metrics

​Unified Dashboard

​Disaster Recovery

​Regional Failover

​Degraded Mode

​Best Practices

​Cost Optimization

​Provider Selection by Region

​Request Volume Balancing

​Next Steps