GraphTally Guide

Learn about The Graph’s payment system, GraphTally (previously Timeline Aggregation Protocol)⁠. GraphTally provides fast, efficient microtransactions with minimized trust.

Overview

GraphTally is the payment system for The Graph, integrated into the core protocol through Graph Horizon. It enables efficient pay-per-query payments between gateways and indexers.

Key benefits:

• Efficient micropayments: Pay-per-query without high transaction costs
• Reduced onchain costs: Receipts aggregate into single blockchain transactions
• Payment guarantees: Indexers control receipt aggregation, ensuring payment for served queries
• Trustless operation: Decentralized gateways with cryptographic verification

How It Works

GraphTally enables a sender (gateway) to make multiple payments to a receiver (indexer) through Receipts, which aggregate into a Receipt Aggregate Voucher (RAV). RAVs can be verified and redeemed on the blockchain.

Payment flow:

1. Gateway sends a signed receipt with each query
2. indexer-service-rs validates and stores receipts in your database
3. indexer-tap-agent periodically aggregates receipts into RAVs
4. RAVs accumulate value as new receipts arrive
5. After allocation closure, the final RAV is redeemed onchain

RAV Details

• RAVs represent pending payments waiting for blockchain redemption
• The system aggregates receipts continuously, keeping unaggregated value below max_amount_willing_to_lose_grt
• Each RAV redeems once onchain, so final RAVs are created after allocation closure

Redeeming RAVs

RAV redemption is fully automated when running indexer-tap-agent and indexer-agent:

1. Indexer closes an allocation
2. After the recently-closed-allocation-buffer period, indexer-tap-agent aggregates remaining receipts into a final RAV marked as last
3. indexer-agent submits redeem transactions to the blockchain
4. During the finality-time period, indexer-agent monitors for blockchain reorganizations:
   • Reverted transactions are resubmitted
   • Confirmed transactions are marked final

Blockchain Addresses

Contracts

Gateway

Getting Started

Software Requirements

• indexer-agent: Latest version⁠
• indexer-service-rs: Latest release⁠
• indexer-tap-agent: Latest release⁠

Docker Images

1docker pull ghcr.io/graphprotocol/indexer-service-rs:latest2docker pull ghcr.io/graphprotocol/indexer-tap-agent:latest

Architecture Overview

Your indexer stack consists of:

• indexer-service-rs: Handles incoming queries, validates receipts, routes to graph-node. Stateless and horizontally scalable.
• indexer-tap-agent: Aggregates receipts into RAVs. Run exactly one instance.
• indexer-agent: Manages allocations and redeems RAVs onchain.

All three components share the same PostgreSQL database.

Configuration

Both indexer-service-rs and indexer-tap-agent use a shared TOML configuration file. Pass it with --config /path/to/config.toml.

Configuration Example

1[indexer]2indexer_address = "0x1111111111111111111111111111111111111111"3operator_mnemonic = "your twelve word mnemonic phrase here ..."45# For key rotation, you can specify multiple mnemonics:6# operator_mnemonics = [7#     "current mnemonic phrase here",8#     "previous mnemonic if you rotated keys"9# ]1011[database]12postgres_url = "postgresql://user:password@localhost:5432/indexer_db"1314[graph_node]15query_url = "http://graph-node:8000"16status_url = "http://graph-node:8000/graphql"1718[subgraphs.network]19# The Graph Network Subgraph (includes escrow data with Graph Horizon)20# Use query_url for hosted service, or deployment_id for local indexing (recommended)21query_url = "https://api.thegraph.com/subgraphs/name/graphprotocol/graph-network-arbitrum"22# deployment_id = "QmUVskWrz1ZiQZ76AtyhcfFDEH1ELnRpoyEhVL8p6NFTbR"2324# Note: With Graph Horizon, escrow data is included in the Network Subgraph.25# The separate [subgraphs.escrow] config is only needed for legacy V1 receipt processing.2627[blockchain]28chain_id = 4216129# V1 TAP Verifier (for legacy receipts during migration)30receipts_verifier_address = "0x33f9E93266ce0E108fc85DdE2f71dab555A0F05a"31# V2 Horizon TAP Verifier32receipts_verifier_address_v2 = "0x8f69F5C07477Ac46FBc491B1E6D91E2be0111A9e"33subgraph_service_address = "0xb2Bb92d0DE618878E438b55D5846cfecD9301105"3435# For testnet:36# chain_id = 42161437# receipts_verifier_address = "0xfC24cE7a4428A6B89B52645243662A02BA734ECF"38# receipts_verifier_address_v2 = "0x382863e7B662027117449bd2c49285582bbBd21B"39# subgraph_service_address = "0xc24A3dAC5d06d771f657A48B20cE1a671B78f26b"4041[tap]42max_amount_willing_to_lose_grt = "0.1"4344[tap.sender_aggregator_endpoints]45# Mainnet gateways46"0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467" = "https://tap-aggregator.network.thegraph.com"47"0xDD6a6f76eb36B873C1C184e8b9b9e762FE216490" = "https://tap-aggregator-arbitrum-one.graphops.xyz"4849# For testnet:50# "0xC3dDf37906724732FfD748057FEBe23379b0710D" = "https://tap-aggregator.testnet.thegraph.com"5152[horizon]53enabled = true

Environment Variable Overrides

Override any configuration value using environment variables:

1# Pattern: [PREFIX]__[SECTION]__[KEY]2# PREFIX: INDEXER_SERVICE or TAP_AGENT34export INDEXER_SERVICE__DATABASE__POSTGRES_URL="postgresql://..."5export TAP_AGENT__TAP__MAX_AMOUNT_WILLING_TO_LOSE_GRT="0.5"

Advanced Configuration

• Full configuration reference⁠
• Default values⁠

Logging

Set the log level using the RUST_LOG environment variable:

1# Recommended for production2export RUST_LOG=indexer_service=info,indexer_tap_agent=info34# For debugging5export RUST_LOG=indexer_service=debug,indexer_tap_agent=debug

Monitoring

Metrics

All components expose Prometheus metrics on port 7300:

• http://indexer-service:7300/metrics
• http://tap-agent:7300/metrics

Key metrics to monitor:

• Receipt Processing: Track receipt validation, aggregation rates, and failures
• RAV Creation: Monitor RAV request success/failure rates
• Unaggregated Fees: Ensure fees stay below max_amount_willing_to_lose_grt
• Sender Balances: Track escrow account balances and obligations

Grafana Dashboard

Import the official Grafana dashboard⁠ for monitoring:

• Receipt flow and aggregation status
• RAV creation and redemption lifecycle
• System performance and error rates
• Database query performance

Troubleshooting

Common issues and solutions:

1. RAV requests failing:

   • Check aggregator endpoint connectivity
   • Verify sender configuration in tap.sender_aggregator_endpoints
   • Review debug logs for specific error messages

2. Receipts not aggregating:

   • Ensure indexer-tap-agent is running (only one instance)
   • Check database connectivity
   • Verify max_amount_willing_to_lose_grt is not too high

3. High unaggregated fees:

   • Lower max_amount_willing_to_lose_grt to trigger more frequent aggregation
   • Check if specific senders are having aggregation issues
   • Monitor the Grafana dashboard for aggregation patterns

Horizon (TAP V2) Migration

Horizon is the next-generation Graph protocol with updated smart contracts and a new receipt format (V2). This section covers how to enable Horizon support in your indexer stack.

Critical: Both Services Must Be Configured

Horizon Configuration Checklist

Step 1: Obtain Horizon Contract Addresses

You need two new addresses for Horizon (check The Graph documentation for current values):

Step 2: Update Configuration File

Add the Horizon settings to your shared TOML configuration:

1[blockchain]2chain_id = 421613# Legacy V1 verifier (keep for existing receipts)4receipts_verifier_address = "0x33f9E93266ce0E108fc85DdE2f71dab555A0F05a"5# Horizon V2 addresses6receipts_verifier_address_v2 = "0x..." # Get from The Graph docs7subgraph_service_address = "0x..."     # Get from The Graph docs89[horizon]10enabled = true

Step 3: Configure Environment Variables (if not using config file)

If you use environment variables, you must set them for both services:

1# For indexer-service2export INDEXER_SERVICE__HORIZON__ENABLED=true3export INDEXER_SERVICE__BLOCKCHAIN__RECEIPTS_VERIFIER_ADDRESS_V2="0x..."4export INDEXER_SERVICE__BLOCKCHAIN__SUBGRAPH_SERVICE_ADDRESS="0x..."56# For tap-agent (REQUIRED - don't forget this!)7export TAP_AGENT__HORIZON__ENABLED=true8export TAP_AGENT__BLOCKCHAIN__RECEIPTS_VERIFIER_ADDRESS_V2="0x..."9export TAP_AGENT__BLOCKCHAIN__SUBGRAPH_SERVICE_ADDRESS="0x..."

Step 4: Verify Configuration

After restarting both services, check the logs to confirm Horizon is active:

indexer-service logs (expected):

1INFO indexer_service_rs::service: Horizon mode configured - checking if Horizon contracts are active2INFO indexer_service_rs::service: Horizon contracts detected - using Horizon migration mode

tap-agent logs (expected):

1INFO indexer_tap_agent::agent: Horizon mode configured - checking if Horizon contracts are active2INFO indexer_tap_agent::agent::sender_accounts_manager: horizon_active: true

tap-agent logs (PROBLEM - Horizon not enabled):

1INFO indexer_tap_agent::agent: Horizon not configured - using pure legacy mode2INFO indexer_tap_agent::agent::sender_accounts_manager: horizon_active: false

If you see “pure legacy mode” in tap-agent but have Horizon allocations, your receipts will not be processed!

Horizon Migration Modes

When Horizon is enabled and contracts are detected:

• Hybrid Migration Mode: The system accepts new V2 receipts while continuing to process existing V1 receipts
• V2 Receipts Only: New queries generate V2 receipts (stored in scalar_tap_receipts_v2 table)
• Legacy Processing: Existing V1 receipts continue to be aggregated until allocations close

Troubleshooting Horizon

1. TAP agent shows 0 allocations but you have Horizon allocations:

   • Check that TAP_AGENT__HORIZON__ENABLED=true is set
   • Verify subgraph_service_address is configured for tap-agent
   • Look for “pure legacy mode” in tap-agent logs

2. “mismatched” allocations in tap-agent logs:

   1total: 788, legacy: 0, horizon: 788, mismatched: 788, normalized: 0

   This indicates tap-agent is in legacy mode but all your allocations are Horizon. Enable Horizon on tap-agent.

3. Receipts stored but not aggregated:

   • Confirm both services have matching receipts_verifier_address_v2 values
   • Check that subgraph_service_address matches between services

Deployment Options

Docker Compose

Example compose configuration:

1services:2  indexer-service:3    image: ghcr.io/graphprotocol/indexer-service-rs:latest4    ports:5      - '7600:7600'6      - '7300:7300'7    volumes:8      - ./config.toml:/config.toml9    command: ['--config', '/config.toml']10    environment:11      - RUST_LOG=indexer_service=info12    restart: unless-stopped1314  tap-agent:15    image: ghcr.io/graphprotocol/indexer-tap-agent:latest16    ports:17      - '7301:7300'18    volumes:19      - ./config.toml:/config.toml20    command: ['--config', '/config.toml']21    environment:22      - RUST_LOG=indexer_tap_agent=info23    restart: unless-stopped

Kubernetes

For Kubernetes deployments, see the Graph Launchpad charts⁠ which include:

• Helm charts for both services
• ConfigMap templates for configuration
• Service and ingress definitions
• Prometheus ServiceMonitor resources

Best Practices

1. Database Performance:

   • Ensure your PostgreSQL instance has adequate resources
   • Monitor query performance through metrics
   • Regular maintenance of receipt tables

2. High Availability:

   • Run multiple indexer-service-rs instances behind a load balancer
   • Keep indexer-tap-agent as a single instance with proper monitoring
   • Use database connection pooling

3. Security:

   • Store operator mnemonic securely (use environment variables)
   • Restrict database access to indexer components only
   • Monitor for unusual receipt patterns

4. Capacity Planning:

   • Monitor receipt growth rate
   • Plan database storage accordingly
   • Set appropriate max_amount_willing_to_lose_grt based on query volume

Additional Resources

• indexer-rs Repository⁠
• TAP Core Documentation⁠
• The Graph Discord - Get help in the #indexers channel
• Support⁠ - Report issues or get help