GraphTally Guide

Learn about The Graph’s payment system, GraphTally (previously Timeline Aggregation Protocol)⁠. GraphTally provides fast, efficient microtransactions with minimized trust.

Overview

GraphTally is the payment system for The Graph, integrated into the core protocol through Graph Horizon. It enables efficient pay-per-query payments between gateways and indexers.

Key benefits:

• Efficient micropayments: Pay-per-query without high transaction costs
• Reduced onchain costs: Receipts aggregate into single blockchain transactions
• Payment guarantees: Indexers control receipt aggregation, ensuring payment for served queries
• Trustless operation: Decentralized gateways with cryptographic verification

How It Works

GraphTally enables a sender (gateway) to make multiple payments to a receiver (indexer) through Receipts, which aggregate into a Receipt Aggregate Voucher (RAV). RAVs can be verified and redeemed on the blockchain.

Payment flow:

1. Gateway sends a signed receipt with each query
2. indexer-service-rs validates and stores receipts in your database
3. indexer-tap-agent periodically aggregates receipts into RAVs
4. RAVs accumulate value as new receipts arrive
5. After allocation closure, the final RAV is redeemed onchain

RAV Details

• RAVs represent pending payments waiting for blockchain redemption
• The system aggregates receipts continuously, keeping unaggregated value below max_amount_willing_to_lose_grt
• Each RAV redeems once onchain, so final RAVs are created after allocation closure

Redeeming RAVs

RAV redemption is fully automated when running indexer-tap-agent and indexer-agent:

1. Indexer closes an allocation
2. After the recently-closed-allocation-buffer period, indexer-tap-agent aggregates remaining receipts into a final RAV marked as last
3. indexer-agent submits redeem transactions to the blockchain
4. During the finality-time period, indexer-agent monitors for blockchain reorganizations:
   • Reverted transactions are resubmitted
   • Confirmed transactions are marked final

Allocation Reconciliation

The tap-agent automatically reconciles RAV redemptions using the network subgraph. It queries paymentsEscrowTransactions to track which RAVs have been redeemed onchain and detects blockchain reorganizations that may revert redemption transactions. Reconciliation runs every 5 minutes by default (configurable via tap.allocation_reconciliation_interval_secs).

Blockchain Addresses

Contracts

Gateway

Getting Started

Software Requirements

• indexer-agent: Latest version⁠
• indexer-service-rs v2.0.0+: Latest release⁠
• indexer-tap-agent v2.0.0+: Latest release⁠

Docker Images

1docker pull ghcr.io/graphprotocol/indexer-service-rs:latest2docker pull ghcr.io/graphprotocol/indexer-tap-agent:latest

Architecture Overview

Your indexer stack consists of:

• indexer-service-rs: Handles incoming queries, validates receipts, routes to graph-node. Stateless and horizontally scalable.
• indexer-tap-agent: Aggregates receipts into RAVs and reconciles redemptions. Run exactly one instance.
• indexer-agent: Manages allocations and redeems RAVs onchain.

All three components share the same PostgreSQL database.

Configuration

Both indexer-service-rs and indexer-tap-agent use a shared TOML configuration file. Pass it with --config /path/to/config.toml.

Configuration Example

1[indexer]2indexer_address = "0x1111111111111111111111111111111111111111"3operator_mnemonic = "your twelve word mnemonic phrase here ..."45# For key rotation, you can specify multiple mnemonics.6# All configured mnemonics are tried when creating attestation signers,7# allowing allocations created with different operator keys to work.8# The first mnemonic is used as the primary identity on the /info endpoint.9# operator_mnemonics = [10#     "previous mnemonic phrase here if you rotated keys"11# ]1213[database]14postgres_url = "postgresql://user:password@localhost:5432/indexer_db"1516[graph_node]17query_url = "http://graph-node:8000"18status_url = "http://graph-node:8000/graphql"1920[subgraphs.network]21# The Graph Network Subgraph (includes escrow data for Horizon)22# Use query_url for hosted service, or deployment_id for local indexing (recommended)23query_url = "https://api.thegraph.com/subgraphs/name/graphprotocol/graph-network-arbitrum"24# deployment_id = "QmUVskWrz1ZiQZ76AtyhcfFDEH1ELnRpoyEhVL8p6NFTbR"2526[blockchain]27chain_id = 4216128receipts_verifier_address_v2 = "0x8f69F5C07477Ac46FBc491B1E6D91E2be0111A9e"29subgraph_service_address = "0xb2Bb92d0DE618878E438b55D5846cfecD9301105"30# Legacy V1 verifier (deprecated in v2.0.0, can be removed from your config):31# receipts_verifier_address = "0x33f9E93266ce0E108fc85DdE2f71dab555A0F05a"3233# For testnet:34# chain_id = 42161435# receipts_verifier_address_v2 = "0x382863e7B662027117449bd2c49285582bbBd21B"36# subgraph_service_address = "0xc24A3dAC5d06d771f657A48B20cE1a671B78f26b"3738[tap]39max_amount_willing_to_lose_grt = "0.1"4041[tap.sender_aggregator_endpoints]42# Mainnet gateways43"0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467" = "https://tap-aggregator.network.thegraph.com"44"0xDD6a6f76eb36B873C1C184e8b9b9e762FE216490" = "https://tap-aggregator-arbitrum-one.graphops.xyz"4546# For testnet:47# "0xC3dDf37906724732FfD748057FEBe23379b0710D" = "https://tap-aggregator.testnet.thegraph.com"4849[horizon]50enabled = true

Required Configuration Fields

The following fields are required for operation:

Environment Variable Overrides

Override any configuration value using environment variables:

1# Pattern: [PREFIX]__[SECTION]__[KEY]2# PREFIX: INDEXER_SERVICE or TAP_AGENT34export INDEXER_SERVICE__DATABASE__POSTGRES_URL="postgresql://..."5export TAP_AGENT__TAP__MAX_AMOUNT_WILLING_TO_LOSE_GRT="0.5"

Advanced Configuration

• Full configuration reference⁠
• Default values⁠

Receipt Validation

indexer-service-rs validates every incoming receipt before storing it. The following checks are performed:

• Allocation eligibility: The receipt targets a valid, active allocation
• Payer check: The receipt’s payer field matches a known sender
• Data service check: The receipt’s data_service field matches the configured subgraph_service_address
• Service provider check: The receipt’s service_provider matches the indexer’s address
• Sender balance check: The sender has sufficient escrow balance
• Value check: The receipt value does not exceed max_receipt_value_grt
• Timestamp check: The receipt timestamp is within acceptable bounds

Logging

Set the log level using the RUST_LOG environment variable:

1# Recommended for production2export RUST_LOG=indexer_service=info,indexer_tap_agent=info34# For debugging5export RUST_LOG=indexer_service=debug,indexer_tap_agent=debug

Monitoring

Endpoints

indexer-service-rs exposes the following HTTP endpoints:

The /healthz endpoint checks connectivity to the PostgreSQL database and graph-node, returning a JSON response:

1{2  "status": "healthy",3  "checks": {4    "database": { "status": "healthy" },5    "graph_node": { "status": "healthy" }6  }7}

Returns HTTP 200 when all checks pass, or HTTP 503 when any dependency is unhealthy.

Metrics

All components expose Prometheus metrics on port 7300:

• http://indexer-service:7300/metrics
• http://tap-agent:7300/metrics

Key metrics to monitor:

• Receipt Processing: Track receipt validation, aggregation rates, and failures
• RAV Creation: Monitor RAV request success/failure rates
• Unaggregated Fees: Ensure fees stay below max_amount_willing_to_lose_grt
• Sender Balances: Track escrow account balances and obligations

Grafana Dashboard

Import the official Grafana dashboard⁠ for monitoring:

• Receipt flow and aggregation status
• RAV creation and redemption lifecycle
• System performance and error rates
• Database query performance

Troubleshooting

Common issues and solutions:

1. RAV requests failing:

   • Check aggregator endpoint connectivity
   • Verify sender configuration in tap.sender_aggregator_endpoints
   • Review debug logs for specific error messages

2. Receipts not aggregating:

   • Ensure indexer-tap-agent is running (only one instance)
   • Check database connectivity
   • Verify max_amount_willing_to_lose_grt is not too high

3. High unaggregated fees:

   • Lower max_amount_willing_to_lose_grt to trigger more frequent aggregation
   • Check if specific senders are having aggregation issues
   • Monitor the Grafana dashboard for aggregation patterns

4. Service startup fails with missing configuration:

   • Ensure horizon.enabled = true is set
   • Verify both receipts_verifier_address_v2 and subgraph_service_address are configured
   • Check that the network subgraph is accessible

Upgrading to v2.0.0

Version 2.0.0 of indexer-service-rs and indexer-tap-agent removes support for legacy V1 receipts. All receipt processing now uses Graph Horizon (GraphTally) exclusively.

What Changed

• Horizon is required: The [horizon].enabled field must be set to true. Configs with enabled = false are rejected at startup.
• V1 receipt handling removed: The system no longer processes legacy V1 receipts or queries the escrow subgraph for V2 operations.
• receipts_verifier_address (V1) is deprecated: This field is now optional and unused. You can remove it from your config.
• receipts_verifier_address_v2 is required: This must be set to the Horizon TAP Verifier contract address.
• subgraph_service_address is required: This must be set to the SubgraphService contract address.
• Escrow data from network subgraph: V2 escrow accounts are read from the network subgraph, not a separate escrow subgraph.

Migration Checklist

1. Update your configuration:

   • Add receipts_verifier_address_v2 and subgraph_service_address under [blockchain] if not already present
   • Ensure [horizon].enabled = true
   • You can remove receipts_verifier_address (the old V1 verifier) from your config
   • The [subgraphs.escrow] section can be removed unless you serve the escrow subgraph proxy via serve_escrow_subgraph = true in the [service] section (most indexers do not need this)

2. Ensure all V1 RAVs are redeemed: Before upgrading, confirm that all pending V1 RAVs have been redeemed onchain. The new version cannot process V1 receipts.

3. Update Docker images:

   1docker pull ghcr.io/graphprotocol/indexer-service-rs:latest2docker pull ghcr.io/graphprotocol/indexer-tap-agent:latest

4. Restart both services with the updated configuration.

Deployment Options

Docker Compose

Example compose configuration:

1services:2  indexer-service:3    image: ghcr.io/graphprotocol/indexer-service-rs:latest4    ports:5      - '7600:7600'6      - '7300:7300'7    volumes:8      - ./config.toml:/config.toml9    command: ['--config', '/config.toml']10    environment:11      - RUST_LOG=indexer_service=info12    restart: unless-stopped1314  tap-agent:15    image: ghcr.io/graphprotocol/indexer-tap-agent:latest16    ports:17      - '7301:7300'18    volumes:19      - ./config.toml:/config.toml20    command: ['--config', '/config.toml']21    environment:22      - RUST_LOG=indexer_tap_agent=info23    restart: unless-stopped

Kubernetes

For Kubernetes deployments, see the Graph Launchpad charts⁠ which include:

• Helm charts for both services
• ConfigMap templates for configuration
• Service and ingress definitions
• Prometheus ServiceMonitor resources

Best Practices

1. Database Performance:

   • Ensure your PostgreSQL instance has adequate resources
   • Monitor query performance through metrics
   • Regular maintenance of receipt tables

2. High Availability:

   • Run multiple indexer-service-rs instances behind a load balancer
   • Keep indexer-tap-agent as a single instance with proper monitoring
   • Use database connection pooling

3. Security:

   • Store operator mnemonic securely (use environment variables)
   • Restrict database access to indexer components only
   • Monitor for unusual receipt patterns

4. Capacity Planning:

   • Monitor receipt growth rate
   • Plan database storage accordingly
   • Set appropriate max_amount_willing_to_lose_grt based on query volume

Additional Resources

• indexer-rs Repository⁠
• TAP Core Documentation⁠
• The Graph Discord - Get help in the #indexers channel
• Support⁠ - Report issues or get help