Quick and Easy Subgraph Debugging Using Forks

Reading time: 4 min

As with many systems processing large amounts of data, The Graph's Indexers (Graph Nodes) may take quite some time to sync-up your subgraph with the target blockchain. The discrepancy between quick changes with the purpose of debugging and long wait times needed for indexing is extremely counterproductive and we are well aware of that. This is why we are introducing subgraph forking, developed by LimeChain, and in this article I will show you how this feature can be used to substantially speed-up subgraph debugging!

Subgraph forking is the process of lazily fetching entities from another subgraph's store (usually a remote one).

In the context of debugging, subgraph forking allows you to debug your failed subgraph at block X without needing to wait to sync-up to block X.

When you deploy a subgraph to a remote Graph Node for indexing and it fails at block X, the good news is that the Graph Node will still serve GraphQL queries using its store, which is synced-up to block X. That's great! This means we can take advantage of this "up-to-date" store to fix the bugs arising when indexing block X.

In a nutshell, we are going to fork the failing subgraph from a remote Graph Node that is guaranteed to have the subgraph indexed up to block X in order to provide the locally deployed subgraph being debugged at block X an up-to-date view of the indexing state.

To stay focused on subgraph debugging, let's keep things simple and run along with the example-subgraph indexing the Ethereum Gravity smart contract.

Here are the handlers defined for indexing Gravatars, with no bugs whatsoever:

export function handleNewGravatar(event: NewGravatar): void {
  let gravatar = new Gravatar(event.params.id.toHex().toString())
  gravatar.owner = event.params.owner
  gravatar.displayName = event.params.displayName
  gravatar.imageUrl = event.params.imageUrl
  gravatar.save()
}

export function handleUpdatedGravatar(event: UpdatedGravatar): void {
  let gravatar = Gravatar.load(event.params.id.toI32().toString())
  if (gravatar == null) {
    log.critical('Gravatar not found!', [])
    return
  }
  gravatar.owner = event.params.owner
  gravatar.displayName = event.params.displayName
  gravatar.imageUrl = event.params.imageUrl
  gravatar.save()
}

Oops, how unfortunate, when I deploy my perfect looking subgraph to Subgraph Studio it fails with the "Gravatar not found!" error.

The usual way to attempt a fix is:

1. Make a change in the mappings source, which you believe will solve the issue (while I know it won't).
2. Re-deploy the subgraph to Subgraph Studio (or another remote Graph Node).
3. Wait for it to sync-up.
4. If it breaks again go back to 1, otherwise: Hooray!

It is indeed pretty familiar to an ordinary debug process, but there is one step that horribly slows down the process: 3. Wait for it to sync-up.

Using subgraph forking we can essentially eliminate this step. Here is how it looks:

0. Spin-up a local Graph Node with the appropriate fork-base set.
1. Make a change in the mappings source, which you believe will solve the issue.
2. Deploy to the local Graph Node, forking the failing subgraph and starting from the problematic block.
3. If it breaks again, go back to 1, otherwise: Hooray!

Now, you may have 2 questions:

1. fork-base what???
2. Forking who?!

And I answer:

1. fork-base is the "base" URL, such that when the subgraph id is appended the resulting URL (<fork-base>/<subgraph-id>) is a valid GraphQL endpoint for the subgraph's store.
2. Forking is easy, no need to sweat:

$ graph deploy <subgraph-name> --debug-fork <subgraph-id> --ipfs http://localhost:5001 --node http://localhost:8020

Also, don't forget to set the dataSources.source.startBlock field in the subgraph manifest to the number of the problematic block, so you can skip indexing unnecessary blocks and take advantage of the fork!

So, here is what I do:

1. I spin-up a local Graph Node (here is how to do it) with the fork-base option set to: https://api.thegraph.com/subgraphs/id/, since I will fork a subgraph, the buggy one I deployed earlier, from Subgraph Studio.

$ cargo run -p graph-node --release -- \
    --postgres-url postgresql://USERNAME[:PASSWORD]@localhost:5432/graph-node \
    --ethereum-rpc NETWORK_NAME:[CAPABILITIES]:URL \
    --ipfs 127.0.0.1:5001
    --fork-base https://api.thegraph.com/subgraphs/id/

2. After careful inspection I notice that there is a mismatch in the id representations used when indexing Gravatars in my two handlers. While handleNewGravatar converts it to a hex (event.params.id.toHex()), handleUpdatedGravatar uses an int32 (event.params.id.toI32()) which causes the handleUpdatedGravatar to panic with "Gravatar not found!". I make them both convert the id to a hex.
3. After I made the changes I deploy my subgraph to the local Graph Node, forking the failing subgraph and setting dataSources.source.startBlock to 6190343 in subgraph.yaml:

$ graph deploy gravity --debug-fork QmNp169tKvomnH3cPXTfGg4ZEhAHA6kEq5oy1XDqAxqHmW --ipfs http://localhost:5001 --node http://localhost:8020

4. I inspect the logs produced by the local Graph Node and, Hooray!, everything seems to be working.
5. I deploy my now bug-free subgraph to a remote Graph Node and live happily ever after! (no potatoes tho)