Cookbook

Migrating an Existing Subgraph to The Graph Network

IntroductionLink to this section

This is a guide on how to migrate your subgraph from the Hosted Service to The Graph's decentralized network. Migration to The Graph Network has been successful for projects like Opyn, UMA, mStable, Audius, PoolTogether, Livepeer, RAI, Enzyme, DODO, Pickle, and BadgerDAO all of which are relying on data served by Indexers on the network. There are now over 200 subgraphs live on The Graph Network, generating query fees and actively indexing web3 data.

The process of migration is quick and your subgraphs will forever benefit from the reliability and performance that you can only get on The Graph Network.

AssumptionsLink to this section

  • You have already deployed a subgraph on the hosted service.
  • The subgraph is indexing Ethereum mainnet.
  • The subgraph does not have IPFS or full-text search dependencies (these are not fully supported on the decentralized network yet).

Migrating an Existing Subgraph to The Graph NetworkLink to this section

  1. Get the latest version of the graph-cli installed:
npm install -g @graphprotocol/graph-cli
yarn global add @graphprotocol/graph-cli

Also make sure your apiVersion in subgraph.yaml is 0.0.5 or greater.

  1. Inside the subgraph's main project repository, authenticate the subgraph to deploy and build on the studio:
graph auth --studio <DEPLOY_KEY>
  1. Generate files and build the subgraph:
graph codegen && graph build

If your subgraph has build errors, refer to the AssemblyScript Migration Guide.

  1. Sign into Subgraph Studio with your wallet and deploy the subgraph. You can find your <SUBGRAPH_SLUG> in the Studio UI, which is based on the name of your subgraph.
graph deploy --studio <SUBGRAPH_SLUG>
  1. Test queries on the Studio's playground. Here are some examples for the Sushi - Mainnet Exchange Subgraph:
{
users(first: 5) {
id
liquidityPositions {
id
}
}
bundles(first: 5) {
id
ethPrice
}
}
  1. At this point, your subgraph is now deployed on Subgraph Studio, but not yet published to the decentralized network. You can now test the subgraph to make sure it is working as intended using the temporary query URL as seen on top of the right column above. As this name already suggests, this is a temporary URL and should not be used in production.
  • Remember that publishing is an on-chain action and will require gas to be paid for in Ethereum - see an example transaction here. Prices are roughly around 0.0425 ETH at 100 gwei.
  • Any time you need to upgrade your subgraph, you will be charged an upgrade fee. Remember, upgrading is just publishing another version of your existing subgraph on-chain. Because this incurs a cost, it is highly recommended to deploy and test your subgraph on Goerli before deploying to mainnet. It can, in some cases, also require some GRT if there is no signal on that subgraph. In the case there is signal/curation on that subgraph version (using auto-migrate), the taxes will be split.
  1. Publish the subgraph on The Graph's decentralized network by hitting the "Publish" button.

And that's it! After you are done publishing, you'll be able to view your subgraphs live on the decentralized network via The Graph Explorer.

Feel free to leverage the #Curators channel on Discord to let Curators know that your subgraph is ready to be signaled. It would also be helpful if you share your expected query volume with them. Therefore, they can estimate how much GRT they should signal on your subgraph.

Create an API keyLink to this section

You can generate an API key in Subgraph Studio here.

API key creation page

At the end of each week, an invoice will be generated based on the query fees that have been incurred during this period. This invoice will be paid automatically using the GRT available in your balance. Your balance will be updated after the cost of your query fees are withdrawn. Query fees are paid in GRT via the Polygon network. You will need to add GRT to the Polygon billing contract to enable your API key via the following steps:

  • Purchase GRT on an exchange of your choice.
  • Send the GRT to your wallet.
  • On the Billing page in Studio, click on Add GRT.

Add GRT in billing

  • Follow the steps to bridge the GRT to Polygon.
  • Once bridged, add the GRT to your billing balance.

Billing pane

Note: This process is slightly different if you are using a multisig since you can not bridge GRT to Polygon with a multisig. You would need to send the GRT to an EOA (externally owned address) and bridge it from there to Polygon. Once on Polygon, you can load the multisig address. We have created a special tool to help you through these steps.

It is recommended that you secure the API by limiting its usage in two ways:

  1. Authorized Subgraphs
  2. Authorized Domain

Subgraph lockdown page

Querying your subgraph on the decentralized networkLink to this section

Now you can check the indexing status of the Indexers on the network in Graph Explorer (example here). The green line at the top indicates that at the time of posting 8 Indexers successfully indexed that subgraph. Also in the Indexer tab you can see which Indexers picked up your subgraph.

Rocket Pool subgraph

As soon as the first Indexer has fully indexed your subgraph you can start to query the subgraph on the decentralized network. In order to retrieve the query URL for your subgraph, you can copy/paste it by clicking on the symbol next to the query URL. You will see something like this:

https://gateway.thegraph.com/api/[api-key]/subgraphs/id/S9ihna8D733WTEShJ1KctSTCvY1VJ7gdVwhUujq4Ejo

Important: Make sure to replace [api-key] with an actual API key generated in the section above.

You can now use that Query URL in your dapp to send your GraphQL requests to.

Congratulations! You are now a pioneer of decentralization!

Note: Due to the distributed nature of the network it might be the case that different Indexers have indexed up to different blocks. In order to only receive fresh data you can specify the minimum block an Indexer has to have indexed in order to serve your query with the block: { number_gte: $minBlock } field argument as shown in the example below:

{
stakers(block: { number_gte: 14486109 }) {
id
}
}

More information about the nature of the network and how to handle re-orgs are described in the documentation article Distributed Systems.

Upgrading a Subgraph on the NetworkLink to this section

If you would like to upgrade an existing subgraph on the network, you can do this by deploying a new version of your subgraph to the Subgraph Studio using the Graph CLI.

  1. Make changes to your current subgraph. A good idea is to test small fixes on the Subgraph Studio by publishing to Goerli.
  2. Deploy the following and specify the new version in the command (eg. v0.0.1, v0.0.2, etc):
graph deploy --studio <SUBGRAPH_SLUG>
  1. Test the new version in the Subgraph Studio by querying in the playground
  2. Publish the new version on The Graph Network. Remember that this requires gas (as described in the section above).

Owner Upgrade Fee: Deep DiveLink to this section

An upgrade requires GRT to be migrated from the old version of the subgraph to the new version. This means that for every upgrade, a new bonding curve will be created (more on bonding curves here).

The new bonding curve charges the 2.5% curation tax on all GRT being migrated to the new version. The owner must pay 50% of this or 1.25%. The other 1.25% is absorbed by all the curators as a fee. This incentive design is in place to prevent an owner of a subgraph from being able to drain all their curator's funds with recursive upgrade calls. If there is no curation activity, you will have to pay a minimum of 100 GRT in order to signal your own subgraph.

Let's make an example, this is only the case if your subgraph is being actively curated on:

  • 100,000 GRT is signaled using auto-migrate on v1 of a subgraph
  • Owner upgrades to v2. 100,000 GRT is migrated to a new bonding curve, where 97,500 GRT get put into the new curve and 2,500 GRT is burned
  • The owner then has 1250 GRT burned to pay for half the fee. The owner must have this in their wallet before the upgrade, otherwise, the upgrade will not succeed. This happens in the same transaction as the upgrade.

While this mechanism is currently live on the network, the community is currently discussing ways to reduce the cost of upgrades for subgraph developers.

Maintaining a Stable Version of a SubgraphLink to this section

If you're making a lot of changes to your subgraph, it is not a good idea to continually upgrade it and front the upgrade costs. Maintaining a stable and consistent version of your subgraph is critical, not only from the cost perspective but also so that Indexers can feel confident in their syncing times. Indexers should be flagged when you plan for an upgrade so that Indexer syncing times do not get impacted. Feel free to leverage the #Indexers channel on Discord to let Indexers know when you're versioning your subgraphs.

Subgraphs are open APIs that external developers are leveraging. Open APIs need to follow strict standards so that they do not break external developers' applications. In The Graph Network, a subgraph developer must consider Indexers and how long it takes them to sync a new subgraph as well as other developers who are using their subgraphs.

Updating the Metadata of a SubgraphLink to this section

You can update the metadata of your subgraphs without having to publish a new version. The metadata includes the subgraph name, image, description, website URL, source code URL, and categories. Developers can do this by updating their subgraph details in the Subgraph Studio where you can edit all applicable fields.

Make sure Update Subgraph Details in Explorer is checked and click on Save. If this is checked, an on-chain transaction will be generated that updates subgraph details in the Explorer without having to publish a new version with a new deployment.

Best Practices for Deploying a Subgraph to The Graph NetworkLink to this section

  1. Leveraging an ENS name for Subgraph Development:
  1. The more filled out your profiles are, the better the chances for your subgraphs to be indexed and curated.

Deprecating a Subgraph on The Graph NetworkLink to this section

Follow the steps here to deprecate your subgraph and remove it from The Graph Network.

Querying a Subgraph + Billing on The Graph NetworkLink to this section

The Hosted Service was set up to allow developers to deploy their subgraphs without any restrictions.

In order for The Graph Network to truly be decentralized, query fees have to be paid as a core part of the protocol's incentives. For more information on subscribing to APIs and paying the query fees, check out billing documentation here.

Estimate Query Fees on the NetworkLink to this section

While this is not a live feature in the product UI, you can set your maximum budget per query by taking the amount you're willing to pay per month and dividing it by your expected query volume.

While you get to decide on your query budget, there is no guarantee that an Indexer will be willing to serve queries at that price. If a Gateway can match you to an Indexer willing to serve a query at, or lower than, the price you are willing to pay, you will pay the delta/difference of your budget and their price. As a consequence, a lower query price reduces the pool of Indexers available to you, which may affect the quality of service you receive. It's beneficial to have high query fees, as that may attract curation and big-name Indexers to your subgraph.

Remember that it's a dynamic and growing market, but how you interact with it is in your control. There is no maximum or minimum price specified in the protocol or the Gateways. For example, you can look at the price paid by a few of the dapps on the network (on a per-week basis), below. See the last column, which shows query fees in GRT. For example, Pickle Finance has 8 requests per second and paid 2.4 GRT for one week.

QueryFee

Additional ResourcesLink to this section

If you're still confused, fear not! Check out the following resources or watch our video guide on migrating subgraphs to the decentralized network below:

Edit page

Previous
Quick Start
Next
Quick and Easy Subgraph Debugging Using Forks