Three New Tools to Supercharge the Indexer Tech Stack

Get ready to supercharge your productivity, efficiency, and cost-effectiveness as an Indexer on The Graph! After becoming a core developer of The Graph last summer, the GraphOps team has been hard at work developing new tools to make it easier for Indexers to set up performant nodes on The Graph Network. Today, the GraphOps team is thrilled to announce the launch of Graphcast Network, POI Radio, and Graphcast SDK. These powerful tools are designed to give Indexers and developers a competitive edge by making it easier to share crucial data, and we can't wait for you to explore them.

What makes these tools so game-changing? Graphcast Network enables real-time and decentralised peer-to-peer communication in The Graph ecosystem. Graphcast SDK empowers developers to create innovative gossip-powered applications for Indexers using Graphcast Network. The POI Radio makes it possible for Indexers to cross-check subgraph data integrity with ease, giving Indexers peace of mind and an early warning system in case of a POI discrepancy. This trifecta of tools helps Indexers level up their indexing stack in a collaborative, positive-sum way.

To understand how these tools are connected, we’ve created the diagram below (click to expand the diagram):

We're calling on all Indexers and developers to test out these cutting-edge tools and provide valuable feedback to help the GraphOps team continue to improve. In this post, we will introduce each tool in greater detail, how it works, who it benefits, and how to get started. Together, we can create a more resilient and stronger network.

Introducing Graphcast Network

Do you have any information or insights that you wish to share with your fellow Indexers, but find the associated gas fees to be excessive?

Graphcast Network solves this problem by providing a cost-effective solution to the high gas fees on the blockchain*. It’s a new peer-to-peer communication tool that facilitates real-time information exchange among Indexers in a decentralised and distributed manner. Graphcast provides an optional off-chain infrastructure layer that Indexers can opt into, allowing them to exchange messages at a negligible cost.

Graphcast is expected to bring several advantages to Indexers, as it opens up new possibilities for inexpensive collaboration. One of the main benefits is the ability to create Radios, which are applications powered by gossip protocols that Indexers can use to share specific data.

Here are some examples of Radios that could be developed:

• Conducting auctions and coordination for warp syncing** subgraphs, substreams, and Firehose data from other Indexers.
• Self-reporting on active query analytics, including subgraph request volumes, fee volumes, etc.
• Self-reporting on indexing analytics, including subgraph indexing time, handler gas costs, indexing errors encountered, etc.
• Self-reporting on stack information including graph-node version, Postgres version, Ethereum client version, etc.
• AutoAgora cross-Indexer signals/negotiations for improved automated query pricing.

The GraphOps team has taken the initiative to jumpstart the development of Radios on the Graphcast Network by creating the POI Radio, enabling real-time cross-checking of subgraph data integrity.

• Please be aware that the POI Radio, Graphcast SDK, and Graphcast Network are all currently in beta release. While we have made every effort to ensure their stability and functionality, we recommend that Indexers and developers proceed with caution when using these tools.
• *Warp syncing is the process of sharing a subgraph database that's already been processed, enabling another Indexer to avoid processing that subgraph from scratch.As with any beta release, there may be undiscovered bugs or issues that could impact the performance or reliability of the tools. We encourage users to thoroughly test these tools in a safe and controlled environment, such as The Graph Network’s testnet, before integrating them into any critical systems or workflows. If you discover any bugs, issues, or have suggestions for improvements, please do not hesitate to let us know via the #Graphcast channel in The Graph Discord server, or join the next Indexer Office Hours. Your feedback is essential for us to develop more reliable and robust tools that cater to the needs of The Graph community.

POI Radio

POI Radio was created to help protect Indexers from making costly mistakes while serving queries on The Graph.

In order to receive indexing rewards on The Graph Network, Indexers are required to regularly post a valid Proof of Indexing (POI) on-chain, verifying that they have indexed accurate data. However, posting an incorrect POI can result in liability and being slashed. The POI Radio is an optional component of the Indexer tech stack that leverages Graphcast Network to cross-check subgraph data integrity with other participating Indexers.

By continuously cross-checking POIs (and alerting Indexers immediately in case of a discrepancy), the POI Radio can help give Indexers additional peace of mind that their POIs are valid and helps them take corrective action promptly, avoiding indexing disputes.

Exploring the Inner Workings of the POI Radio

Fetching active allocations

The POI Radio is designed to monitor the active allocations of the Indexer that operates it. It regularly checks for new blocks on the relevant networks, and creates Graphcast topics for each allocation identified by the IPFS hash of the subgraph deployment. The relevant networks are those related to the subgraphs with active allocations.

The POI Radio fetches new active allocations every two minutes to ensure that it has the most up-to-date information. It updates the chainheads for these networks with data from Graph Node, and ensures that it always uses the latest chainhead when processing messages.

Gathering and comparing normalised POIs

The POI Radio fetches the normalised POI (nPOI) for each deployment at a defined interval that differs for each network. It then stores these nPOIs in an in-memory map, which are referred to as local POIs since they are generated by the POI Radio. As other Indexers running the Radio start doing the same, messages containing POIs start propagating through the network. The Radio adds the received POIs to another in-memory map, referred to as remote POIs.

Each message includes:

• Nonce
• Block number
• Signature
• Network

The Radio operator verifies the sender's on-chain identity and amount of tokens staked before saving an entry to the map.

At another interval, the Radio compares the local and remote POIs for each subgraph at each block to determine consensus. The top POI is selected based on the combined stake of all Indexers that attested to it, not just the highest staking Indexer. After a successful comparison, the checked attestations are removed from the store. The default interval for this comparison is 30 seconds, but the operator can set it differently.

We invite all Indexers to take an active role in the ongoing development of the POI Radio by running it and providing feedback. Your insights and experiences are invaluable for refining the tool, ensuring it meets the needs of the community.To get started, head over to the documentation to learn how to set up the POI Radio on your Indexer tech stack and monitor its performance as it cross-checks subgraph data integrity with other participating Indexers. Remember, the POI Radio is designed to give you peace of mind and help you avoid costly indexing disputes.  Once you've had a chance to test the POI Radio, we encourage you to share your feedback, report any bugs, or suggest improvements in the#Graphcast Discord channel within The Graph's server. Your input will help us continue to improve the Radio and better serve the needs of the entire community. Together, we can strengthen The Graph ecosystem and unlock its full potential.

Are you interested in developing messaging applications that can exchange data with other Indexers in real-time? Do you have innovative ideas on what data could be shared to improve communication within The Graph Network? Then you can build a Radio on Graphcast Network using our newly released Software Development Kit (SDK). With this SDK, you can create Radios similar to the POI Radio discussed earlier and contribute to the greater efficiency of The Graph Network.

Still working through your idea? We recommend you share your thoughts with other Indexers in The Graph Forum. After receiving feedback on your idea, you may proceed to submit a proposal for a grant from The Graph Foundation.

Graphcast Software Development Kit (SDK)

Graphcast SDK is a powerful tool that enables developers to build gossip-powered applications, such as Radios, that Indexers can run for a wide variety of indexing-related tasks. With Graphcast SDK, developers can focus on building innovative applications that help Indexers, without worrying about the underlying infrastructure. We encourage all developers to explore the potential of Graphcast SDK and create their own Radios to improve the functionality and efficiency of The Graph ecosystem.

To gain practical experience in creating a Radio with Graphcast SDK,go to the documentation page and start building a simple sample Radio application called Ping Pong. When one participant sends a Ping message, all other participants listening on the ping pong topic will respond with a Pong message. It's a straightforward process that will give you a good understanding of what it's like to create a Radio on top of the Graphcast Network using Graphcast SDK.

Once you've successfully created the ping pong Radio application, the possibilities for the Radios you can build are limitless and depend only on your imagination. So, start building and exploring the full potential of the Graphcast network. Learn how to get started!

New Tools, New Possibilities

We’ve launched Graphcast Network, POI Radio, and Graphcast SDK to enhance efficiency, cost-effectiveness, and productivity within The Graph ecosystem. These tools allow Indexers and developers to communicate and exchange information in real-time, in a decentralised and distributed manner. Graphcast Network provides a cost-effective solution to the high gas fees on the blockchain and opens up new possibilities for inexpensive collaboration. Indexers can create Radios, powered by gossip protocols, to accomplish specific tasks. The POI Radio ensures Indexers' peace of mind by cross-checking subgraph data integrity and avoiding any liability of a dispute or being slashed by the protocol. Indexers can also save time and effort by freeing themselves up to focus on other important tasks.

Get Started:

• Learn more about Graphcast in The Graph Docs
• Indexers: Please try the POI Radio and share feedback with us via the #Graphcast Discord channel in The Graph’s server, and during Indexer Office Hours, to help us improve the Radio.
• Developers: Please try building a Radio on Graphcast Network and share your experience using our SDK. Together we can improve the SDK and support you in building Radio(s).

Together we can continue to build a stronger and more resilient network. Indexers on The Graph Network are exemplary of the positive-sum, collaborative spirit of web3. By using tools that enhance collaboration, Indexers can increase the efficacy of their indexing operation, creating cascading positive effects for other network participants like developers, data consumers, Curators, and Delegators. Stay tuned for more contributions by GraphOps via the official GraphOps Twitter, and share feedback on Indexer tooling in The Graph Forum or The Graph Discord!

As we wrap up this exciting announcement, we cannot express enough gratitude towards the phenomenal Waku team for their unwavering commitment and collaboration throughout the entire process. The Waku team has been exceptional in their engagement with us, embracing our feedback and ensuring that our experience with the bindings is seamless. They've been instrumental in customising Graphcast to cater to our unique needs at GraphOps, truly understanding the value of synergistic development. We are incredibly thankful for their round-the-clock support, willingness to share insights, and inspiring discussions about the future of both projects. This partnership has been an extraordinary journey, and we eagerly look forward to our continued collaboration as we unlock new possibilities together. We, also, would like to extend our heartfelt thanks to IndexerDAO, StakeSquid, Wavefive, Ellipfra, Nodeify, Mind Heart Soul, Stake Machine, and Data Nexus for their invaluable help in testing the POI Radio before our launch. Their early adoption and constructive feedback have been instrumental in shaping and refining the POI Radio, Graphcast Network, and Graphcast SDK, allowing us to build solutions that truly cater to the needs of the community.

About GraphOps

GraphOps is a blockchain data infrastructure company. They extract, transform and serve blockchain data. They’re driven by the mission to build an uncensorable, equitable and self-sovereign web. Public data is at the heart of that mission, and they thrive on building open data standards and protocols to empower the next great wave of institutions and applications for the world. Collaborating with other exceptional core development teams, GraphOps are building on the world’s leading protocol for indexing and querying data on public networks: The Graph. Follow GraphOps on Twitter to stay up-to-date with all the work that they do.

About The Graph

The Graph is the source of data and information for the decentralized internet. As the original decentralized data marketplace that introduced and standardized subgraphs, The Graph has become web3’s method of indexing and accessing blockchain data. Since its launch in 2018, tens of thousands of developers have built subgraphs for dapps across 40+ blockchains - including  Ethereum, Arbitrum, Optimism, Base, Polygon, Celo, Fantom, Gnosis, and Avalanche.

As demand for data in web3 continues to grow, The Graph enters a New Era with a more expansive vision including new data services and query languages, ensuring the decentralized protocol can serve any use case - now and into the future.

Discover more about how The Graph is shaping the future of decentralized physical infrastructure networks (DePIN) and stay connected with the community. Follow The Graph on X, LinkedIn, Instagram, Facebook, Reddit, and Medium. Join the community on The Graph’s Telegram, join technical discussions on The Graph’s Discord.

The Graph Foundation oversees The Graph Network. The Graph Foundation is overseen by the Technical Council. Edge & Node, StreamingFast, Semiotic Labs, The Guild, Messari, GraphOps, Pinax and Geo are eight of the many organizations within The Graph ecosystem.