基于Substreams的子图

Reading time: 7 min

Substreams is a new framework for processing blockchain data, developed by StreamingFast for The Graph Network. A substreams modules can output entity changes, which are compatible with Subgraph entities. A subgraph can use such a Substreams module as a data source, bringing the indexing speed and additional data of Substreams to subgraph developers.

This cookbook requires yarn, the dependencies necessary for local Substreams development, and the latest version of Graph CLI (>=0.52.0):

npm install -g @graphprotocol/graph-cli

graph init --from-example substreams-powered-subgraph

A Substreams package is composed of types (defined as Protocol Buffers), modules (written in Rust), and a substreams.yaml file which references the types, and specifies how modules are triggered. Visit the Substreams documentation to learn more about Substreams development, and check out awesome-substreams and the Substreams cookbook for more examples.

该Substreams包可以检测以太坊主网上的合约部署，并跟踪所有新部署合约的创建块和时间戳。为此，在/proto/example.proto中有一个专门的Contract类型（了解更多关于定义Protocol Buffers的信息learn more about defining Protocol Buffers）：

syntax = "proto3";

package example;

message Contracts {
  repeated Contract contracts = 1;
}

message Contract {
    string address = 1;
    uint64 blockNumber = 2;
    string timestamp = 3;
    uint64 ordinal = 4;
}

Substreams包的核心逻辑是lib.rs中的map_contract模块，它处理每个区块，过滤未回滚的Create调用，并返回Contracts：

#[substreams::handlers::map]
fn map_contract(block: eth::v2::Block) -> Result<Contracts, substreams::errors::Error> {
    let contracts = block
        .transactions()
        .flat_map(|tx| {
            tx.calls
                .iter()
                .filter(|call| !call.state_reverted)
                .filter(|call| call.call_type == eth::v2::CallType::Create as i32)
                .map(|call| Contract {
                    address: format!("0x{}", Hex(&call.address)),
                    block_number: block.number,
                    timestamp: block.timestamp_seconds().to_string(),
                    ordinal: tx.begin_ordinal,
                })
        })
        .collect();
    Ok(Contracts { contracts })
}

只要Substreams包有一个模块输出与子图兼容的实体变更，就可以被子图使用。示例Substreams包中还有一个额外的graph_out模块在lib.rs中，它返回一个substreams_entity_change::pb::entity::EntityChanges输出，Graph Node可以处理该输出。

#[substreams::handlers::map]
pub fn graph_out(contracts: Contracts) -> Result<EntityChanges, substreams::errors::Error> {
    // hash map of name to a table
    let mut tables = Tables::new();

    for contract in contracts.contracts.into_iter() {
        tables
            .create_row("Contract", contract.address)
            .set("timestamp", contract.timestamp)
            .set("blockNumber", contract.block_number);
    }

    Ok(tables.to_entity_changes())
}

这些类型和模块在substreams.yaml中汇集在一起：

specVersion: v0.1.0
package:
  name: 'substreams_test' # the name to be used in the .spkg
  version: v1.0.1 # the version to use when creating the .spkg

imports: # dependencies
  entity: https://github.com/streamingfast/substreams-entity-change/releases/download/v0.2.1/substreams-entity-change-v0.2.1.spkg

protobuf: # specifies custom types for use by Substreams modules
  files:
    - example.proto
  importPaths:
    - ./proto

binaries:
  default:
    type: wasm/rust-v1
    file: ./target/wasm32-unknown-unknown/release/substreams.wasm

modules: # specify modules with their inputs and outputs.
  - name: map_contract
    kind: map
    inputs:
      - source: sf.ethereum.type.v2.Block
    output:
      type: proto:test.Contracts

  - name: graph_out
    kind: map
    inputs:
      - map: map_contract
    output:
      type: proto:substreams.entity.v1.EntityChanges # this type can be consumed by Graph Node

您可以通过运行substreams graph来检查从区块到map_contract到graph_out的整体"流程"：

为了准备这个Substreams包供子图使用，您必须运行以下命令：

yarn substreams:protogen # 生成/src/pb中的类型
yarn substreams:build # 构建Substreams
yarn substreams:package # 将Substreams打包成.spkg文件

# 或者，yarn substreams:prepare会调用上述所有命令

这将根据substreams.yaml中的包名称和版本生成一个spkg文件。spkg文件包含了Graph Node需要获取此Substreams包的所有信息。

Substreams-powered subgraphs引入了一种名为"substreams"的新型数据源。这样的子图只能包含一个数据源。

基于Substreams的子图引入了一种新的数据源类型，"substreams"。这样的子图只能有一个数据源。该数据源必须指定Substreams网络、Substreams包（spkg）的相对文件位置，以及该Substreams包中生成子图兼容实体变更的模块（在本例中为map_entity_changes）。映射是指定的，但只是标识了映射类型（"substreams/graph-entities"）和apiVersion。

specVersion: 0.0.4
description: Ethereum Contract Tracking Subgraph (powered by Substreams)
repository: https://github.com/graphprotocol/graph-tooling
schema:
  file: schema.graphql
dataSources:
  - kind: substreams
    name: substream_test
    network: mainnet
    source:
      package:
        moduleName: graph_out
        file: substreams-test-v1.0.1.spkg
    mapping:
      kind: substreams/graph-entities
      apiVersion: 0.0.5

Subgraph.yaml还引用了一个模式文件。对于这个文件的要求没有变化，但是指定的实体必须与subgraph.yaml中引用的Substreams模块生成的实体变更兼容。

type Contract @entity {
  id: ID!

  "The timestamp when the contract was deployed"
  timestamp: String!

  "The block number of the contract deployment"
  blockNumber: BigInt!
}

有了以上内容，子图开发者可以使用Graph CLI部署这个基于Substreams的子图。

yarn install # 安装graph-cli
yarn subgraph:build # 构建子图
yarn subgraph:deploy # 部署子图

就是这样！您已经构建并部署了一个基于Substreams的子图。

为了服务基于Substreams的子图，Graph Node必须配置与相关网络的Substreams提供程序，以及用于跟踪链头的Firehose或RPC。这些提供程序可以通过config.toml文件进行配置：

[chains.mainnet]
shard = "main"
protocol = "ethereum"
provider = [
  { label = "substreams-provider-mainnet",
    details = { type = "substreams",
    url = "https://mainnet-substreams-url.grpc.substreams.io/",
    token = "exampletokenhere" }},
  { label = "firehose-provider-mainnet",
    details = { type = "firehose",
    url = "https://mainnet-firehose-url.grpc.firehose.io/",
    token = "exampletokenhere" }},
]