Skip to main content

The Graph Migration

SubQuery TeamAbout 11 min

The Graph Migration

Hosting existing Subgraphs during migration

SubQuery provides a superior indexing SDK to Subgraphs and this document outlines the migration process. However SubQuery also provides a one-step migration for hosting your Subgraphs on The Graph’s hosted service / decentralised network, or other Subgraph hosting service. This is a drop-in replacement with the following benefits:

  • An enterprise level hosting platform with targeted 99.9% uptime
  • Blue/green hot-swap deployments of subgraphs, allowing for seamless updates on your frontend
  • Project alert notification of outages and indexing issues
  • A team who monitors your Subgraphs 24/7, with potential for service level agreements real time support with your team

Find out more about Subgraph hosting here.

Graph Migration

SubQuery provides a superior developer experience to The Graph, while maintaining a similar development workflow that makes migration quick and easy. Migrating to SubQuery brings the following benefits:

  • Performance improvements - SubQuery is up to 3.9x faster for common projects over The Graph (e.g the standard Ethereum POAP subgraph) thanks to multi-threading and a store cache layer
  • More flexibility - SubQuery's mapping functions are written in TypeScript, and so you can import additional libraries, make external API calls, or do anything you want in order to retrieve and process your data.
  • Wider chain support - SubQuery supports all EVM networks, as well as many other non-EVM networks, like Cosmos, Polkadot, Algorand, NEAR, and more
  • Lightweight - SubQuery is designed to connect to external public RPC endpoints, you don't need to run an archive node locally when developing
  • Multi-chain indexing support - SubQuery allows you to index data from across different layer-1 networks into the same database, this allows you to query a single endpoint to get data for all supported networks. Read more.
  • More control - A large library of command line parameters to allow you to run, monitor, and optimise your locally hosted project
  • Managed Service hosting - We have no plans to sunset our Managed Service, which provides enterprise-level infrastructure hosting and handles over hundreds of millions of requests each day
  • A decentralised network supporting all chains - Our decentralised network supports all chains that SubQuery support, there is no second-class chain support in the SubQuery ecosystem
  • The same query API - We support a Subgraph compatible query service, providing the same GraphQL API that you are currently using.

Competitor Comparison

The migration from a SubGraph to a SubQuery Project is (by design) easy and quick. It may take you an hour or two to complete the migration, depending on the complexity of your SubGraph.

Tip

Want Support During Migration?

Reach out to our team at professionalservices@subquery.network and get a professional service to manage the migration for you.

Migration Overview

SubQuery makes it easy to migrate your existing SubGraph to SubQuery in the shortest time possible, we even have an automated tool support you with migrating most of your project's boilerplate code. At a high level:

  • Both SubGraph and SubQuery use the same schema.graphql file to define schema entities. In addition, both have similar sets of supported scalars and entity relationships (SubQuery adds support for JSON types though).
  • The manifest file shows the most differences but you can easily overcome these differences once you understand them.
  • In addition, Mapping files are also quite similar with an intentionally equivalent set of commands, which are used to access the Graph Node store and the SubQuery Project store.

This is the recommended process that we use at SubQuery whenever we migrate projects from a SubGraph to SubQuery.

Firstly, use the SubQuery CLI tool to migrate an existing SubGraph by using the subql migrate command. This will:

  • Initialise a new SubQuery project in your chosen directory for the matching target chain
  • Copy over basic information like project name and other metadata
  • Enable ts strict mode to assist you in identifying potential issues when migrating your project.
  • Migrate the schema.graphql to be compatible with SubQuery, there is a chance you will need to edit this after migration to fine-tune the migrated code.
  • Copy over the existing project.ts mapping to save you time (you will need to edit this)

An example of using this tool for the Graph Gravatar project is as follows

  • -f should point to a local project, or a subgraph on GitHub (if so include /tree/<branchname>)
  • -d is optional and used when the GitHub project is in a subdirectory
  • -o is the output directory
# Github
subql migrate -f  https://github.com/graphprotocol/graph-tooling/tree/main -d examples/ethereum-gravatar -o ~/subquery/gravatar-starter

# Local file
subql migrate -f ~/path/to/subgraph -o ~/path/to/output

Once this is done, follow along and complete the remaining steps:

  1. Review and edit the schema.graphql, it will remove any unsupported features and modify relations to work with SubQuery. You may need to change the field parameter in the @derivedFrom directive. More info.
  2. Review and edit the copied data sources in the project.ts manifest, specifically the handlers (we suggest that you retain the same handler names). More info.
  3. Perform code generation using the yarn codegen, this will generate GraphQL entity types, and generate types from ABIs. More info.
  4. Copy over the mappings directory, and then go through one by one to migrate them across. The key differences:
    • Imports will need to be updated
    • Store operations are asynchronous, e.g. <entityName>.load(id) should be replaced by await <entityName>.get(id) and <entityName>.save() to await <entityName>.save() (note the await).
    • With strict mode, you must construct new entities with all the required properties. You may want to replace new <entityName>(id) with <entityName>.create({ ... })
    • More info.
  5. Either use the Subgraph compatible query service, or test and update your clients to follow the SubQuery native GraphQL API. There are some minor and you can take advantage of additional features. More info

Differences in the GraphQL Schema

Both SubGraphs and SubQuery projects use the same schema.graphql to define entities and include both similar scalar types as well as full text search.

Visit this full documentation for schema.graphql. You can copy this file from your SubGraph to your SubQuery project in most cases.

Notable differences include:

  • SubQuery does not have support for Bytes (use String instead), BigDecimal (use Float instead) and Timestamp (use Date instead).
  • SubQuery has the additional scalar types of Float, Date, and JSON (see JSON type).
  • SubQuery does relations slightly differently and the @derivedFrom directive is needed in more cases.
  • Comments are added to SubQuery Project GraphQL files using hashes (#).

Differences in the Manifest File

The manifest file contains the largest set of differences, but once you understand those they can be easily overcome. Most of these changes are due to the layout of this file, you can see the full documentation of this file here.

Notable differences include:

  • SubQuery has moved to by default offering a Typescript based manifest file for better type safety and feature discoverability. SubQuery does however allow you to define your manifest in YAML if you wish to do so.
  • SubQuery has a section in the manifest for the network:. This is where you define what network your SubQuery project indexes, and the RPC endpoints (non-pruned archive nodes) that it connects to in order to retrieve the data. Make sure to include the dictionary: endpoint in this section as it will speed up the indexing speed of your SubQuery project.
  • Both SubGraphs and SubQuery projects use the dataSources: section to list the mapping files.
  • Similarly, you can define the contract ABI information for the smart contract that you are indexing.
    • In SubQuery, this is under the options: property rather than source:.
    • In both, SubQuery and SubGraph, you import a custom ABI spec that is used by the processor to parse arguments. For SubGraphs, this is done within the mapping: section under abis:. For a SubQuery project, this is at the same level of options: under assets: and the key is the name of the ABI.
  • In a SubQuery project, you can document both block handlers, call handlers, and event handlers in the same mapping: object.
  • In a SubQuery project, you do not list all mapping entities in the project manifest.
  • Handlers and Filters - Each mapping function is defined slightly differently in a SubQuery project:
    • Instead of listing the blocks/events/calls as the key and then denoting the handler that processes it. In SubQuery, you define the handler as the key and then what follows is the description of how this handler is triggered.
    • In a SubQuery project, you can document both block handlers, call handlers, and event handlers in the same mapping: object, the kind: property notes what type we are using.
    • SubQuery supports advanced filtering on the handler. The format of the supported filter varies amongst block/events/calls/transactions, and between the different blockchain networks. You should refer to the documentation for a detailed description of each filter.
SubQuery (TS)
const project: EthereumProject = {
  specVersion: "1.0.0",
  version: "0.0.1",
  name: "subquery-example-gravatar",
  description:
    "This project can be used as a starting point for developing your new Ethereum SubQuery project, it indexes all Gravatars on Ethereum",
  repository: "https://github.com/subquery/ethereum-subql-starter",
  runner: {
    node: {
      name: "@subql/node-ethereum",
      version: ">=3.0.0",
    },
    query: {
      name: "@subql/query",
      version: "*",
    },
  },
  schema: {
    file: "./schema.graphql",
  },
  network: {
    chainId: "1",
    endpoint: ["https://eth.api.onfinality.io/public"],
    dictionary: "https://gx.api.subquery.network/sq/subquery/eth-dictionary",
  },
  dataSources: [
    {
      kind: EthereumDatasourceKind.Runtime,
      startBlock: 6175243,
      options: {
        abi: "gravity",
        address: "0x2E645469f354BB4F5c8a05B3b30A929361cf77eC",
      },
      assets: new Map([["gravity", { file: "./abis/Gravity.json" }]]),
      mapping: {
        file: "./dist/index.js",
        handlers: [
          {
            kind: EthereumHandlerKind.Event,
            handler: "handleNewGravatar",
            filter: {
              topics: ["NewGravatar(uint256,address,string,string)"],
            },
          },
          {
            kind: EthereumHandlerKind.Event,
            handler: "handleUpdatedGravatar",
            filter: {
              topics: ["UpdatedGravatar(uint256,address,string,string)"],
            },
          },
        ],
      },
    },
  ],
};
export default project;

Differences in Codegen

The codegen command is also intentionally similar between SubQuery and SubGraphs

All GraphQL entities will have generated entity classes that provide type-safe entity loading, read and write access to entity fields - see more about this process in the GraphQL Schema. All entities can be imported from the following directory:

import { Gravatar } from "../types";

For ABI's registered in the project.ts, similar type safe entities will be generated using npx typechain --target=ethers-v5 command, allowing you to bind these contracts to specific addresses in the mappings and call read-only contract methods against the block being processed. It will also generate a class for every contract event to provide easy access to event parameters, as well as the block and transaction the event originated from. All of these types are written to src/types directory. In the example Gravatar SubQuery project, you would import these types like so.

import {
  NewGravatarLog,
  UpdatedGravatarLog,
} from "../types/abi-interfaces/Gravity";

Differences in Mapping Functions

Mapping files are also quite identical to an intentionally equivalent set of commands, which are used to access the Graph Node store and the SubQuery Project store.

In SubQuery, all mapping handlers receive a typed parameter that depends on the chain handler that calls it. For example, an ethereum/LogHandler will receive a parameter of type EthereumLog and avalanche/LogHandler will receive a parameter of type AvalancheLog.

The functions are defined the same way. Moreover, entities can be instantiated, retrieved, saved, and deleted from the SubQuery store in a similar way as well. The main difference is that SubQuery store operations are asynchronous.

SubQuery
// ******* SubQuery *******
import {
  NewGravatarLog,
  UpdatedGravatarLog,
} from "../types/abi-interfaces/Gravity";
import { Gravatar } from "../types";
import assert from "assert";

export async function handleNewGravatar(log: NewGravatarLog): Promise<void> {
  const gravatar = Gravatar.create({
    id: log.args.id.toHexString()!,
    owner: log.args.owner,
    displayName: log.args.displayName,
    imageUrl: log.args.imageUrl,
    createdBlock: BigInt(log.blockNumber),
  });
  await gravatar.save();
}

export async function handleUpdatedGravatar(
  log: UpdatedGravatarLog
): Promise<void> {
  const id: string = log.args.id.toHexString()!;
  let gravatar = await Gravatar.get(id);
  if (!gravatar) {
    gravatar = Gravatar.create({
      id,
      createdBlock: BigInt(log.blockNumber),
      owner: "",
      displayName: "",
      imageUrl: "",
    });
  }
  gravatar.owner = log.args.owner;
  gravatar.displayName = log.args.displayName;
  gravatar.imageUrl = log.args.imageUrl;
  await gravatar.save();
}

Differences in Querying Contracts

We globally provide an api object that implements an Ethers.js Provider. This will allow querying contract state at the current block height being indexed. The easiest way to use the api is with Typechain, with this you can generate typescript interfaces that are compatible with this api that make it much easier to query your contracts.

You can then query contract state at the right block height. For example to query the token balance of a user at the current indexed block height (please note the two underscores in Erc20__factory):

// Create an instance of the contract, you can get the contract address from the Transaction or Log
// Note the two underscores __ in `Erc20__factory`
const erc20 = Erc20__factory.connect(contractAddress, api);

// Query the balance of an address
const balance = await erc20.balanceOf(address);

The above example assumes that the user has an ABI file named erc20.json, so that TypeChain generates ERC20__factory class for them. Check out this example to see how to generate factory code around your contract ABI using TypeChain

Differences in the GraphQL Query Interface

Query Service Variants

SubQuery provides two query service variants, a SubQuery native query service (@subql/query) and a version that is compatible with the standard query service used by Subgraphs (@subql/query-subgraph). We recommend the SubQuery native query service but for ease of migration you may want to use the version that is the same as the Subgraph query service.

There are minor differences between the native SubQuery GraphQL query service for SubQuery, and that of the Graph. Remember there is a version that is the same as the Subgraph query service.

Query format

Note the additional nesting of entity properties under the nodes syntax.

SubQuery
{
  exampleEntities {
    nodes {
      field1
      field2
    }
  }
}

Filters

Instead of where, SubQuery uses filter. The Graph also uses a list of suffixes on the end of the entity field for filtering, when using SubQuery, you will need to add an extra layer on the GraphQL to specify the operator for the filters.

SubQuery
{
  exampleEntities(filter: { field1: { equalTo: "<value>" } }) {
    nodes {
      field1
      field2
    }
  }
}

Sorting

Instead of using orderBy and orderDirection, SubQuery creates directional values for the orderBy property (e.g. CREATED_AT_ASC and CREATED_AT_DSC)

SubQuery
{
  exampleEntities(orderBy: "FIELD_1_ASC") {
    nodes {
      field1
      field2
    }
  }
}

Historical queries

There is no difference when querying historical data.

SubQuery
{
  exampleEntities(block: { number: 123 }) {
    nodes {
      field1
      field2
    }
  }
}

Offset and Skip

SubQuery uses the offset field to skip a number of elements, while Subgraphs use the skip field.

SubQuery
{
  exampleEntities(first: 1, offset: 10) {
    nodes {
      field1
      field2
    }
  }
}

Metadata

SubQuery does not support historical metadata querying. However deployments will still show the deployments with their heights and other key metrics

SubQuery
{
  _metadata {
    deployments
    indexerHealthy
  }
}

Other changes

  • SubQuery has a larger support for query pagination. You have the options of using first and offset, or cursors on edges.
  • SubQuery supports advanced aggregate functions to allow you to perform a calculation on a set of values during your query.
  • Full text search is implemented slightly differently, please review the full text query docs.

What's Next?

Now that you have a clear understanding of how to build a basic SubQuery project, what are the next steps of your journey?

Now, you can easily publish your project. SubQuery provides a free Managed Service where you can deploy your new project. You can deploy it to SubQuery Managed Service and query it using our Explorer. Read this complete guide on how to publish your new project to SubQuery Projects.

To dive deeper into the developer documentation, jump to the Build section and learn more about the three key files: the manifest file, the GraphQL schema, and the mappings file.

If you want to practice with more real examples, then head to our Courses section and learn important concepts with related exercises and lab workbooks. Get access to readily available and open-source projects, and get a hands-on experience with SubQuery projects.

In the end, if you want to explore more ways to run and publish your project, refer to Run & Publish section. Get complete information about all the ways to run your SubQuery project, along with advanced GraphQL aggregation and subscription features.

Summary

As you can clearly see, it's only a small amount of work to migrate your SubGraphs to SubQuery. Once completed, you can take advantage of SubQuery's absolute performance. Now, start iterating and delivering faster with quicker sync times and indexing optimisations to ensure that you always have fresh data.

Moreover, SubQuery is multi-chain by design. Hence, the same project can be easily extended across chains and the SubQuery Network supports them all. Indexing across different chains is incredibly easy with SubQuery's unified design.

Finally, be assured that we're right behind you. Thousands of supporters and builders on our Discord server can provide you with the technical support that you need. Additionally, please reach out via start@subquery.network, we'd love to chat, hear what you're building, and work with you to make your migration easier.