Publish to SubQuery Managed Services

Benefits of hosting your project with SubQuery's Managed Service

Subgraphs are now supported in SubQuery's Managed Service

Run your projects side by side together with SubQuery's Subgraph hosting. One hosting provider for all your data indexing needs. Read more about how to quickly deploy and host your Subgraphs in the SubQuery Hosted Service

The biggest dApps depend on SubQuery's enterprise level Managed Service. With 100's of millions of daily requests and hundreds of active projects, SubQuery's Managed Service provides industry leading hosting for our customers.

• We'll run your SubQuery projects for you in a high performance, scalable, and managed public service.
• This service is being provided to the community with a generous free tier! You can host your first two SubQuery projects for absolutely free!
• You can make your projects public so that they'll be listed in the SubQuery Explorer and anyone around the world can view them.

You can upgrade to take advantage of the following paid services:

• Production ready hosting for mission critical data with zero-downtime blue/green deployments
• Dedicated databases
• Multiple geo-redundant clusters and intelligent routing
• Advanced monitoring and analytics.

Publish your SubQuery project to IPFS

When deploying to SubQuery's Managed Service, you must first host your codebase in IPFS. Hosting a project in IPFS makes it available for everyone and reduces your reliance on centralised services like GitHub.

Warning

GitHub Deployment flows have been deprecated for IPFS

If your project is still being deployed via GitHub, read the migration guide for IPFS deployments here

Requirements

• @subql/cli version 0.21.0 or above.
• Manifest specVersion 1.0.0 and above.
• Get your SUBQL_ACCESS_TOKEN ready.
• To make sure your deployment is successful, we strongly recommend that you build your project with the subql build command, and test it locally before publishing.

Prepare your SUBQL_ACCESS_TOKEN

• Step 1: Go to SubQuery Managed Service and log in.
• Step 2: Click on your profile at the top right of the navigation menu, then click on Refresh Token.
• Step 3: Copy the generated token.
• Step 4: To use this token:
  • Option 1: Add SUBQL_ACCESS_TOKEN in your environment variables. EXPORT SUBQL_ACCESS_TOKEN=<token> (Windows) or export SUBQL_ACCESS_TOKEN=<token> (Mac/Linux)
  • Option 2: Coming soon, subql/cli will support storing your SUBQL_ACCESS_TOKEN locally.

How to publish a project

Run the following command, which will read the project's default manifest project.ts for the required information. If you have @subql/cli version 5.0.0 or above, you will need to install @subql/common-<network> package in the dependencies before execute this command.

// Publish it from your project's root directory
subql publish

// OR point to your project root
subql publish -f ~/my-project/

Alternatively, if your project has multiple manifest files, for example you support multiple networks but share the same mapping and business logic, and have a project structure as follows:

L projectRoot
 L src/
 L package.json
 L polkadot.yaml (Manifest for Polkadot network)
 L kusama.yaml   (Manifest for Kusama network)
 ...

You can always publish the project with your selected manifest file.

 # This will publish project support indexing Polkadot network
subql publish -f ~/my-projectRoot/polkadot.yaml

After successfully publishing the project, the logs below indicate that the project was created on the IPFS cluster and have returned its CID (Content Identifier). Please note down this CID.

Building and packing code... done
Uploading SupQuery project to IPFS
SubQuery Project uploaded to IPFS: QmZ3q7YZSmhwBiot4PQCK3c7Z6HkteswN2Py58gkkZ8kNd  //CID

Note:

• With @subql/cli version 1.3.0 or above, when using subql publish, a copy of the project's IPFS CID will be stored in a file in your project directory. The naming of the file will be consistent with your project.ts. For example, if your manifest file is named project.ts, the IPFS file will be named .project-cid.
• With @subql/cli version 5.0.0 or above, you will need to install @subql/common-<network> package in the dependencies before execute this command. For example, for substrate project, please make sure@subql/common-substrate 4.0.0 is installed

What happens during IPFS Deployment?

IPFS deployment represents an independent and unique existence of a SubQuery project on a decentralised network. Therefore, any changes with the code in the project will affect its uniqueness. If you need to adjust your business logic, e.g. change the mapping function, you must republish the project, and the CID will change.

For now, to view the project you have published, use a REST API tool such as Postman, and use the POST method with the following example URL to retrieve it:https://ipfs.subquery.network/ipfs/api/v0/cat?arg=<YOUR_PROJECT_CID>.

You should see the example project deployment as below.

This deployment looks very similar to your manifest file. You can expect those descriptive fields, and the network and dictionary endpoint has been removed as they did not directly affect the outcome of project execution.

Those files been used in your local project has been packed and published to IPFS as well.

dataSources:
  - kind: substrate/Runtime
    mapping:
      file: ipfs://QmTTJKrMVzCZqmRCd5xKHbKymtQQnHZierBMHLtHHGyjLy
      handlers:
        - handler: handleBlock
          kind: substrate/BlockHandler
        - filter:
            method: Deposit
            module: balances
          handler: handleEvent
          kind: substrate/EventHandler
        - handler: handleCall
          kind: substrate/CallHandler
    startBlock: 8973820
network:
  genesisHash: "0x91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3"
schema:
  file: ipfs://QmTP5BjtxETVqvU4MkRxmgf8NbceB17WtydS6oQeHBCyjz
specVersion: 0.2.0

Publish your Subgraph project to IPFS

In order to publish a Subgraph project to the SubQuery Network, you must first upload it to IPFS and retrieve a publicly accessible IPFS CID.

There are two ways to do this. First, if the Subgraph is already live on the Graph Network, you can retrieve the IPFS CID from the Graph Explorer. In the Graph Explorer this value is called the Deployment ID on the website.

Alternatively, if you have not deployed the Subgraph project on the Graph Network, you can use the Graph's command line interface graph build -i https://unauthipfs.subquery.network/ipfs/api/v0. This will return you an IPFS CID that you can use.

Login to SubQuery Projects

To create your first project, head to SubQuery Managed Service. You'll need to authenticate with your GitHub account to login.

On first login, you will be asked to authorise SubQuery. We only need your email address to identify your account, and we don't use any other data from your GitHub account for any other reasons. In this step, you can also request or grant access to your GitHub Organisation account so you can post SubQuery projects under your GitHub Organisation instead of your personal account.

SubQuery Projects is where you manage all your hosted projects uploaded to the SubQuery platform. You can create, delete, and even upgrade projects all from this application.

If you have a GitHub Organisation accounts connected, you can use the switcher on the header to change between your personal account and your GitHub Organisation account. Projects created in a GitHub Organisation account are shared between members in that GitHub Organisation. To connect your GitHub Organisation account, you can follow the steps here.

Create Your First Project

Before starting, please make sure that your SubQuery project codebase is published to IPFS.

There are two methods to create a project in the SubQuery Managed Service: you can use the UI or directly via the subql cli tool

Using the UI

Start by clicking on "Create Project". You'll be taken to the new project form. Start by selecting what project type you would like to deploy (SubQuery project or Subgraph), and then follow the steps and enter the following (you can change this in the future):

• Project Name: Name your project.
• Description: Provide a description of your project.
• Database: Premium customers can access dedicated databases to host production SubQuery projects from. If this interests you, you can contact sales@subquery.network to have this setting enabled.
• Visible in Explorer: If selected, this will show the project from the public SubQuery explorer to share with the community.

Create your project and you'll see it on your SubQuery Project's list. Next, we just need to deploy a new version of it.

Using the CLI

You can also use @subql/cli to create a new deployment of your project to our Managed Service. Please follow the guide on how to create a new project on the SubQuery Managed Service in the CLI documentation.

Subgraph deployments are not supported in the CLI

Deploy your First Version

There are three methods to deploy a new version of your project to the SubQuery Managed Service, you can use the UI or directly, via the subql cli tool, or using an automated GitHub Action.

Using the UI

While creating a project will setup the display behaviour of the project, you must deploy a version of it before it becomes operational. Deploying a version triggers an indexing operation to start, and sets up the required query service to start accepting GraphQL requests. You can also deploy new versions to existing projects here.

With your new project, you'll see a "Deploy your first version" button. Click this, and fill in the required information about the deployment:

• CID: Provide your IPFS deployment CID. Retrieve this by following the steps for SubQuery Projects here and Subgraphs here
• Manifest: The details are obtained from the contents of the provided CID.
• Override Network and Dictionary Endpoints: You can override the endpoints in your project manifest here.
• Indexer Version: This is the version of SubQuery's node service that you want to run this SubQuery on. See @subql/node.
• Query Version: This is the version of SubQuery's query service that you want to run this SubQuery on. See @subql/query.
• Advanced Settings: There are numerous advanced settings which are explained via the in built help feature.

If deployed successfully, you'll see the indexer start working and report back progress on indexing the current chain. This process may take time until it reaches 100%.

Using the CLI

You can also use @subql/cli to create a new deployment of your project to our Managed Service. Please follow the guide on how to deploy to an existing project on the SubQuery Managed Service in the CLI documentation.

Subgraph deployments are not supported in the CLI

Deploy new versions of your SubQuery project

Although you have the freedom to always upgrade and deploy new versions of your SubQuery project, please be considerate during this process if your SubQuery project is public to the world. Some key points to note:

• If your upgrade is a breaking change, either create a new project (e.g. My SubQuery Project V2) or give your community plenty of warning of the change through social media channels.
• Deploying a new SubQuery project version causes some downtime as the new version indexes the complete chain from the genesis block.

There are three methods to deploy a new version of your project to the SubQuery Managed Service: you can use the UI, create it directly via the subql cli tool, or use an automated GitHub action.

Using the UI

Log into SubQuery Project and select the project you want to deploy a new version of. You can choose to either deploy to the production or staging slot. These two slots are isolated environments and each has their own databases and synchronise independently.

We recommend deploying to your staging slot only for final staging testing or when you need to resync your project data. You can then promote it to production with zero downtime. You will find testing is faster when running a project locally as you can more easily debug issues.

The staging slot is perfect for:

• Final validation of changes to your SubQuery Project in a separate environment. The staging slot has a different URL to production that you can use in your dApps.
• Warming up and indexing data for an updated SubQuery project to eliminate downtime in your dApp.
• Preparing a new release for your SubQuery Project without exposing it publicly. The staging slot is not shown to the public in the Explorer and has a unique URL that is visible only to you.

Fill in the IPFS CID of the new version of your SubQuery project codebase that you want deployed (see the documentation to publish to IPFS here. This will cause a longer downtime depending on the time it takes to index the current chain. You can always report back here for progress.

Using the CLI

You can also use @subql/cli to create a new deployment of your project to our Managed Service. Please follow the guide on how to deploy a new version of your project on the SubQuery Managed Service in the CLI documentation.

Subgraph deployments are not supported in the CLI

Using GitHub actions

Subgraph deployments are not supported with GitHub Actions

With the introduction of the deployment feature for the CLI, we've added a Default Action Workflow to the starter project in GitHub that will allow you to publish and deploy your changes automatically:

• Step 1: After pushing your project to GitHub, create DEPLOYMENT environment on GitHub, and add the secret SUBQL_ACCESS_TOKEN and another secret with the name ENDPOINT which matches the RPC API endpoint that you want to connect (you can retrieve this from your project.ts and include a private API key).
• Step 2: If you haven't already, create a project on SubQuery Managed Service. This can be done using the UI or CLI.
• Step 3: Once your project is created, navigate to the GitHub Actions page of your project, and select the workflow CLI deploy.
• Step 4: You'll see an input field where you can enter the unique code of your project created on SubQuery Projects. You can get the code from the URL in SubQuery's Managed Service SubQuery Managed Service. The code is based on the name of your project, where spaces are replaced with hyphens -. e.g. my project name becomes my-project-name.

Tip

Once the workflow is complete, you should be able to see your project deployed to our Managed Service.

A common approach is to extend the default GitHub Action to automatically deploy changes to our Managed Service when code is merged into the main branch. The following change to the GitHub Action workflow do this:

on:
  push:
    branches:
      - main
jobs:
  deploy:
    name: CLI Deploy
    ...

Next Steps - Connect to your Project

Once your deployment has successfully completed and our nodes have indexed your data from the chain, you'll be able to connect to your project via the displayed GraphQL Query endpoint.

Alternatively, you can click on the three dots next to the title of your project, and view it on SubQuery Explorer. There you can use the in browser playground to get started - read more about how to use our Explorer's GraphQL playground here.

Project Alert Notifications

Subgraph deployments do not support Project Alert Notifications

SubQuery Managed Service provides a service where you can receive alerts on the health status of your SubQuery projects. This means you can be alerted in real-time when your project becomes unhealthy and you can quickly resolve the issue to avoid any impact to your users.

You can easily set up a webhook endpoint to receive alert notifications on the health status of your projects on the Alerting page inside of the Managed Service. All you need to do is enter the URL of the endpoint that you would like us to send webhooks to (e.g. Slack, Telegram). For example, you can easily receive notifications in Slack by following this guide, or Discord by following this guide.

SubQuery Managed Service makes POST requests to send these notifications to your specified endpoint as a JSON payload. You can then use these notifications to execute actions in your backend systems. The JSON payload is in the following format

{
  "event_type": "indexer_unhealthy", // Event Type enum
  "event_message": "The indexer service for [jamesbayly/transaction-list][primary] is now unhealthy",
  "text": "The indexer service for [https://explorer.subquery.network/subquery/jamesbayly/projects/transaction-list> ][primary] is now unhealthy", // A longer version of event_message that is compatible with Slack
  "project": "jamesbayly/transaction-list", // Project key
  "project_name": "Polkadot Transactions",
  "project_url": "https://explorer.subquery.network/subquery/jamesbayly/projects/transaction-list?stage=false",
  "slot": "primary" // Either primary or stage
}

We currently support the following event types.

Event Type	What will trigger this event
block_sync_stalled	The block height has stalled in the last 15 mins
block_sync_recover	The block height resumes syncing after a block_sync_stalled event
indexer_unhealthy	The Indexer service transitions to unhealthy
indexer_healthy	The Indexer service transitions to healthy status
query_unhealthy	The Query service transitions to unhealthy
query_healthy	The Query service transitions to healthy status
deployment_started	A deployment starts
deployment_success	A deployment succeeds
deployment_failed	A deployment fails

Upgrade to the Latest Indexer and Query Service

If you just want to upgrade to the latest indexer (@subql/node) or query service (@subql/query) to take advantage of our regular performance and stability improvements, just select a newer versions of our packages and save. This will cause only a few minutes of downtime as the services running your project are restarted.

Add GitHub Organization Account to SubQuery Projects

It is common to publish your SubQuery project under the name of your GitHub Organization account rather than your personal GitHub account. At any point you can change your currently selected account on SubQuery Managed Service using the account switcher.

If you can't see your GitHub Organization account listed in the switcher, then you may need to grant access to SubQuery for your GitHub Organization (or request it from an administrator). To do this, you first need to revoke permissions from your GitHub account to the SubQuery Application. Then, login to your account settings in GitHub, go to Applications, and under the Authorized OAuth Apps tab, revoke SubQuery - you can follow the exact steps here. Don't worry, this will not delete your SubQuery project and you will not lose any data.

Once you have revoked access, log out of SubQuery Managed Service and log back in again. You should be redirected to a page titled Authorize SubQuery where you can request or grant SubQuery access to your GitHub Organization account. If you don't have admin permissions, you must make a request for an administrator to enable this for you.

Once this request has been approved by your administrator (or if are able to grant it youself), you will see the correct GitHub Organization account in the account switcher.

Sync your Managed Service Account to the SubQuery Network

You can connect your SubQuery Managed Service account to your Ethereum wallet to enable seamless integration between the SubQuery Network and the SubQuery Managed Service. This may provide discounts when running your projects in the hosted service, and will allow you to migrate your project to the network in a single click.

Head to the Wallet menu and follow the guide to connect your wallet.

To bind a wallet, you will be asked to sign a simple transaction, this costs nothing. You are able to disconnect the wallet or connect a different wallet if you need to.