Publish to Managed Services
Publish to Managed Services
Benefits of hosting your project with SubQuery's Managed Service
Subgraphs are now supported in SubQuery's Managed Service
Run your projects side by side together with SubQuery's Subgraph hosting. One hosting provider for all your data indexing needs. Read more about how to quickly deploy and host your Subgraphs in the SubQuery Hosted Service
The biggest dApps depend on SubQuery's enterprise level Managed Service. With 100's of millions of daily requests and hundreds of active projects, SubQuery's Managed Service provides industry leading hosting for our customers.
- We'll run your SubQuery projects for you in a high performance, scalable, and managed public service.
- This service is being provided to the community with a generous free tier! You can host your first two SubQuery projects for absolutely free!
- You can make your projects public so that they'll be listed in the SubQuery Explorer and anyone around the world can view them.
You can upgrade to take advantage of the following paid services:
- Production ready hosting for mission critical data with zero-downtime blue/green deployments
- Dedicated databases
- Multiple geo-redundant clusters and intelligent routing
- Advanced monitoring and analytics.
Publish your SubQuery project to IPFS
When deploying to SubQuery's Managed Service, you must first host your codebase in IPFS. Hosting a project in IPFS makes it available for everyone and reduces your reliance on centralised services like GitHub.
Warning
GitHub Deployment flows have been deprecated for IPFS
If your project is still being deployed via GitHub, read the migration guide for IPFS deployments here
Requirements
@subql/cli
version 0.21.0 or above.- Manifest
specVersion
1.0.0 and above. - Get your SUBQL_ACCESS_TOKEN ready.
- To make sure your deployment is successful, we strongly recommend that you build your project with the
subql build
command, and test it locally before publishing.
Prepare your SUBQL_ACCESS_TOKEN
- Step 1: Go to SubQuery Managed Service and log in.
- Step 2: Click on your profile at the top right of the navigation menu, then click on Refresh Token.
- Step 3: Copy the generated token.
- Step 4: To use this token:
- Option 1: Add SUBQL_ACCESS_TOKEN in your environment variables.
EXPORT SUBQL_ACCESS_TOKEN=<token>
(Windows) orexport SUBQL_ACCESS_TOKEN=<token>
(Mac/Linux) - Option 2: Coming soon,
subql/cli
will support storing your SUBQL_ACCESS_TOKEN locally.
- Option 1: Add SUBQL_ACCESS_TOKEN in your environment variables.
How to publish a project
Run the following command, which will read the project's default manifest project.ts
for the required information. If you have @subql/cli
version 5.0.0
or above, you will need to install @subql/common-<network>
package in the dependencies before execute this command.
// Publish it from your project's root directory
subql publish
// OR point to your project root
subql publish -f ~/my-project/
Alternatively, if your project has multiple manifest files, for example you support multiple networks but share the same mapping and business logic, and have a project structure as follows:
L projectRoot
L src/
L package.json
L polkadot.yaml (Manifest for Polkadot network)
L kusama.yaml (Manifest for Kusama network)
...
You can always publish the project with your selected manifest file.
# This will publish project support indexing Polkadot network
subql publish -f ~/my-projectRoot/polkadot.yaml
After successfully publishing the project, the logs below indicate that the project was created on the IPFS cluster and have returned its CID
(Content Identifier). Please note down this CID
.
Building and packing code... done
Uploading SupQuery project to IPFS
SubQuery Project uploaded to IPFS: QmZ3q7YZSmhwBiot4PQCK3c7Z6HkteswN2Py58gkkZ8kNd //CID
Note:
- With
@subql/cli
version 1.3.0 or above, when usingsubql publish
, a copy of the project'sIPFS CID
will be stored in a file in your project directory. The naming of the file will be consistent with your project.ts. For example, if your manifest file is namedproject.ts
, the IPFS file will be named.project-cid
. - With
@subql/cli
version5.0.0
or above, you will need to install@subql/common-<network>
package in the dependencies before execute this command. For example, for substrate project, please make sure@subql/common-substrate
4.0.0 is installed
What happens during IPFS Deployment?
IPFS deployment represents an independent and unique existence of a SubQuery project on a decentralised network. Therefore, any changes with the code in the project will affect its uniqueness. If you need to adjust your business logic, e.g. change the mapping function, you must republish the project, and the CID
will change.
For now, to view the project you have published, use a REST
API tool such as Postman, and use the POST
method with the following example URL to retrieve it:https://ipfs.subquery.network/ipfs/api/v0/cat?arg=<YOUR_PROJECT_CID>
.
You should see the example project deployment as below.
This deployment looks very similar to your manifest file. You can expect those descriptive fields, and the network and dictionary endpoint has been removed as they did not directly affect the outcome of project execution.
Those files been used in your local project has been packed and published to IPFS as well.
dataSources:
- kind: substrate/Runtime
mapping:
file: ipfs://QmTTJKrMVzCZqmRCd5xKHbKymtQQnHZierBMHLtHHGyjLy
handlers:
- handler: handleBlock
kind: substrate/BlockHandler
- filter:
method: Deposit
module: balances
handler: handleEvent
kind: substrate/EventHandler
- handler: handleCall
kind: substrate/CallHandler
startBlock: 8973820
network:
genesisHash: "0x91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3"
schema:
file: ipfs://QmTP5BjtxETVqvU4MkRxmgf8NbceB17WtydS6oQeHBCyjz
specVersion: 0.2.0
Publish your Subgraph project to IPFS
In order to publish a Subgraph project to the SubQuery Network, you must first upload it to IPFS and retrieve a publicly accessible IPFS CID.
There are two ways to do this. First, if the Subgraph is already live on the Graph Network, you can retrieve the IPFS CID from the Graph Explorer. In the Graph Explorer this value is called the Deployment ID on the website.
Alternatively, if you have not deployed the Subgraph project on the Graph Network, you can use the Graph's command line interface graph build -i https://unauthipfs.subquery.network/ipfs/api/v0
. This will return you an IPFS CID that you can use.
Login to SubQuery Projects
To create your first project, head to SubQuery Managed Service. You'll need to authenticate with your GitHub account to login.
On first login, you will be asked to authorise SubQuery. We only need your email address to identify your account, and we don't use any other data from your GitHub account for any other reasons. In this step, you can also request or grant access to your GitHub Organisation account so you can post SubQuery projects under your GitHub Organisation instead of your personal account.
SubQuery Projects is where you manage all your hosted projects uploaded to the SubQuery platform. You can create, delete, and even upgrade projects all from this application.
If you have a GitHub Organisation accounts connected, you can use the switcher on the header to change between your personal account and your GitHub Organisation account. Projects created in a GitHub Organisation account are shared between members in that GitHub Organisation. To connect your GitHub Organisation account, you can follow the steps here.
Create Your First Project
Before starting, please make sure that your SubQuery project codebase is published to IPFS.
There are two methods to create a project in the SubQuery Managed Service: you can use the UI or directly via the subql
cli tool
Using the UI
Start by clicking on "Create Project". You'll be taken to the new project form. Start by selecting what project type you would like to deploy (SubQuery project or Subgraph), and then follow the steps and enter the following (you can change this in the future):
- Project Name: Name your project.
- Description: Provide a description of your project.
- Database: Premium customers can access dedicated databases to host production SubQuery projects from. If this interests you, you can contact sales@subquery.network to have this setting enabled.
- Visible in Explorer: If selected, this will show the project from the public SubQuery explorer to share with the community.
Create your project and you'll see it on your SubQuery Project's list. Next, we just need to deploy a new version of it.
Using the CLI
You can also use @subql/cli
to create a new deployment of your project to our Managed Service. Please follow the guide on how to create a new project on the SubQuery Managed Service in the CLI documentation.
Subgraph deployments are not supported in the CLI
Deploy your First Version
There are three methods to deploy a new version of your project to the SubQuery Managed Service, you can use the UI or directly, via the subql
cli tool, or using an automated GitHub Action.
Using the UI
While creating a project will setup the display behaviour of the project, you must deploy a version of it before it becomes operational. Deploying a version triggers an indexing operation to start, and sets up the required query service to start accepting GraphQL requests. You can also deploy new versions to existing projects here.
With your new project, you'll see a "Deploy your first version" button. Click this, and fill in the required information about the deployment:
- CID: Provide your IPFS deployment CID. Retrieve this by following the steps for SubQuery Projects here and Subgraphs here
- Manifest: The details are obtained from the contents of the provided CID.
- Override Network and Dictionary Endpoints: You can override the endpoints in your project manifest here.
- Indexer Version: This is the version of SubQuery's node service that you want to run this SubQuery on. See
@subql/node
. - Query Version: This is the version of SubQuery's query service that you want to run this SubQuery on. See
@subql/query
. - Advanced Settings: There are numerous advanced settings which are explained via the in built help feature.
If deployed successfully, you'll see the indexer start working and report back progress on indexing the current chain. This process may take time until it reaches 100%.
Using the CLI
You can also use @subql/cli
to create a new deployment of your project to our Managed Service. Please follow the guide on how to deploy to an existing project on the SubQuery Managed Service in the CLI documentation.
Subgraph deployments are not supported in the CLI
Deploy new versions of your SubQuery project
Although you have the freedom to always upgrade and deploy new versions of your SubQuery project, please be considerate during this process if your SubQuery project is public to the world. Some key points to note:
- If your upgrade is a breaking change, either create a new project (e.g.
My SubQuery Project V2
) or give your community plenty of warning of the change through social media channels. - Deploying a new SubQuery project version causes some downtime as the new version indexes the complete chain from the genesis block.
There are three methods to deploy a new version of your project to the SubQuery Managed Service: you can use the UI, create it directly via the subql
cli tool, or use an automated GitHub action.
Using the UI
Log into SubQuery Project and select the project you want to deploy a new version of. You can choose to either deploy to the production or staging slot. These two slots are isolated environments and each has their own databases and synchronise independently.
We recommend deploying to your staging slot only for final staging testing or when you need to resync your project data. You can then promote it to production with zero downtime. You will find testing is faster when running a project locally as you can more easily debug issues.
The staging slot is perfect for:
- Final validation of changes to your SubQuery Project in a separate environment. The staging slot has a different URL to production that you can use in your dApps.
- Warming up and indexing data for an updated SubQuery project to eliminate downtime in your dApp.
- Preparing a new release for your SubQuery Project without exposing it publicly. The staging slot is not shown to the public in the Explorer and has a unique URL that is visible only to you.
Fill in the IPFS CID of the new version of your SubQuery project codebase that you want deployed (see the documentation to publish to IPFS here. This will cause a longer downtime depending on the time it takes to index the current chain. You can always report back here for progress.
Using the CLI
You can also use @subql/cli
to create a new deployment of your project to our Managed Service. Please follow the guide on how to deploy a new version of your project on the SubQuery Managed Service in the CLI documentation.
Subgraph deployments are not supported in the CLI
Using GitHub actions
Subgraph deployments are not supported with GitHub Actions
With the introduction of the deployment feature for the CLI, we've added a Default Action Workflow to the starter project in GitHub that will allow you to publish and deploy your changes automatically:
- Step 1: After pushing your project to GitHub, create
DEPLOYMENT
environment on GitHub, and add the secret SUBQL_ACCESS_TOKEN and another secret with the nameENDPOINT
which matches the RPC API endpoint that you want to connect (you can retrieve this from yourproject.ts
and include a private API key). - Step 2: If you haven't already, create a project on SubQuery Managed Service. This can be done using the UI or CLI.
- Step 3: Once your project is created, navigate to the GitHub Actions page of your project, and select the workflow
CLI deploy
. - Step 4: You'll see an input field where you can enter the unique code of your project created on SubQuery Projects. You can get the code from the URL in SubQuery's Managed Service SubQuery Managed Service. The code is based on the name of your project, where spaces are replaced with hyphens
-
. e.g.my project name
becomesmy-project-name
.
Tip
Once the workflow is complete, you should be able to see your project deployed to our Managed Service.
A common approach is to extend the default GitHub Action to automatically deploy changes to our Managed Service when code is merged into the main branch. The following change to the GitHub Action workflow do this:
on:
push:
branches:
- main
jobs:
deploy:
name: CLI Deploy
...
Next Steps - Connect to your Project
Once your deployment has successfully completed and our nodes have indexed your data from the chain, you'll be able to connect to your project via the displayed GraphQL Query endpoint.
Alternatively, you can click on the three dots next to the title of your project, and view it on SubQuery Explorer. There you can use the in browser playground to get started - read more about how to use our Explorer's GraphQL playground here.
Project Alert Notifications
Subgraph deployments do not support Project Alert Notifications
SubQuery Managed Service provides a service where you can receive alerts on the health status of your SubQuery projects. This means you can be alerted in real-time when your project becomes unhealthy and you can quickly resolve the issue to avoid any impact to your users.
You can easily set up a webhook endpoint to receive alert notifications on the health status of your projects on the Alerting page inside of the Managed Service. All you need to do is enter the URL of the endpoint that you would like us to send webhooks to (e.g. Slack, Telegram). For example, you can easily receive notifications in Slack by following this guide, or Discord by following this guide.
SubQuery Managed Service makes POST requests to send these notifications to your specified endpoint as a JSON payload. You can then use these notifications to execute actions in your backend systems. The JSON payload is in the following format
{
"event_type": "indexer_unhealthy", // Event Type enum
"event_message": "The indexer service for [jamesbayly/transaction-list][primary] is now unhealthy",
"text": "The indexer service for [https://explorer.subquery.network/subquery/jamesbayly/projects/transaction-list> ][primary] is now unhealthy", // A longer version of event_message that is compatible with Slack
"project": "jamesbayly/transaction-list", // Project key
"project_name": "Polkadot Transactions",
"project_url": "https://explorer.subquery.network/subquery/jamesbayly/projects/transaction-list?stage=false",
"slot": "primary" // Either primary or stage
}
We currently support the following event types.
Event Type | What will trigger this event |
---|---|
block_sync_stalled | The block height has stalled in the last 15 mins |
block_sync_recover | The block height resumes syncing after a block_sync_stalled event |
indexer_unhealthy | The Indexer service transitions to unhealthy |
indexer_healthy | The Indexer service transitions to healthy status |
query_unhealthy | The Query service transitions to unhealthy |
query_healthy | The Query service transitions to healthy status |
deployment_started | A deployment starts |
deployment_success | A deployment succeeds |
deployment_failed | A deployment fails |
Upgrade to the Latest Indexer and Query Service
If you just want to upgrade to the latest indexer (@subql/node
) or query service (@subql/query
) to take advantage of our regular performance and stability improvements, just select a newer versions of our packages and save. This will cause only a few minutes of downtime as the services running your project are restarted.
Add GitHub Organization Account to SubQuery Projects
It is common to publish your SubQuery project under the name of your GitHub Organization account rather than your personal GitHub account. At any point you can change your currently selected account on SubQuery Managed Service using the account switcher.
If you can't see your GitHub Organization account listed in the switcher, then you may need to grant access to SubQuery for your GitHub Organization (or request it from an administrator). To do this, you first need to revoke permissions from your GitHub account to the SubQuery Application. Then, login to your account settings in GitHub, go to Applications, and under the Authorized OAuth Apps tab, revoke SubQuery - you can follow the exact steps here. Don't worry, this will not delete your SubQuery project and you will not lose any data.
Once you have revoked access, log out of SubQuery Managed Service and log back in again. You should be redirected to a page titled Authorize SubQuery where you can request or grant SubQuery access to your GitHub Organization account. If you don't have admin permissions, you must make a request for an administrator to enable this for you.
Once this request has been approved by your administrator (or if are able to grant it youself), you will see the correct GitHub Organization account in the account switcher.
Sync your Managed Service Account to the SubQuery Network
You can connect your SubQuery Managed Service account to your Ethereum wallet to enable seamless integration between the SubQuery Network and the SubQuery Managed Service. This may provide discounts when running your projects in the hosted service, and will allow you to migrate your project to the network in a single click.
Head to the Wallet menu and follow the guide to connect your wallet.
To bind a wallet, you will be asked to sign a simple transaction, this costs nothing. You are able to disconnect the wallet or connect a different wallet if you need to.