Proposal for SubQuery to Bring Indexing Support to CKB

SubQuery Grant Application - CKB

Lead Applicant Name: Brittany Seales & James Bayly

Lead Applicant Email: [email protected] & [email protected]

Telegram Handle: @brittanyseales @jamesbayly

Project Name: SubQuery Pte Ltd

Relevant links: SubQuery | Discord | Twitter | Youtube | LinkedIn | Medium | LinkTree

About our Team

The inception of SubQuery occurred in February 2021 and since then the below core team of 4 has rapidly grown to more than 25 members spread across New Zealand, Singapore, Malaysia, Hong Kong, China and Berlin. We’re a high-performing team, mostly comprised of software developers, devOps engineers, and business development staff with a track record of experience in extending our tooling to different layer 1 chains.

Sam Zou - CEO (https://www.linkedin.com/in/sam-zou-5b8169a/)

Entrepreneur, Investors and more than 20 years of IT experience specialising in infrastructure and cloud service design

Ian He - CTO (https://www.linkedin.com/in/yin-he-7a266345/)

Blockchain Architect, Contributor of polkadot-js, Early adopter of substrate technology and won second place in the first polkadot hackathon.

James Bayly - COO (https://www.linkedin.com/in/james-bayly/)

Software engineer with experience creating and growing 3 startups over the past 5 years

Description of what we plan to build on CKB

Almost every blockchain has a need to process and query data, however everyone knows that a core weakness of blockchain data is that the processing and query performance is extremely inefficient. SubQuery is an open-source platform that solves that and provides a more performant (3.9x faster) and feature-rich alternative to The Graph.

SubQuery’s mission is to make decentralised data more accessible. SubQuery is your own custom open-source API between blockchain data and your dApps and tools. Our indexer works on over 270 networks including Ethereum, BNB, Polygon, Polkadot, Substrate, Avalanche, Cosmos, Near, and Algorand chains (full list here) and then provides that data for developers to use for a wide array of projects (wallets, explorers, custom chains, or any other decentralised app).

We have three products:

  • An open source SDK: includes the instructions on how any Indexer should traverse the blockchain, what data to collect, and how it should be shown to users.

  • A Managed Service: provides enterprise-level hosting (99.9% uptime) for SubQuery projects by our team (with a generous free tier)

  • A decentralised SubQuery Network: The SubQuery Network was launched in February 2024 and is our effort to move towards a decentralised and tokenised network in order to ensure no single point of failure and superior performance for SubQuery. We will add CKB support to the network and will encourage as many participants in the process as possible.

In our view, the CKB ecosystem is poised for an extreme growth period and it requires tools such as SubQuery in order to thrive.

How far along is our Project

SubQuery is an established project launched early 2021. We’ve been building and scaling our tool to hundreds of customers across a huge range of different layer 1s and 2s (see list of over 270 networks here) for some time now. We have a proven product with large customers across most verticals (wallets, deFi, NFTs, gameFi, explorers) etc. You can see some examples of projects using SubQuery in our public explorer.

While our indexer is our hero product, when building the decentralised SubQuery Network we quickly realised it had all the ingredients to address the centralisation issues in web3 more broadly. That’s why our network also hosts fast and decentralised RPC access and more recently, AI inference workloads to power AI apps.

Competitive advantages?

SubQuery is a flexible, cross-chain indexing service similar to the Graph. There are endless possibilities for the variety of data sources that can be analysed and served using SubQuery.

We build SubQuery with the following key competitive advantages in mind:

  • Faster than others. We’re focusing on making SubQuery faster than other solutions with advanced indexing caches and precomputed indices saving developers time, our solution is fast to set-up, fast to manage, and fast to index. During our last benchmark testing, SubQuery was 3.9x faster than the Graph and we’re not stopping there.

  • More Flexible and Feature rich. SubQuery is a scaffold for building custom APIs and we provide additional features like GraphQL subscriptions, automated historical tracking, and more.

  • Open. Customers have already extended our open-source SDK to suit their own custom implementation.

  • Universal. A universal infrastructure stack bringing communities together, developers now have a tool to search, sort, filter and query any data for their app across multiple blockchains.

Additionally, we are committed to running our managed hosted service over the long term. We have made huge investments into it and have many customers relying on it. This will provide an alternative to customers that have been effected by the sunsetting of the Graph’s hosted service.

Business Model

SubQuery’s SDK is open source under the Apache 2.0 Licence. This SDK provides everything that you need to build and run an indexer for your dApp. You can see detailed instructions on how you can run SubQuery locally in our documentation. The SubQuery Network will be open source under the same licence.

The SubQuery managed service is a hosted environment that is closed source and run privately by SubQuery. It does not have any features that customers can’t do themselves and runs the same open source versions of SubQuery SDK.

Value to Nervos & CKB

SubQuery is an incredibly useful tool that developers can use to build dApps faster. It provides a critical level of infrastructure that developers need to access, organise, and utilise data efficiently. By supporting CKB, we will allow our existing customers to easily migrate over to CKB from other ecosystems by providing a common tool. We specialise in new layer-1 chains, and our customers generally take a cross chain approach to development.

We can see why people are excited about Nervos’ aim to foster a thriving Bitcoin ecosystem - SubQuery is designed to support with this. Our goals have always been to build tools and services that accelerate the world’s transition to web3 and CKB is a fascinating chain that has attracted a large developer base.

Growth Strategy Once Funding Received

We build an advanced developer tool that makes a technically challenging process easy. As a result, the sole measure of our success is the number of developers and projects in CKB using our tool to index data for their dApp/service. We put a lot of effort into business development, developer education, and ecosystem outreach to achieve this.

But if we don’t build the best indexer tool, with unmatched features, faster performance, and higher reliability - then we will fail at onboarding developers to use our tool. That’s why we focus on both BD and product development. Additionally, our Managed service solves a huge pain point with small teams that don’t want to focus on reliability and uptime. We completely manage this critical infrastructure for many teams and we’ve become experts at it.

All this to say, we will spend the first 3 months completing the integration work and then putting resources and efforts into cementing ourselves as the go-to indexer in the CKB ecosystem. We will heavily invest our time into the community and prioritise our growth efforts into CKB.

Existing support and partnerships

Read through https://blog.subquery.network/ for all partnership announcements.

Chains we support: https://subquery.network/networks

Project Goals

This funding grant aims to help us to build the next stage of our vision for SubQuery to become a multi-chain tool - SubQuery support encompassing CKB through full testing and implementation.

  • The SubQuery SDK will be open-sourced / free for the CKB community and (test) network agnostic

  • The SubQuery Hosted Service will be available for free for all customers on CKB (up to a basic level with paid enterprise-level support for larger ones to cover infrastructure costs)

  • The decentralised SubQuery Network will be fully compatible with CKB

  • We will deliver a considerable amount of developer documentation, tutorials, and example projects that will help developers in your ecosystem build

  • Our BD team will reach out to multiple developers in your ecosystem make themselves available for any demos or workshops

Milestones and Cost Projection

We estimate the project to take 34 FTE days and cost $29,700.

We have a table detailing the deliverables and cost projection at the bottom of this document:
Full CKB Grant Application

4 Likes

A tool for developers to manage/fetch data easier/faster sounds great for dApps. Curious what everyone else thinks.

1 Like

Would the data provided by SubQuery be any different than the data provided by a CKB node itself?

2 Likes

I’m glad you’re interested in Nervos. How is your proposal different from this project:

3 Likes

On top of @jm9k question. Say the data you provide is indeed higher level than the data natively provided by a native indexer on ckb node, how do you plan on de-serializing the data for every protocol?

As you may be aware, in Nervos L1 there is no single agree upon ABI for the data memorized on-chain for Script Args, Cell Data and Witnesses:

  • Each protocol adopts its own encoding of the data
  • Each protocol schema may or may not not be publicly available

This is the roadblock that has hampered most of the efforts in this direction in the past. At the beginning of this year there was a valiant effort to bridge this metadata gap with CoBuild and more recently with SSRI, with Pausable UDT being currently audited.

How does your solution plan on integrating both these upcoming projects and the ones already deployed on-chain?

Love & Peace, Phroi

6 Likes

Hi @d3fus7.bit thank you for the question!

SubQuery differs by having user-specific projects which allow customised indexing rather than a general set of indexed data. This includes transforming data specific to a user’s need.

1 Like

Hi @phroi appreciate you raising this. I am having the dev team look into this further as you’re right it may make indexing very difficult for us.

1 Like