Building a reliable and performant indexer for CKB can be a complex task. It often involves handling chain reorgs, managing parallel data fetching, and ensuring data consistency, all while trying to keep the codebase maintainable.
To address these challenges, I’ve been working on ChainDrain, a lightweight and extensible indexing toolkit.
Core Concepts:
ChainDrain is built around a simple core loop:
- Fetch blocks from a CKB node.
- Process blocks using a developer-defined
handlerfunction. - Persist the indexed data to a storage of your choice.
This is augmented by a powerful middleware system that allows you to extend the core functionality. For example, you can add middleware to:
- Handle chain reorgs gracefully.
- Implement retry logic for failed operations.
- Add timeouts to prevent long-running tasks.
- Batch operations for improved performance.
Key Features:
- Lightweight and dependency-free: ChainDrain has zero dependencies, making it easy to integrate into any project.
- TypeScript-first: Written entirely in TypeScript, providing strong typing and a better developer experience.
- Chain-agnostic by design: While my primary focus is CKB, the core logic is chain-agnostic, allowing you to index data from multiple chains (e.g., tracking RGB++ commitments on BTC and preimage on CKB).
- Performance-oriented: Supports parallel block processing and batched database operations to maximize indexing speed.
- Extensible via middleware: The middleware architecture allows you to customize the indexing process to your specific needs.
I believe ChainDrain can significantly simplify the process of building custom indexes on CKB. Would love to get your feedback.
You can check out the source code and examples on GitHub.
Let me know what you think!