Nervos CKB Development Update (will keep update)

English CKB Development & Technical Discussion
1

Nervos CKB Development Update #1

Ian Yang

Dec 29, 2018

covering Nov.26–Dec.21 2018

Hello from the CKB team, since the open-source of Nervos CKB, we have released v0.2.0 this month. I put together a development update covering Nov.26 to Dec.21, and we will be sharing the CKB development update on a bi-weekly basis starting next month.

Thanks for the input from Xuejie Xiao, dingwei.zhang, Jan Xie, James Chen, Jiang Jinyang, Qian Linfeng and Luo Chao.

TL;DR

  • Upgraded Rust to 1.31.0 and migrated to numext.
  • Miner is now a separate process.
  • Started the brand-new P2P framework.
  • Enhanced peers management with inbound connections eviction, group and a new peer store.
  • Revised and refactored the syscalls for future scripts development.
  • More demos: partial signature, non-interactive transfer, fixed cap UDT.
  • Initiated Swift and Java SDK.

Changes in RFCs

The RFC (Request for Comments) process is intended to provide an open and community driven path for new protocols, improvements and best practices. One month later after open source, we have 11 RFCs in draft or proposal status. We haven’t finalized them yet, discussions and comments are welcome.

  • RFC0002 provides an overview of the Nervos Common Knowledge Base (CKB), the core component of the Nervos Network, a decentralized application platform with a layered architecture. The CKB is the layer 1 of Nervos, and serves as a general purpose common knowledge base that provides data, asset, and identity services.
  • RFC0003 introduces the VM for scripting on CKB the layer 1 chain. VM layer in CKB is used to perform a series of validation rules to determine if transaction is valid given transaction’s inputs and outputs. CKB uses RISC-V ISA to implement VM layer. CKB relies on dynamic linking and syscalls to provide additional capabilities required by the blockchain, such as reading external cells or other crypto computations. Any compilers with RV64I support, such as riscv-gcc, riscv-llvm or Rust can be used to generate CKB compatible scripts.
  • RFC0004 is the protocol how CKB nodes synchronize blocks via the P2P network. Block synchronization must be performed in stages with Bitcoin Headers First style. Block is downloaded in parts in each stage and is validated using the obtained parts.
  • RFC0006 proposes Complete Binary Merkle Tree(CBMT) to generate Merkle Root and Merkle Proof for a static list of items in CKB. Currently, CBMT is used to calculate Transactions Root . Basically, CBMT is a complete binary tree , in which every level, except possibly the last, is completely filled, and all nodes are as far left as possible. And it is also a full binary tree, in which every node other than the leaves has two children. Compare with other Merkle trees, the hash computation of CBMT is minimal, as well as the proof size.*
  • RFC0007 describes the scoring system of CKB P2P Networking layer and several networking security strategies based on it.
  • RFC0009 describes syscalls specification, and all the RISC-V VM syscalls implemented in CKB so far.
  • RFC0010 defines the consensus rule “cellbase maturity period”. For each input, if the referenced output transaction is cellbase, it must have at least CELLBASE_MATURITY confirmations; else reject this transaction.
  • RFC0011, transaction filter protocol, allows peers to reduce the amount of transaction data they send. Peer which wants to retrieve transactions of interest, has the option of setting filters on each connection. A filter is defined as a Bloom filter on data derived from transactions.
  • RFC0012 proposes a P2P node discovery protocol. CKB Node Discovery Protocol mainly refers to Satoshi Client Node Discovery, with some modifications to meet our requirements.

Changes in CKB

CKB has released v0.2.0 this month and we will release v0.3.0 next month.

:tada: Rust 2018 :tada:. We have upgraded all the major repositories to Rust 1.31.0 and 2018 edition. After the Rust upgrade, we can switch to numext , which is a high-performance big number library relying on some new features in 1.31.0.

CKB is dockerized. It has never been easier to run a CKB node: docker run -ti nervos/ckb:latest run

The node started via ckb run no longer produces new blocks. This feature is now in a new process which is launched by ckb miner (#52). The new process gets block template from a node and submits new block with resolved PoW puzzle via node’s RPC. The RPC interface for miners is temporary, and we are working on an RFC proposal for this. After this change, we also modularized RPCs (#118). Now each RPC module can be disabled via config file.

Another feature we are actively developing is peers management. This month, we have implemented network group and inbound peer eviction which described in RFC0007. We also delivered a new version of peerstore which allow us to support security strategies defined in RFC0007 in the future.

Annoyed by the problems of existing P2P libraries, we started to work on a brand new P2P protocol from the ground up. It is still in an early stage and is a minimal implementation for a multiplexed p2p network based on yamux that supports mounting custom protocols. We already implemented 3 core components yamux/secio/service. yamux and secio are mainly refer to their corresponding golang implementations, API are clear and easy to use. Those 3 core components are all use channel based lock-free design with good code readability and maintainability. We are adding up more custom protocols layers, and is going to integrate the discovery protocol soon as described in RFC0012.

We have refactored the rust utility library to mock time for debug and test (#111). It is now available as a separate crate.

There are some other features we are still working on, such as implementation of RFC0006 and RFC0011, and the RFC about serialization format CFB. We are going to release them in next month.

Changes in VM

All the CKB related syscalls used in the VM have been revisited and refactored to be more future proof.

CKB VM used in CKB has upgraded to the latest revision with the following notable changes:

  • With experience learned from real contracts, the maximum memory in CKB VM has been reduced from 128MB to 16MB
  • CKB VM has adopted Rust 2018 style

Changes in SDK

Thanks for checking this, and we’d love to hear from you, email us at [email protected] if you have any questions! Wish you a happy new year and may your journey be wonderful.

# Nervos CKB Development Update #2

Ian Yang

Jan 15

Covering Dec22, 2018–Jan 13, 2019

1*XRbUGhqvMCzYaF7rbYkODQ.png1067×563 662 KB

Happy New Year! Here is the first report in 2019. Thanks to Xuejie Xiao, James Chen, Jiang Jinyang, they have also contributed to this writing.

TL;DR

  • Proposed RFC0013 block template protocol and RFC0014 cycle limit.
  • Implemented RFC0011 transaction filter protocol and RFC0014 cycle limit.
  • Added new consensus rule to verify block timestamp.
  • Working on VM JIT.

Changes in RFCs

New RFC

  • RFC0013: block template RFC describes the decentralized CKB mining protocol.
  • RFC0014: cycle limit RFC describes cycle limits used to regulate VM scripts. CKB VM is a flexible VM that is free to implement many control flow constructs, such as loops or branches. As a result, we will need to enforce certain rules in CKB VM to prevent malicious scripts, such as a script with infinite loops.

Updates

  • RFC0003: update CKB VM examples based on latest development (#63)
  • RFC0006: use more reasonable proof structure (#62)

Changes in CKB

CKB has released v0.4.0.

This version includes the implementations of several RFCs, including the two new ones.

  • RFC0011, transaction filter protocol, allows peers to reduce the amount of transaction data they send.
  • RFC0014: cycle limit RFC.

The implementation of RFC0013 block template has started and is scheduled to release in the next version.

CKB has adopted a new consensus rule to verify block timestamp based on the past blocks median time (c63d64b). It will also print a warning message when the node finds out its local time has a big difference with its peers.

We have removed the broken KAD discovery protocol (f2d86ba). Thus the node only connects to the peers listed in the configuration file. The new P2P framework will include the discovery protocol as mentioned in RFC0012.

The peer store uses SQLite now to ease implementing rules defined in RFC0007.

There is also a bundle of incompatible refactorings:

  • CellStatus uses dead and live now. CellOutput#contract is renamed to CellOutput#_type.
  • JSONRPC encodes binary fields in hex.
  • P2P encodes H256 and ProposalShortId using the FlatBuffers struct instead of bytes array before.

See the change log in the release about all the breaking changes.

Changes in VM

  • Add support for RFC0014, the cycle limit RFC.
  • Use checked_add when calculating cycles to avoid integer overflows.
  • CKB VM JIT research and initial implementation (ckb-vm#20).

CKB VM JIT will be a 2-level JIT:

  • A baseline JIT which handles most common code segments, which works quite like QEMU’s TCG or rv8. At this level, JIT compilation time will be as important as execution speed of generated code, so we would only employ certain easier optimizations here.
  • A more sophisticated SSA-based JIT for handling very hot code segments. At this layer, the execution speed of generated code will take much higher priority than JIT compilation time, so more effective but time-consuming optimizations would occur here. We might also introduce existing compiling frameworks to solve this.

Changes in P2P

  • Refactor network service API (PR#9)

Thanks for checking, we plan to have this development update on a bi-weekly basis. See you next time!

3 Likes