Some client questions from a potential contributor

A potential contributor (Thao) I got in contact with through the foundation’s hiring pipeline had a couple inquiries, appreciate any information anyone can share pertaining to them:

  1. “About this issue, I had
    some experiences with LMDB and I also checked the source code of that part.
    I think the KeyValueBackend trait should be improved, like an LMDB
    Wrapper. And that trait should be Send & Sync (because Rust automatically
    derived, methodless & marker traits) to guarantee thread safety, as well as
    atomic transactions to ensure database integrity.”

  2. "As I can see, we’re still using JSON-RPC.
    We could use gRPC instead of JSON-RPC for the APIs because it is better fit
    for transferring binary data, faster, more compact and can reuse the
    proto-described structures transparently to fit many languages (Python, Go,
    Rust) without manually implementing and maintaining
    serialization/deserialization for each one.
    Besides that, Protobuf or “proto” is a schema-based binary
    serialization/deserialization system. It is designed to be fast and compact.
    gRPC & Protobuf support streaming and versioning as well, which I think
    also contributed an important part in high performance.

(Continued) IMHO, for the APIs, gRPC and Protobuf seems to be the most mature ecosystem.
Actually, we can use a mix of Protobuf and custom serialization in the gRPC
endpoint. Switching to full protobuf is not mandatory and could be kept
partially to support the new chosen CKB core serialization format.

In addition, gRPC supports transcoding Protobuf to JSON via proxies
(internal or external) and transport over HTTP/1 and WebSockets via proxies
(internal or external). So here is the answer if someone is worried about
gRPC not matching frontend libs.

To summarize, using gRPC together with Protobuf will bring huge benefits in
terms of performance, as well as simplify the data ser/deser process. There
is also no need to write any additional client code because the protoc
compiler will autogenerate client libraries - including wire serialization
and deserialization code - for just about every major language. The
compiler writes thousands of lines of code so we don’t have to. Also, No
more problems streaming data from a server. gRPC supports both 1-to-1 RPCs
and 1-to-many streaming requests, no more awkward REST / WebSocket combo

Many blockchain applications use JSON-RPC due to its lightweight, simplicity and yet sufficient. JSON-RPC does not require message encoding and decoding using Protocol Buffers and does not depend on the mode and method of transport. Implementing a JSON-RPC client is very, very simple. while gRPC or other protocols may offer better performance and more advanced features in certain cases, they are also more complex and requires more resources and technical support.

Moreover, there is a historical reason why many blockchain projects extensively use JSON-RPC. Starting from Bitcoin and later on Ethereum, the blockchain community has widely adopted JSON-RPC, and it has become the de facto standard for many blockchain-related applications such as wallets, DApp developers, mining pools, and more. JSON-RPC has become a well-established technology within the blockchain community, and many developers are familiar with its usage and implementation.

In summary, we have no plans to eliminate or replace JSON-RPC, but we are open to considering other options as additional like gRPC or GraphQL if there is sufficient demand from users to drive.


Thanks @matt.bit brought my points for discussion here.
@zhangsoledad Thank you for taking the time to answer my questions.
yes, I totally agree using JSON-RPC because of its history, in my email it was also mentioned that we don’t necessarily have to get rid of JSON-RPC completely but should only combine it with gRPC where needed. For example, BE-BE communication especially in CKB core, it completely improves performance significantly.
We could expose grpc endpoints for uni and bi-directional streaming, as well as unary calls which can be a 2nd alternative to JSON-RPC and that could allow ppl to interact with the node from within backends more efficiently. We definitely need to keep JSON-RPC.

But, the above is purely my opinion.
I haven’t penetrated CKB long enough, so if there are any mistakes, please forgive me.