LS-IDL: a Lock Script Interface Description Language for CKB (derive, validate, commit)

Infrastructure
1

Hey, everyone

I have spent most of my time on CKB looking at low-level transaction structure and scripts. I love that scripts on CKB are fully programmable, unlke Bitcoin where you have a fixed set of script types (P2TR, P2WPKH, etc.) with fixed witness shapes.

However, that flexibility creates a real problem when building transactions. Because I can lock a cell with literally any script I write, there is no machine-readable description of what witness that script expects. Wallets and tx builders either have out-of-band knowledge of the format, or they guess. If the guess is wrong, the transaction fails. The bad thing is that it doesn’t fail at construction time, but with a VM error after it hits the chain.

The worst case: a group of people passing a partially signed transaction around. Some add witnesses in the wrong format, and nobody finds out until the transaction is rejected on-chain. That is bad experience.

This post describes what I am building to address it.

The idea: lock scripts with machine-readable interface

In Ethereum, every smart contract has an Application Binary Interface, which is a JSON description of its functions, evemts, argument types, etc. You can find a contract’s ABI, decode its calldata, and validate that you have built a transaction correctly before submitting it. CKB currently has no such thing. LS-IDL is my attempt to build it.

The system has two Rust crates:

ckb-idl-derive: this is the script side. It is a procedural macro that lets you declare your witness layout directly into your Rust lock script and automatically generate a machine-readable idl.json at compile time. It also generates the runtime deserializer, so the struct you annotate is the one your script actually uses.

ckb-idl-client: this is the tooling side. It is a library that fetches an IDL, verifies it against an onchain commitment in the deployed code cell, and then uses the IDL to structurally validate a proposed witness buffer before you submit a transaction.

Both are written in Rust. The wire format is documented in a test-vectors.json so any reimplementation in Typescript, Python, or some other language can verify correctness against the same cases.

How the Script side Works:

When writing scripts with this, the changes would be adding: #derive(CkbWitness)] to a named-field struct in your lock script crate.

use ckb_idl_derive::CkbWitness;

#[derive(CkbWitness)]
pub struct Witness {
    #[witness(description = "Preimage whose blake2b-256 hash must match the hash in script args")]
    pub preimage: Vec<u8>,
}

Each of the fields would be described with an optional description and required field, which defaults to true if not available.

At compile time, the macro writes idl.json next to your Cargo.toml thus:

{
  "witness": [
    {
      "name": "preimage",
      "type": "bytes",
      "required": true,
      "description": "Preimage whose blake2b-256 hash must match the hash in script args"
    }
  ]
}

And it generates a from_witness_args method on the struct due to the macro, so your script logic looks like this:

fn check_hash() -> Result<(), Error> {
    let witness = Witness::from_witness_args(0, Source::GroupInput)
        .map_err(|_| Error::MissingWitness)?;

    let actual = blake2b_256(witness.preimage.as_slice());
    // compare to expected hash in script args...
}

The above assumes check_hash() is the function that ultimately protects the script (secures the script with your lock).

The struct is live code, not documentation. The same type declaration that drives the IDL also drives the runtime deserializer, so that they do not drift apart.

Another example of this is:

#[derive(CkbWitness)]
pub struct Witness {
    #[witness(description = "secp256k1 ECDSA signature authorising the spend")]
    pub signature: [u8; 65],

    #[witness(description = "Unix timestamp in milliseconds; cell cannot be spent before this")]
    pub unlock_after_ms: u64,

    #[witness(required = false, description = "Optional auxiliary payload")]
    pub extra: Vec<u8>,
}

For multiple fields, and this produces:

{
  "witness": [
    { "name": "signature",       "type": "secp256k1_sig", "required": true  },
    { "name": "unlock_after_ms", "type": "uint64",        "required": true  },
    { "name": "extra",           "type": "bytes",         "required": false }
  ]
}

There is a list of currently supported types: u8, u32, u64, [u8; 65](which is taken as a secp256k1 sig), [u8; 33] (secp256k1 pubkey), [u8; 64] (schnorr sig), and Vec<u8>. Using anything else is a compile-time error. Vec<u8> should cover for most array types with unsupported lengths, and intended fixed-length arrays can be handled in-code as Vecs, with length validation. (The first four bytes of the vec are read as the length prefix, in little-endian). The type registry is intentionally small, I tried to include types that have a single unambiguous CKB meaning at this size, hoping that the community would indicate what is missing.

How the Client side works:

When you deploy a script, you sha256-hash the idl.json and append the 32 bytes to the code cell data.

code_cell_data = risc_v_binary || sha256(idl.json)

The client then has three things to do when you want to build a transaction spending a cell locked by that script:

  • Fetch the IDL from a registry (in my examples, I load it locally)
  • Verify it: take the last 32 bytes of the code cell data and check that sha256(idl_json) == those 32 bytes. If this passes, you know with certainty that the IDL you have fetched is the correct original IDL used in the exact deployed code. The purpose of this step is so you do not have to trust any registry.
  • Validate your witness buffer structurally: parse the witness buf field-by-field against the IDL’s declared types, catch FieldTooShort, TrailingBytes, and UnknownType errors before the transaction leaves your machine.
// Verify the IDL is authentic
client.verify(code_hash, &idl_json_bytes, &code_cell_data)?;

// Validate your proposed witness
let validated = client.validate_witness_bytes(&idl_doc.witness, &wire_bytes)?;
// If Ok, the encoding is structurally valid. Submit the transaction.

Structural validation is the correctness-check before submission. Whether a signature is actually cryptographically valid, whether a timelock has passed, those semantic checks are enforced by the VM. The IDL client handles structure, and leaves actual semantics for the chain.

What is Missing: the registry:

Currently, an uncertain part of the system right now is the fetch step. In my examples, I load the scripts locally, not fetch from a registry. For a scripts to be used by anyone, the IDL has to be available to that person that wants to use it. I have not built a registry yet.

Lately, several people in the ecosyste have talked about script registries, so I thought this could naturally fit into one of the implementations, without me needing to build it myself. The registry would store the IDL alongside the code hash, and provide a query-able endpoint for the client. I wanted to bring this to the community before deciding where to go next.

If you are one of the developers building a registry, I am open and willing to discuss the specs of what I need.

What is currently built:

  • ckb-idl-derive: the proc macro, the type registry and the from_witness_args code generator exist
  • ckb-idl-types: a no_std-compatible companion crate with the WitnessError type used by generated code
  • ckb-idl-client: the IDL commitment verification, structural witness validation, and the test vector runner
  • Two reference lock scripts using the system: simple-lock (simple preimage hash), and timelock-lock (signature + timestamp + original payload)
  • 16 canonical test vectors covering every field type, every error condition, and both reference scripts — these are the normative specification for any reimplementation

Currently underway is a TypeScript implementation, that will be tested against those same test vectors. Also, more complicated scripts than the simple-lock and timelock-lock are currently being written and tested using the LS-IDL

Currently, this only supports Lock Scripts, but Type Scripts are in the roadmap. Once this is working and running, and adopted with Lock Scripts, it would be improved to also support TypeScripts.

Known Limitations:

  • No [u8; 32] support: a 32-byte array is ambiguous (hash? pubkey? arbitrary bytes?). A #[witness(type = "blake2b_hash")] override attribute is the planned path, but still reconsidering if that is good design.
  • Wire format not recorded in the IDL: the JSON describes field names and types but not the length-prefix encoding. A future “encoding” top-level key will fix this.
  • No Molecule support: scripts using WitnessArgs or Molecule-generated types can’t use the macro on those fields today.
  • required = false is documentation only: the runtime decoder doesn’t yet treat optional fields differently. This needs Option field types and updated codegen.
  • The registry doesn’t exist yet: this is the biggest gap needed to be covered for this to work yet.

Questions I am uncertain about:

  • Registry design: where should IDLs live? Should a script-registry own this, or a separate IDL-specific registry, and why? What does a healthy fetch path look like for a wallet that encounters an unknown script?
  • Type registry scope: the current handful of types cover some common cases. How many real scripts are people writing that need types that are not on the list? Should the types (fixed array types of lengths 33, 64, or 65) even have those tags attached to them (schnorr sig, secp256k1 pubkey, etc.)? What other types could be added
  • TypeScript priority: How important would an implementation in TypeScript matter for adoption? I have not had much reason to write any frontend code or node scripts in CKB, but would those clients make use of this as well?

Repositories:

What are your thoughts on this?

9 Likes