CKB Transaction CoBuild Protocol Overview

[janx]

Appendix: Lock/Type Script Validation Rules

中文版本

This section describes the validation rules that a lock/type script needs to include in order to meet the requirements of the CoBuild protocol.

From a convenience and practicality standpoint, we recommend all lock/type scripts support legacy WitnessArgs, then add support for SighashAll Variant, and support for Otx Variant.

Legacy Flow Handling

Considering forward compatibility, lock / type scripts that support the CoBuild protocol may also need to support the legacy flow, which is the existing process based on WitnessArgs structure.

A lock/type script can simultaneously support both CoBuild flow and legacy flow. However, for a specific transaction, a lock/type script must choose one of the two for verification: either use CoBuild flow or use legacy flow.

Usually, all scripts in a transaction should use the same flow for verification. But there’s an exception: if a tranasction uses script A that supports both flows and script B that only supports legacy flow, it’s possible that script A verifies according to CoBuild flow while script B verifies according to legacy flow.

Lock/type scripts should follow these rules to determine the flow to use:

• If there is a witness of the type WitnessLayout in current transaction, use the CoBuild Flow.
  • Note: The condition here is its existence, not uniqueness, not if different WitnessLayout witnesses coexist, nor its compliance with other CoBuild rules.
• Otherwise, use the Legacy Flow.

Lock Script

The main responsibility of a CKB lock script is to perform ownership (owner signature) verification - checking that the input cell is unlocked under the correct conditions, while also ensuring that data in transactions are not tampered with. A lock script that supports both SighashAll and Otx modes should verify transactions according to the following rules:

1. Maintain a set of variables: is, ie, os, oe, cs, ce, hs, he and set them all to 0.
   • is: input start
   • ie: input end
   • os: output start
   • oe: output end
   • cs: cell dep start
   • ce: cell dep end
   • hs: header start
   • he: header end
2. Loop through all witnesses of current tx (loop variable i), check if there’s a witness in current tx using WitnessLayout::OtxStart type.
3. If no witness in current tx uses WitnessLayout::OtxStart, skip to step 8 for further validation.
4. Ensure only one witness is of type WitnessLayout::OtxStart in current tx or else exit with an error message. Set loop variable i as this witness’s index value.
5. Set values of is and ie as start_input_cell in OtxStart, os and oe as start_output_cell in OtxStart, cs and ce as start_cell_deps in OtxStart, hs and he as start_header_deps in OtxStart.
6. Starting from witness at index i+1 , loop through each WitnessLayout::Otx typed witness (exit loop directly if encounter any witness not in WitnessLayout::Otx type), validate each WitnessLayout::Otx witness (i.e. each Otx in this tx) as follows:
   • a. Let variable ‘otx’ store the WitnessLayout::Otx type witness, then the following data constitutes current Otx (open transaction):
     • i. Input cells with index within [ie, ie + otx.input_cells)
     • ii. Output cells with index within [oe, oe + otx.output_cells)
     • iii. Cell deps with index within [ce, ce + otx.cell_deps)
     • iv. Header deps with index within [he, he + otx.header_deps)
     • v. The current WitnessLayout::Otx typed witness
   • b. Check otx, if all values for ‘otx.input_cells’, ‘otx.output_cells’, ‘otx.cell_deps’, and ‘otx.header_deps’ are 0, the script should exit with an error.
   • c. Parse otx.message, check each Action in the message, the script_hash in Action must either be the same as the type script hash of one input/output cells in transaction, or be the same as the lock script hash of one input cells in transaction. Otherwise, the script shall terminate with an error.
     • Note that this check applies to the entire transaction not just current otx.
   • d. Check input cells in the range [ie, ie + ox.input_cells), skip to next iteration if none use current lock script.
   • e. Calculate corresponding signing message hash based on Otx mode rules defined in Appendix: CoBuild Hashes.
   • f. Fetch SealPair from seals, using the current lock script’s lock script hash as key . Exit with error if not found.
   • g. Extract signature from SealPair according to the lock’s own logic, verify it against previously generated signing message hash. Exit with error if verification fails.
   • h. ie += otx.input_cells; oe += otx.output_cells; ce += otx.cell_deps; he += otx.header_deps
7. Assume the first witness not of type WitnessLayout::Otx starting from index i+1 is at index j , check that all witnesses in the the range [0, i) and [j, +infinity) are not of type WitnessLayout::Otx or else exit with an error.
8. Check the input cells within the range of [0, is) and [ie, +infinity), if any of them uses current lock script, perform a SighashAll signature verification:
   • a. Check if there’s a WitnessLayout::SighashAll variant among all witnesses in the current tx.
   • b. If there exists a witness of type WitnessLayout::SighashAll, ensure that only one such witness exists in the current tx and store its witness.message in variable message. If no such witness exists in the current tx, set variable message as empty.
   • c. Assuming that message is not empty, check that for each Action contained in message, the script_hash in Action must either be the same as the type script hash of one input/output cells in transaction, or be the same as the lock script hash of one input cells in transaction. Otherwise, the script shall terminate with an error.
   • d. Read the first witness from the script group corresponding to current lock script. If this witness isn’t of type WitnessLayout::SighashAll or WitnessLayout::SighashAllOnly, exit with error.
   • e. Ensure that apart from the first witness position inside current lock script group, there’re either no other witnesses or they’re empty.
   • f. Extract seal from the first witness within current lock script group and retrieve signature according to the lock script’s own custom logic.
   • g. Calculate signing message hash by following the SighashAll/SighashAllOnly rules outlined under Appendix: CoBuild Hashes.
   • h. Verify the extracted signature along with the signing message hash; if verification fails then exit with error.

Type Script

CKB type scripts mainly verify application-specific on-chain state transitions. For instance, in UDT type scripts validate token issuance and transfer. In CoBuild process, apart from validating those state transitions, type scripts should also verify consistency between message and full/open transaction behaviors. During message consistency verification, a type script needs to validate each otx individually as well as a possible SighashAll witness containing message.

A type script supporting both SighashAll and Otx variant should follow these validation rules:

1. Execute the app-specific state transition logic defined by the type script.
2. Maintain a set of variables is, ie, os, oe, cs, ce, hs, he all initialized to 0.
3. Loop through all witnesses in current tx (loop variable i), checking if there’s a witness of type WitnessLayout::OtxStart.
4. If no witness of type WitnessLayout::OtxStart exists in current tx, jump to step 9 for further verification.
5. Ensure only one witness is of type WitnessLayout::OtxStart in the transaction; otherwise exit with error. Set variable i as index value of this witness.
6. Set is and ie values equal to start_input_cell in OtxStart; similarly set os and oe values equal to start_output_cell in OtxStart; set cs and ce values equal to start_cell_deps in OtxStart; finally, set hs and he values equal to start_header_deps in OtxStart.
7. Starting from the witness with index of i + 1, loop through each witness of type WitnessLayout::Otx (exit the loop immediately when encountering a witness that is not of type WitnessLayout::Otx) and perform the following verification for each WitnessLayout::Otx typed witness (i.e. for each Otx in tx):
   • a. Assume that the variable otx stores the witness of type WitnessLayout::Otx, then the following data constitutes the current Otx (Open Transaction):
     • i. Input cells within index range [ie, ie + otx.input_cells)
     • ii. Output cells within index range [oe, oe + otx.output_cells)
     • iii. Cell deps within index range [ce, ce + otx.cell_deps)
     • iv. Header deps within index range [he, he + otx.header_deps)
     • v. The currently processed witness of type WitnessLayout::Otx
   • b. Check input cells within the range of [ie, ie+otx.input_cells) and output cells within the range of [oe, oe+otx.output_cells). If all these input/output cells do not use current type script then skip to next iteration.
   • c. Verify message in WitnessLayout::OTX based on current OTX scope. (Refer to the next section for specific verification process).
   • d.ie += otx.input_cells; oe += otc.output_cells; ce += otz.cell_deps; he+= otz.header_deps
8. Assume starting from the witness at index i+1, the first witness that isn’t of type WitnessLayout::OTX has an index j. Make sure witnesses within [0,i) and [j,+infinity) ranges are not of Witnesslayout::OTX type, otherwise exit with error.
9. Check input and output cells within [0, is) and [ie, +infinity), if any of these cells use the current type script, perform the following verification:
   • a. Scan all witnesses of the current transaction to determine whether there is a witness of WitnessLayout::SighashAll type.
   • b. If no witness of WitnessLayout::SighashAll type exists in the current transaction, consider the script validation successful and exit execution with return code 0.
   • c. Check if there’s exactly one Witnesslayout::SighashAll typed witness in current tx, else exit with an error.
   • d. Verify message in WitnessLayout::SighashAll, based on the scope including input cells within [0, is) & [ie, +infinity) and output cells within [0, os) & [oe, +infinity). (Refer to the next section for specific verification rules.)

Message Validation

This section describes how the type script validates messages.

Whether it’s an Otx or SighashAll mode transaction, we need to validate the Action in message corresponding with this type script against a range of related input/output cells.

Note that the validation is against “a range of cells”: due to the possible existence of otx , this range may not necessarily be the entire tx but could be intervals like [3,6), or the union of two intervals such as [0,4) || [8,11). However there will only ever be union of two ranges maximum; never more than that.

The type script needs to traverse the actions in message, find an Action where Action.script_hash matches its own script hash. This Action is the one corresponding to the current type script.

• If multiple Actions are found that correspond to the type script hash, it is recommended to throw an error. The purpose is to prevent third parties from inserting Actions calling the same type script into the transaction later to override user’s Action.
  • Third-party Actions may appear after user’s Action on wallet UX or even be hide due to insufficient display space, leading users to believe that only their own actions are being signed.
• Example code

Note: The on-chain type script cannot fetch the complete ScriptInfo of the current Action, nor can it verify the correctness of Action.script_info_hash.

Therefore, type scripts can make the following two assumptions:

• The content of Action, including Action.script_info_hash, has been covered by lock scripts during verification process, ensuring no tampering.
• In the signing process, wallets will get the correct ScriptInfo through some offchain channel, verify the consistency between the ScriptInfo and Action.script_info_hash, parse ‘Action.data’ with the correct ‘ScriptInfo.schema’, and present the correctly parsed data to the cell holder for signing.

Although a type script does not have the ScriptInfo, it should have information like ScriptInfo.schema and ScriptInfo.message_type in its source code. Type scripts can use these information to parse Action.data, get the structured action data (as input) and validating cells. For instance, if Action.data presents a Spore NFT Mint operation, the type script needs to verify that cells within certain scope indeed expressed a mint operation; if Action.data presents a UDT transfer, similar verification is required by type script to ensure correct amount of UDT has been transferred to the right recipient and change UDT returned to payer.

[janx]

附录：Lock/Type Script Validation Rules

本节描述符合 CoBuild 流程要求的 lock/type script 需要包含的校验规则。

从方便和实用的角度，我们推荐所有的 lock/type scripts 在支持 legacy WitnessArgs 的基础上，先支持 SighashAll Variant，再支持 Otx Variant。

Legacy Flow Handling

考虑到向前兼容，支持 CoBuild 流程的 lock / type script 很可能也需要支持 legacy flow，即以 WitnessArgs 结构保存 witness 数据的现有流程。

一个 lock / type script 可以同时支持 CoBuild flow 和 legacy flow。但是对于某个特定的交易，一个 lock / type script 只能选取其中一种 flow 来进行校验：要么使用 CoBuild flow，要么使用 legacy flow。

通常情况下，一个 Transaction 中的多个 script 应该都使用同一种 flow 来校验，但是有一种情况例外：在一个 Transaction 中混用支持两种 flow 的 script A，以及只支持 legacy flow 的 script B 时，可能会发生 script A 按照 CoBuild flow 来校验，而 script B 因为只支持 legacy flow 而按照 legacy flow 来校验的情况。

Lock / type script 应按照如下的规则判断在当前交易中，应使用哪种 flow：

• 如果当前交易中存在一个使用 WitnessLayout 结构的 witness 时，则使用 CoBuild flow 进行校验；
  • 注意：这里只要求存在，不要求唯一，也不需要考虑 CoBuild 模式下，不同的 WitnessLayout 结构彼此之间是否可以共存，是否符合 CoBuild 规则。
• 否则，使用 legacy flow 进行校验。

Lock Script

CKB lock script 的主要任务是进行所有权（所有者签名）验证——校验 input cell 以正确的条件被解锁，同时也确保交易中的数据不被篡改。一个同时支持 SighashAll 以及 Otx 模式的 lock script，应按照如下规则对交易进行校验：

1. 维护一组变量 is, ie, os, oe, cs, ce, hs, he, 并全部置为 0.
   • is: input start
   • ie: input end
   • os: output start
   • oe: output end
   • cs: cell dep start
   • ce: cell dep end
   • hs: header start
   • he: header end
2. 循环扫描当前 tx 的所有 witnesses (循环变量 i)，检查当前 tx 中，是否有一个 witness，使用 WitnessLayout::OtxStart 类型
3. 如果当前 tx 中不存在 WitnessLayout::OtxStart 类型的 witness，跳到 step 8 继续执行验证
4. 确保当前 tx 中只有一个 witness 使用 WitnessLayout::OtxStart 结构，否则报错退出。并把当前循环变量 i 设置为该 witness 的 index 值。
5. 把 is 和 ie 置为 OtxStart 中的 start_input_cell 值，把 os 和 oe 置为 OtxStart 中的 start_output_cell 值，把 cs 和 ce 置为 OtxStart 中的 start_cell_deps 值，把 hs 和 he 置为 OtxStart 中的 start_header_deps 值
6. 从 index 为 i + 1 的 witness 开始，循环遍历每一个类型为 WitnessLayout::Otx 的 witness (如果遇到类型不为 WitnessLayout::Otx 的 witness 就直接跳出循环)，并对每一个 WitnessLayout::Otx witness（对应 tx 中包含的每一个 Otx），进行如下校验：
   • a. 假设 otx 变量保存 WitnessLayout::Otx 类型的 witness，那么以下数据构成当前 Otx (open transaction)：
     • i. index 在 [ie, ie + otx.input_cells) 区间内的 input cells
     • ii. index 在 [oe, oe + otx.output_cells) 区间内的 output cells
     • iii. index 在 [ce, ce + otx.cell_deps) 区间内的 cell deps
     • iv. index 在 [he, he + otx.header_deps) 区间内的 header deps
     • v. 当前在处理的， WitnessLayout::Otx 类型的 witness
   • b. 检查 WitnessLayout::Otx 类型的 otx 变量中的内容，如果 otx.input_cells, otx.output_cells, otx.cell_deps, otx.header_deps 全部为 0，script 应该报错退出。
   • c. 解析 WitnessLayout::Otx 类型的 otx 变量中的 message 结构，针对 message 中包含的所有 Action ，检查当前 CKB transaction 中，要么存在一个 input / output cell，它的 type script hash 与 Action 中 script_hash 相同，要么存在一个 input cell，它的 lock script hash 与 Action 中 script_hash 相同，否则报错退出。
     • 注意这里的检查范围是完整的 CKB transaction，并不只是当前的 Otx。
   • d. 检查 index 在 [ie, ie + otx.input_cells) 中的 input cells，如果所有的 input cells 都不使用当前的 lock script，则跳到循环的下一个 iteration 继续执行 (i.e. continue).
   • e. 根据 Appendix: CoBuild Hashes 中定义的计算规则，根据 Otx 计算对应的 signing message hash.
   • f. 以当前 lock script hash 为 key，从 seals 中取出对应当前 lock script 的 SealPair , 如果找不到，则报错退出。
   • g. 从 SealPair 中根据 lock script 的自定义逻辑取出签名，针对前面生成的 signing message hash 进行验签。如果验签失败，则报错退出。
   • h. ie += otx.input_cells; oe += otx.output_cells; ce += otx.cell_deps; he += otx.header_deps
7. 假设从 index 为 i + 1 的 witness 开始，第一个不为 WitnessLayout::Otx 类型的 witness，index 为 j。检查当前 tx 中，index 在 [0, i) 以及 [j, +infinity) 范围内的 witness 都不是 WitnessLayout::Otx 类型，否则报错退出。
8. 检查 index 在 [0, is) 以及 [ie, +infinity) 范围内的 input cells，如果其中有任意一个使用了当前 lock script，则执行一次 SighashAll 验签操作：
   • a. 检查当前 tx 的所有 witnesses 中是否有一个是 WitnessLayout::SighashAll variant.
   • b. 如果存在一个 WitnessLayout::SighashAll 类型的 witness ，确保当前 tx 中只有一个 witness 是 WitnessLayout::SighashAll 类型，并取出 witness.message 保存到 message 变量待后续使用；如果当前 tx 中没有 WitnessLayout::SighashAll 类型的 witness，则把 message 变量设为空。
   • c. 假设 message 不为空，对 message 中包含的每一个 Action ，检查当前 CKB transaction 中，要么存在一个 input / output cell，它的 type script hash 与 Action 中 script_hash 相同，要么存在一个 input cell，它的 lock script hash 与 Action 中 script_hash 相同，否则报错退出。
   • d. 读取当前 lock script 对应 script group 中的第一个 witness，如果该 witness 不为 WitnessLayout::SighashAll 类型，也不为 WitnessLayout::SighashAllOnly 类型，则报错退出。
   • e. 确保当前 lock script group 中除了第一个 witness 的位置，要么不存在 witness，要么为空。
   • f. 从当前 lock script group 的第一个 witness 中取出 seal，并按照 lock script 的自定义逻辑取出签名。
   • g. 依照 Appendix: CoBuild Hashes 的计算规则，计算当前 SighashAll / SighashAllOnly 结构对应的 signing message hash。
   • h. 对从 seal 中取出的签名以及 signing message hash 进行验签。如果验签失败，则报错退出。

Type Script

CKB type script 主要负责链上状态迁移的逻辑校验。例如在 UDT 中，type script 验证不能增发。在 CoBuild 流程中，type script 除了自定义业务逻辑校验之外，也要负责校验 full tx / otx 中，message 与交易内容的一致性。对于数据逻辑本身的验证，type script 可以忽略 otx 的存在，但是在 message 校验中，type script 不仅需要对每一个 otx 分别校验，还要对可能存在的 SighashAll witness 中包含的 message 进行校验。

一个同时支持 SighashAll 以及 Otx variant 的 type script，应该按照如下的规则对 CKB Transaction 进行校验：

1. 执行 type script 自定义状态迁移逻辑的校验。
2. 维护一组变量 is, ie, os, oe, cs, ce, hs, he, 并全部置为 0。
3. 循环扫描当前 tx 的所有 witnesses (循环变量 i)，检查当前 tx 中，是否有一个 witness，使用 WitnessLayout::OtxStart 类型。
4. 如果当前 tx 中不存在 WitnessLayout::OtxStart 类型的 witness，跳到 step 9 继续执行验证。
5. 确保当前 tx 中只有一个 witness 使用 WitnessLayout::OtxStart 结构，否则报错退出。把变量 i 设置为该 witness 的 index 值。
6. 把 is 和 ie 置为 OtxStart 中的 start_input_cell 值，把 os 和 oe 置为 OtxStart 中的 start_output_cell 值，把 cs 和 ce 置为 OtxStart 中的 start_cell_deps 值，把 hs 和 he 置为 OtxStart 中的 start_header_deps 值
7. 从 index 为 i + 1 的 witness 开始，循环遍历每一个类型为 WitnessLayout::Otx 的 witness（当遇到类型不为 WitnessLayout::Otx 的 witness 就直接跳出循环），并对每一个 WitnessLayout::Otx 类型的 witness（对应 tx 中包含的每一个 Otx），进行如下校验：
   • a. 假设 otx 变量保存了 WitnessLayout::Otx 类型的 witness，那么以下数据构成当前的 Otx (Open Transaction)：
     • i. index 在 [ie, ie + otx.input_cells) 区间内的 input cells
     • ii. index 在 [oe, oe + otx.output_cells) 区间内的 output cells
     • iii. index 在 [ce, ce + otx.cell_deps) 区间内的 cell deps
     • iv. index 在 [he, he + otx.header_deps) 区间内的 header deps
     • v. 当前在处理的， WitnessLayout::Otx 类型的 witness
   • b. 检查 index 在 [ie, ie + otx.input_cells) 中的 input cells，以及 index 在 [oe, oe + otx.output_cells) 中的 output cells，如果这些所有的 input / output cells 都不使用当前的 type script，则跳到循环的下一个 iteration 继续执行(i.e. continue)。
   • c. 以当前的 Otx 范围，对 WitnessLayout::Otx 中包含的 message 进行校验（具体校验过程参考下一小节）
   • d. ie += otx.input_cells; oe += otx.output_cells; ce += otx.cell_deps; he += otx.header_deps
8. 假设从 index 为 i + 1 的 witness 开始，第一个不为 WitnessLayout::Otx 类型的 witness，index 为 j。检查当前 tx 中，index 在 [0, i) 以及 [j, +infinity) 范围内的 witness 都不使用 WitnessLayout::Otx 类型，否则报错退出
9. 检查 index 在 [0, is) 以及 [ie, +infinity) 范围内的 input 和 output cells，如果这些 input 和 output cells 中，存在使用当前 type script 的任意一个 cell，进行如下校验：
   • a. 扫描当前 tx 的所有 witnesses，判断当前 tx 中，是否有一个 witness 是 WitnessLayout::SighashAll 类型
   • b. 如果当前 tx 中不存在 WitnessLayout::SighashAll 类型的 witness，则认为当前合约校验成功，以 0 作为 return code 退出执行
   • c. 检查当前 tx 中有且只有一个 witness 是 WitnessLayout::SighashAll 类型，否则报错退出。
   • d. 以 index 在 [0, is) 以及 [ie, +infinity) 内的 input cells，和 index 在 [0, os) 以及 [oe, +infinity) 内 output cells 作为校验范围，对 WitnessLayout::SighashAll 中包含的 message 进行校验（具体校验过程参考下一小节）

Message Validation

本节描述 type script 对 message 的校验过程。

无论是 Otx 还是 SighashAll 模式的交易，都要针对一个范围内的 input / output cells，对 message 中属于当前 type script 的那部分信息（包含在一个 Action 中），进行校验。

注意这里提到 “一个范围” 的 cells：因为 otx 的存在，所以这里的范围不一定是当前的完整 tx，可能是比如 [3, 6) 的这个区间，或者是 [0, 4) || [8, 11) 的这样两个区间的并集。但是最多只会有两个区间的并集，不会有更多区间。

type script 需要遍历 message 中包含的 actions，找到一个 Action.script_hash 与自身的 script hash 相同的 Action，这个 Action 即为对应当前 type script 的 Action。

• 如果发现与自身 script hash 对应的多个 Actions, 推荐做法是报错。这样处理的目的是防止第三方在用户构造 Action 之后往交易里面插入调用同样的 type script 的 Action 覆盖用户 Action。
  • 第三方 Action 有可能在 wallet UX 上显示在用户 Action 后面甚至由于展示空间不够被挡住，导致用户以为要签名的只包含自己的 Action。
• 参考代码

需要注意的是：链上的 type script 无法获取当前 Action 的完整 ScriptInfo，也无法对 Action.script_info_hash 的正确性进行校验。

因此 Type script 可以做如下两点假设：

• 整个 Action 的内容，包含 Action.script_info_hash ，都被 lock script 在验签过程中覆盖以确保无法篡改。
• 在生成签名的过程中，钱包应该使用正确的 ScriptInfo 对 Action.script_info_hash 进行校验，并通过 ScriptInfo.schema 正确呈现 Action.data 中的内容给持有该 cell 的用户进行了确认。

虽然 type script 没有完整的 ScriptInfo，但是 type script 应该在源码中以某种方式包含了 ScriptInfo.schema 以及 ScriptInfo.message_type 的内容。Type script 可以借助这些内容解析 Action.data , 还原出对应的输入数据，并以此对特定范围内的 cells 进行校验。例如，如果 Action.data 是一个 Spore NFT Mint 操作，type script 就要校验当前范围内的 cells 的确是一个 mint 操作；如果 Action.data 是一个 UDT transfer 操作，type script 也要进行类似的校验，确保正确数量的 UDT 被转给了正确的收款方，多余的 UDT 返还给了支付方。

[janx]

Appendix: Custom Union Ids in WitnessLayout

中文版本

In the WitnessLayout structure, each union variant has a special custom union id (a larger magic number). This is a trick used to maintain forward compatibility: such special values allow WitnessLayout and previous WitnessArgs to coexist without affecting each other.

The widely used WitnessArgs is a molecule table containing three fields. Its serialized data looks roughly like this:

<length> <offset 1> <offset 2> <offset 3> ...

Both length and offset are of little endian u32 type.

And the serialization of a molecule union looks something like this:

<union id> ....

Here, the union id is also of little endian u32 type.

For WitnessLayout, it currently uses custom union ids such as 0xFF000001, 0xFF000002. Considering future expansion, WitnessLayout should only use the range from 0xFF000001 to 0xFFFFFFFF for new variants’ union ids (this range has over 16 million values which are enough).

Why? Assuming there’s some serialized data in the format of WitnessLayout, if you read out its first four bytes as little endian u32, the result will definitely not be less than 0xFF000001. If you try to parse this data using the WitnessArgs type, these first four bytes would be read out as length for whole data. For example, 0xFF000001 == 4278190081 ~=4080MB. But CKB block size maxes at around 600k, even considering future expansion, it’s hard to reach 4000MB. Therefore, it is practically impossible to construct a WitnessArgs of length 0xFF000001 or longer. So if you parse the binary data serialized in the format of WitnessLayout as WitnessArgs, the molecule verification will definitely fail.

Similarly, for data serialized in the format of WitnessArgs, the value read from its first four bytes would be far less than 0xFF000001, and could not possibly be any union id used by WitnessLayout. Thus if you try parsing this data as WitnessLayout, molecule verification will also certainly fail.

Therefore, in transaction witnesses you can arbitrarily insert either WitnessArgs formatted or WitnessLayout formatted data. On-chain scripts can deterministically determine what format a certain witness uses. Data in WitnessLayout format will surely fail validation when parsed as WitnessArgs, and vice versa. In future we can further expand WitnessLayout to support more variants.

If we look at the encoding, the WitnessLayout union places a 4-byte ID in front of the actual witness content. This 4-byte ID works like a version flag, helps differentiate new format from existing WitnessArgs.

[janx]

附录：Custom Union Ids in WitnessLayout

在 WitnessLayout 结构中，每一个 union variant 都有一个特殊的 custom union id（一个较大的magic number）。这是一个为了保持向前兼容用的技巧：通过这样特殊的取值可以让 WitnessLayout 和之前的 WitnessArgs 互不影响实现共存.

现在广泛使用的 WitnessArgs 是一个包含三个字段的 molecule table，它序列化后的数据大致是这样子的：

<length> <offset 1> <offset 2> <offset 3> ...

其中 length 和 offset 都是 little endian 的 u32 类型。

而一个 molecule union 的序列化结构大致是如下的样子：

<union id> ....

这里的 union id 也是 little endian 的 u32 类型。

对于 WitnessLayout 来说，它目前使用的 union id 是 0xFF000001, 0xFF000002 …。考虑未来扩充，WitnessLayout 应该只使用 0xFF000001 - 0xFFFFFFFF 这个范围作为这个类型允许的 union id 范围（这个范围有超过 16 million 个值，足够使用）。

假设有一个 WitnessLayout 格式序列化后的数据，这段数据前四个字节如果作为 little endian 的 u32 读取出来肯定不小于 0xFF000001。如果尝试把这段数据以 WitnessArgs 的类型进行解析，这段数据的前四个字节会作为整段数据的长度读取出来，e.g. 0xFF000001 == 4278190081 ~= 4080MB。但是 CKB 一个 block 最大只有 600k 左右，即使考虑将来可能的扩充，也很难达到 4000MB。因此实际上不可能构造出一个长度为 0xFF000001 甚至更长的 WitnessArgs 结构。所以如果把 WitnessLayout 格式序列化后的二进制数据，以 WitnessArgs 格式来解析，molecule 验证过程一定失败。

同样的，对于以 WitnessArgs 格式序列化好的数据，前四个字节读出来的值一定远远小于 0xFF000001，不可能是 WitnessLayout 使用的 union id。所以以 WitnessArgs 格式序列化好的数据，如果以 WitnessLayout 的结构来解析，molecule 验证过程也一定会失败。

因此交易 witness 中可以任意塞入 WitnessArgs 格式或是 WitnessLayout 格式的数据。链上合约也可以确定性的判断某一个 witness 使用的是什么格式。WitnessLayout 格式的数据以 WitnessArgs 格式解析校验一定会失败，同样的，WitnessArgs 格式的数据以 WitnessLayout 格式解析校验也一定会失败。未来还可以进一步扩充 WitnessLayout 格式，支持更多的可选类型。

如果只看编码的话，这里的 WitnessLayout union，也很是像先在 witness 上放置一个 4 个 bytes 的 ID，然后再放置实际的 witness 内容。同时靠这个 ID 与现有的 WitnessArgs 结构可以区分开来。WitnessLayout 通过 custome union id 实现了类似 version flag 的功能。

[janx]

Appendix: CoBuild-Only or CoBuild+Legacy?

中文版本

There are currently two input data (witness) encoding modes for CKB scripts:

• CoBuild mode: Encode input data in the WitnessLayout format defined in this document;
• Legacy mode: Encode input data in the WitnessArgs format.

CKB transactions only care whether the included scripts and witnesses can be verified, not how these scripts parse witnesses. This means that scripts using different witness encoding mode can coexist within a transaction.

A CKB script can also support both modes: for example, such a script can first look for witness in the WitnessLayout format, and if it’s missing, it tries to find a WitnessArgs witness.

Usually when ‘CoBuild mode’ or ‘Legacy mode’ is mentioned, it refers to a certain script being in one of these modes, after deciding which input data encoding to use. At this point, this script has determined either to only read from WitnessLayout witnesses (CoBuild Mode), or only from WitnessArgs witnesses (Legacy Mode).

However when discussing “a script supports CoBuild mode” below , it means that a script is able to operate under CoBuild mode (not “only” or “already determined”) and read from WitnessLayout witnesses. Correspondingly, “a script supports Legacy mode”, means that this script is able to operate under Legacy Mode (not “only” or “already determined”) and reads from WitnessArgs witnesses.

Lock Script

In the transition period (from now until CoBuild becomes widely used as default), we do not recommend lock script supporting CoBuild mode only without supporting Legacy mode.

It’s not difficult for a lock script to support both CoBuild mode and Legacy mode. The two modes’ difference lie only on the calculation of signing message hash. A well structured lock script could reuse almost all verification codes in the two modes.

If a lock script wants to support as many CKB apps as possible, the optimal solution is to support both CoBuild and Legacy modes.

Type Script

The considerations for type script are somewhat different from those of lock script.

Without considering some extreme cases in OTX, a lock script that only supports CoBuild mode cannot be used with a type script that can only process WitnessArgs typed witnesses in the same cell.

What about the other way around? Can a type script that only supports CoBuild mode be used with a lock script (such as the default secp256k1-blake160 lock) that can only process WitnessArgs typed witnesses? The answer is yes.

Let’s look at this example:

Assume here I1 and I2 both use the secp256k1-blake160 lock, while I1 uses a type script that only supports CoBuild mode. A valid transaction can use the above layout: witness #0 and witness #1 are of the WitnessArgs type, witness #2 is of CoBuild mode’s WitnessLayout::SighashAll type which contains an Action required by the type script in I1.

According to the conventions used in legacy lock scripts, although the lock scripts used by I1 and I2 cannot parse the WitnessLayout typed witness#2, these lock scripts will still sign all data of witness#2. Meanwhile, the type script used by I1 could parse Message from witness#2, and find its corresponding Action to complete the verification.

We can see that there won’t be compatibility issues for type scripts supporting only CoBuild mode without supporting Legacy mode. For type script, you could choose either to support CoBuild mode or be compatible with both modes. Our suggestion new type scripts to consider supporting CoBuild mode first, and being compatible with Legacy mode if necessary.

Action May Not Exist

In most cases, there will be a WitnessLayout::SighashAll witness in CoBuild transactions, which contains the Message, as well as the Actions corresponding to the type scripts used. However, it should be noted that the CoBuild protocol does not require that a WitnessLayout::SighashAll must exist.

Suppose there is a type script that only supports CoBuild mode. One problem it needs to face is: Should it be able to handle situations where its own Action is missing when a WitnessLayout::SighashAll doesn’t exist?

Different type scripts can take different actions here based on their own requirements. A type script can choose whether to exit with an error directly or ignore the issue and continue verification when its required ‘Action’ does not exist.

It should be pointed out that for most scenarios, existence of an Action can greatly simplify on-chain verification. It’s usually difficult to guess the user action by reverse analyzing transaction data.

Therefore, we suggest adding this validation rule for type scripts: it must find the required Action in transaction before it can proceed.

NervosDAO Type Script

The NervosDAO type script is a special example for CoBuild process because its code was deployed into genesis block with unspendable lock (lock hash = 0x0000…, all zero). This means NervosDAO cannot be upgraded via methods other than hard fork.

If we still want the NervosDAO type script to be compatible with CoBuild without upgrading it through hard fork, additional special handling is needed. One approach is to hardcode specific logic related to NervosDAO within CoBuild toolchain. If a CoBuild tool detects a transaction involving NervosDAO, it should perform some preprocessing:

• If the transaction does not contain a NervosDAO Action, CoBuild tools should construct one using raw transaction data for UX presentation.
• If it contains a NervosDAO Action, these tools should validate the action, take the responsibility that would normally be taken by the NervosDAO type script.

Remember, during transition period (from now until CoBuild becomes widely used), we don’t recommend CKB lock scripts to only support CoBuild mode without supporting Legacy mode. However, due to the speciality of Nervos DAO type script, we suggest that for all operations involving Nervos DAO, the tools should generate transactions using WitnessArgs regardless of whether or not the lock scripts used support CoBuild. Even if for some operations the generated transaction can use CoBuild because the NervosDAO type script doesn’t need to read witness in that case, the generated transaction shouldn’t. This way we simplify implementation by the requirement of using WitnessArgs in all cases.

Then fully support for CoBuild can be added to NervosDAO type script when it is going to be upgraded for some reason in future (no plan for now).

[janx]

附录：CoBuild-Only or CoBuild+Legacy?

CKB script 现在有两种输入数据 (witness) 编码模式：

• Cobuild 模式：使用本文定义的 WitnessLayout 格式读取数据
• Legacy 模式：使用 WitnessArgs 格式读取数据

CKB 交易只关心包含的 scripts 和 witnesses 是否能验证通过，并不关心这些 scripts 是如何解析 witnesses 的。这意味着一个交易里面使用不同输入数据编码的 scripts 可以共存。

一个 CKB script 也可以做到两种模式都支持：例如优先寻找 WitnessLayout 格式读取数据，发现WitnessLayout 类型的 witness 缺失时，再从 WitnessArgs 格式中读取数据。

通常我们讨论 CoBuild 模式或者 Legacy 模式时，是指某一个 script 在已经确定了要用哪种输入数据格式后所处的某一个模式。此时这个 script 已经确定：要么只从 WitnessLayout 中读取数据（CoBuild 模式），要么只从 WitnessArgs 中读取数据（Legacy 模式）。

但是在下文中谈到 “一个 script 支持 CoBuild 模式”，是指一个 script 可以运作在 CoBuild 模式下（而不是“只能”或者“已经确定”运作在 CoBuild 模式下），从 WitnessLayout 中读取数据。与此对应的，“一个 script 支持 legacy 模式”，是指这个合约可以运作在 Legacy 模式下（而不是“只能”或者“已经确定”运行在 Legacy 模式下），从 WitnessArgs 中读取数据。

Lock Script

在过渡时期（现在直到 CoBuild 被广泛使用成为默认之前），我们不建议一个 lock script 只支持 CoBuild 模式而不支持 Legacy 模式。

Lock script 要同时支持 CoBuild 模式与 Legacy 模式并不复杂。它们的差异点只在 signing message hash 的局部计算上。除此之外，一个架构良好的 lock script 完全可以在两种模式下共享其余几乎所有的验签代码。

如果一个 lock script / 钱包希望尽可能支持尽可能多的 CKB apps，那么同时支持 CoBuild 模式以及 Legacy 模式是最优解。

Type Script

Type script 的考量点与 lock script 有一些不同。

不考虑 OTX 中一些极端情况，一个只支持 CoBuild 模式的 lock script，是无法与一个只能从 WitnessArgs 类型的 witness 中读取输入数据的 type script 在一个 cell 中一起使用的。

如果反过来呢？一个只支持 CoBuild 模式的 type script，可不可以与一个只能从 WitnessArgs 类型的 witness 中读取签名的 lock script（比如创始块中的单签多签）一起使用？答案是可以。

让我么看下面这个例子：

假设这里 I1 和 I2 都使用了创始块中的单签 lock，同时 I1 中使用了一个只支持 CoBuild 模式的 type script。一个有效的交易可以使用如上布局：witness #0 和 witness #1 使用 WitnessArgs 类型，witness #2 使用 CoBuild 模式的 WitnessLayout::SighashAll 类型，包含 I1 中 type script 所需要的 Action。

按照传统意义上 lock script 的 convention，虽然 I1 和 I2 使用的 lock script 不能解析 witness #2 所使用的 WitnessLayout 格式，但是这里的 lock script 依然会把 witness #2 的全部数据签入签名。同时，I1 所使用的 type script 可以从 witness #2 中解析 Message，并找到对应自己的 Action。完成校验。

由此可以看出，只支持 CoBuild 模式不支持 Legacy 模式的 type script 不会有兼容性问题。对于 type script 来说，可以自行选择是只支持 CoBuild 模式，还是两种模式都兼容。我们的建议是实现 type script 时默认只支持 CoBuild 模式，有特别需要再考虑兼容 Legacy 模式。

Action 可能不存在

大部分情况下，CoBuild 交易中都会有 WitnessLayout::SighashAll witness，其中包含 Message，也包含当前交易用到的 type script 所对应的 Actions。但需要注意的是，CoBuild 协议没有要求 WitnessLayout::SighashAll 必须存在。

假设有一个 type script 只支持 CoBuild 模式，它需要面对的一个问题是：它是否应该能够处理因为 WitnessLayout::SighashAll 不存在，导致当前交易中没有对应自己的 Action 的情况呢？

对于这个问题，不同的 type script 需要根据自己的需求做决定。虽然 CoBuild 协议上没有禁止 Action 不存在的情况，但是 type script 可以选择在自己需要的 Action 不存在时，是直接报错退出，还是忽略 Action 继续校验。

应当指出的是，对于大部分场景，Action 的存在可以大大简化链上校验逻辑。不是所有的场景都能通过识别交易而（容易地）反向分析出输入数据。

因此对于 type script, 我们建议增加一条校验规则：一定要能在交易中找到所需的 Action 才能继续执行。

NervosDAO Type Script

NervosDAO type script 对于 CoBuild 流程来说是一个非常特殊的例子，它的特别之处在于其 type script 代码是以一个无法花费的 lock (lock hash = 0x0000…, all zero) 部署在创世块中的，这意味着 NervosDAO type script 无法通过硬分叉之外的方法升级。

如果想要在不做硬分叉不升级 NervosDAO type script 的情况下让它能兼容 CoBuild 流程，需要额外的特殊处理。一种做法是，在支撑 CoBuild 工具中 hardcode 针对 NervosDAO 的特殊逻辑，如果发现一个交易是 Nervos DAO 交易，就先做一些预处理：

• 如果这个交易不包含 Nervos DAO Action，CoBuild 工具应该利用 raw tx data 构造一个可以用于前端呈现的 Nervos DAO Action。
• 如果这个交易包含 Nervos DAO Action，CoBuild 工具应该对这个 Nervos DAO Action 进行校验，在前端实现本应该是 Nervos DAO type script 要做的工作。

注意前面说过：在过渡时期（现在直到 CoBuild 被广泛使用成为默认之前），我们不建议一个 CKB lock script 只支持 CoBuild 模式而不支持 Legacy 模式。但由于 Nervos DAO type script 的特殊性，我们建议对于所有 Nervos DAO 操作，无论一个 lock 支持 CoBuild 模式与否，都应该只生成使用 WitnessArgs 类型的交易。即使有部分操作因为 Nervos DAO 本身不需要读取 witness，可以用 CoBuild 来做，也不应该用 CoBuild 来做，这样把所有情况都统一成用 WitnessArgs，简化实现。

在这个基础上，等未来某一天 Nervos DAO type script 由于某个原因升级时（目前没有明确计划），再通过升级 Nervos DAO type script，实现对 CoBuild 的完全支持。

[boc]

能不能在github里备份一份，论坛里有好多这种高质量的design（草稿），不放github以后不好找

[orange-xc]

Can Action add a new option? In addition to constraining type_script through script_hash, you can also set code_hash, hash_type, and args. Then lock will be responsible for calling the corresponding script through dynamic linking or spawn, checking, and ensuring that the return value is correct. I’ve thought of some usage scenarios.

Action能不能新增一种选项，除了通过script_hash约束type_script以外，还可以设置 code_hash, hash_type, args，然后lock会负责通过动态链接或者spawn来调用对应的脚本，进行检查，同时确保返回值正确。我已经思考到了一些使用场景。

[xxuejie]

Personally I feel it’s better to include a special Action using lock’s own script hash as script_hash, then the data of this particular Action can contain code_hash, hash_type or args as you wish.

When the very lock script captures the designated Action, it can then execute passed code_hash / hash_type as it wishes, via exec / spawn / dynamic linking or whatever solution it defines.

My concern is that it could be very hard to reach a consensus so every lock script wants to do dynamic linking / spawn / exec. It will bring a completely different set of security assumptions. So I would consider it makes more sense to introduce this first on per-lock-script basis, later if this is really becoming a popular design, cobuild can then introduce it in a V2 design.

[orange-xc]

Using lock’s own script hash as script_hash is inconvenient, it’s make the code invoked by dynamic linking, exec or spawn hard to find which otx called it.

Compared to binding verification to a specific cell’s type script, using exec allows for a more abstract expression of the user’s intention. The invoked code then checks the entire transaction to assess whether the user’s intention has been met, without needing to specify which component fulfills this intent.

[phroi]

@xxuejie @janx can you confirm or deny the following statement?

CoBuild validation rules require all Action object to have matching type script in the fully assembled CKB Transaction, so a Type Lock is required for validating Lock Script Actions by CoBuild, especially in a OTX environment.

[xxuejie]

Actually we’ve relaxed this restriction(tho I was on vacation in the past couple of days so never got a chance to edit): an Action object can have either a matching type script, or a lock script. So a lock script can have Action object, there is no need to introduce a type script when not needed.

[phroi]

CoBuild is a way of representing the intentions of transitions happening within a TX / OTX. But let’s say my interface get wrong the CKB change cell by 1M CKB, there is no indication that 1M CKB is up for grabs (either going to the miner or to someone who submit an OTX spending that 1M).

Then again there may be use-cases where a OTX may intentionally leave this unbalanced, for example limit orders. An OTX may have a negative delta of CKB (- 1000 CKB) and positive delta of a UDT (+990 iCKB UDT). This would be a sell order of 990 iCKB UDT for 1000 CKB.

More commonly, exists a small positive CKB delta for paying the miner/aggregator fee.

@xxuejie is there a standard way to represent in CoBuild these phantom/implied actions? Would it be considered a good practice?

[phroi]

@janx

Note that the validation is against “a range of cells”: due to the possible existence of otx , this range may not necessarily be the entire tx but could be intervals like [3,6), or the union of two intervals such as [0,4) || [8,11). However there will only ever be union of two ranges maximum; never more than that.

I didn’t understand the reasoning here, could you explain a bit more?

[xxuejie]

The flow of CKB is not something we want to express using cobuild Messages. A wallet is expected to visualize the cobuild message to users(which would include the flow of UDT tokens as well as other contract defined information), the wallet is also expected to visualize the basic flow of CKB tokens, L1 tx fee, and any other information defined in the consensus level of CKBs.

So I personally think a wallet should warn users that 1M CKB is up for grabs, this is not something we want to do in cobuild messages.

[phroi]

You may want to document this implicit requirement for wallet developers in the CoBuild proposal, otherwise they may not be aware that this is expected of them.

By the way, where does happen the validation that all script_hash of message actions are indeed included in the transaction? Wallet/DApp level or is there a planned softfork for direct support of CoBuild?

Wallet or on-chain validation would change the security assumptions when developing scripts, as a CoBuild soft-fork would guarantee that full on-chain message validation always occurs, while wallet validation can be bypassed by an attacker using a customized front-end.

[orange-xc]

@phroi

By the way, where does happen the validation that all script_hash of message actions are indeed included in the transaction?

Lockscript (omnilock) will validate it. Why frontend can bypass this validation?

[xxuejie]

Cobuild does not need any hardforks or softforks. Enough work has been done to ensure cobuild stays compatible and secure with existing conventions. The adoption of cobuild, to me, will be a gruadual process as more locks and types start to use it.

That being said, I do want to point out that CKB is a permissionless platform. Theorectically one can ignore all the cobuild process, one can even ignore the whole sighash-all process and roll out one new stack here. There is inherently nothing against it, nor can we say “you must not do it”. That’s the merit and the curse of permissionless systems. Even when we introduced cobuild here, the best we can say is “okay we have thought hard on this problem, and we believe if enough players pay attention to this cobuild protocol, together we can have an ecosystem that brings us interoperability powers”, after that we can only hope for the best. All things discussed here, might never be requirements, but only conventions, or recommendations.

[phroi]

@janx

c. Parse otx.message, check each Action in the message, the script_hash in Action must either be the same as the type script hash of one input/output cells in transaction, or be the same as the lock script hash of one input cells in transaction. Otherwise, the script shall terminate with an error.

I crucially missed this, (in case the tx include in input at least one CoBuild compatible lock) you are indeed correct, thanks!

[phroi]

CoBuild enables protocols to formalize user intentions, but defines no standard to describe how a set of intentions is transformed into a tx/otx, so a standard for defining a function f(intentions, l1_state, tx_pool, ...) -> tx/otx. I imagine this step similarly to how a molecule schema can be translated into an executable.

Of course this is a much more complex problem, then again this second formalization step would truly enable an EVM-alike developer experience with the security of a Cell Model.

Have any thoughts been given on this? If so, what’s its progress status?