Displaying raw, ABI-encoded calldata has been a drag on web3 user experience for years. It should be obvious that most users cannot read blocks of hex data; I wrote about this in early 2021 when GridPlus introduced a new contract readability feature. We have recently replaced this with a “just-in-time” calldata decoder, which takes advantage of self-validation in the ABI spec. With this new approach, transaction requests may include “decoder data”, which is used to mark down the calldata in a more readable way in the same request. This means users no longer need to pre-load ABI definitions, which is a significant UX improvement. It also saves a good bit of code space on our secure microcontroller, which always makes me happy.

Being more of an engineer than a theoretician, I designed this decoder-data encoding protocol by trial and error so I cannot guarantee its correctness. However, I think the design is pretty good and figured it might be useful for other wallet teams to see.

Basically, the goal is to generate a small piece of data that can be included with transaction calldata to display much more human readable information on a signing screen. Because naming things is fun, I will term this Calldata Decoder Data (CDD) and will soon outline a protocol to encode it.

As many readers will know, Ethereum and all other EVM chains build contract function calldata with a bespoke ABI encoding protocol. This protocol is interrelated with a much simpler, separate protocol which is used to build strings representing the function’s name and parameter set (a.k.a. “function signature”). Since this latter protocol doesn’t seem to have a name, I will call it “FSB” for “function signature builder”. I will also refer to a “function signature” as fSig from now on.

NOTE: The ABI docs do kind of mention the FSB, though I found the specification underwhelming and will attempt to elaborate below. For posterity, the official “spec” says: “The signature is defined as the canonical expression of the basic prototype without data location specifier, i.e. the function name with the parenthesised list of parameter types. Parameter types are split by a single comma - no spaces are used.”

FSB and ABI are closely related because:

1. FSB uses [a subset of] ABI types.

2. The FSB output is an input to the ABI-encoded calldata.

FSB builds a string (fSig) to represent the function using the following serialization:

"${functionName}(${param0Type},…,${paramNType})"

This results in something like myFunction(address,bytes32,uint256). Note the absence of spaces!

It is important to note that only a subset of ABI types can be used in building fSigs. These are called “canonical types”. To be fair, it is a very large subset, but there are a few exceptions that require conversion from their loosey-goosey Solidity types:

• int → int256

• uint → uint256

• fixed → fixed128x18

• ufixed → ufixed128x18

• function → bytes24

• byte → bytes1

NOTE: I have never used fixed or ufixed types and have never seen them in the wild, so they are not part of the GridPlus CDD encoding protocol that I will present shortly, though they could be added in a future version.

Once you have the fSig string constructed, you need to hash it with keccak256 and take the first four bytes of that hash. This is called the “function selector” and it prefixes EVM transaction calldata. Simply put, the function selector is the first four bytes of transaction calldata; the rest is param data serialized using ABI. Therefore, the only purpose of FSB is to build that function selector.

This is an important relationship because it means EVM calldata is, in a sense, self-validating. You cannot construct a function selector of e.g. myFunction(address,bool) and get away with throwing a dynamic bytes buffer into the calldata - that will not pass network consensus! It also gives wallets a lot of useful sanity checks, such as making sure that bool param value is not some large integer in the calldata. Of course there are situations where these checks do not hold, but they are still, on the whole, pretty helpful.

Now yes yes, I am aware that 4 bytes is not very large; it means there is a 0.0000000023% chance of collision on any two random fSigs. But it’s big enough to still be very useful in practice and since the param structure in calldata always needs to match the fSig, attacks in this domain are pretty limited (they generally require deploying a separate contract and changing the function name to some random colliding value).

NOTE: Lattice users are reminded to always use address tags with high value contract interactions. A full user-based sanity check would involve validating the address, function name, and param order/values. Tags make this much easier.

We seek to construct a minimal piece of data such that we can rebuild a transaction’s fSig and decode calldata parameters into individual values for better readability on the wallet interface (e.g. a Lattice1 secure screen).

At a high level, we need to find each individual parameter and describe it in the context of all other parameters. We call the individual parameter “atomic” because it cannot be further reduced. Don’t worry, this will be clearer with examples.

Each canonical param type (e.g. uint256, bytes8, etc) can be encoded with a four-item array descriptor containing:

[
  paramName,
  paramTypeIdx,
  size,
  arraySizes
]

• paramName is a string representing the parameter name. Note that this is [annoyingly] not defined in the fSig, so its usage is left to the wallet (and integrations). Speaking for GridPlus: when we only have an fSig, we use “#1”, “#2”, etc to name the parameters because, well, we don’t know what they’re called. If instead we are fetching a Solidity JSON ABI (which is what you get from Etherscan’s API), we use those param names here.

• paramTypeIdx is an enum value (uint8 type) based on the following set of ABI-ish types: [address, bool, uint, int, bytes, string, tuple]. We use these as basic types that can be expanded to build the canonical type, if necessary. address, bool, and string are canonical types already, as is bytes in the case of a dynamic buffer type (as opposed to a fixed buffer, e.g. bytes32). tuple is a “meta type” that will be discussed in the next section.

• size is a uint8 type used to further specify the canonical type if the paramTypeIdx enum value maps to uint, int, or bytes.

If a parameter can be described using only these four values, we call it an “atomic parameter”. Some types (currently only tuple) cannot be described atomically and require additional rules.

Atomic parameters are the basic building blocks of CDD but they are not sufficient because sometimes they must be nested, for example with tuple types. Fortunately, the solution for building non-atomic descriptors is pretty simple: recursively fetch atomic descriptors and concatenate them.

Take the following fSig params: (uint256,(bool,address)). This produces two descriptors:

• uint256 → [“#1”, 2, 32, []] (atomic)

• (bool,address) → [“#2”, 6, 0, []] (not atomic)

The use of enum value 6 indicates this is a tuple, so when building the CDD we would need to recurse until we describe all nested atomic params. For this example, we would build the following nested atomic descriptors:

• bool → [“#2-1”, 1, 0, []]

• address → [“#2-2”, 0, 0, []]

These would be concatenated to the tuple’s descriptor like so:

[  ["#1", 2, 4, []],
  [    "#2", 6, 0, [],
    ["#2-1", 1, 0, []],
    ["#2-2", 0, 0, []]
  ]
]

If there were additional tuples after we recursed, we would need to keep recursing. For example, fSig params ((bool, address),(bool,(bytes8[],bytes)[2],bool) would lead to the following (unserialized) CDD:

[  [    "#1", 6, 0, [],
    ["#1-1", 1, 0, []],
    ["#1-2", 0, 0, []]
  ],
  [    "#2", 6, 0, [],
    ["#2-1", 1, 0, []],
    [      "#2-2", 6, 0, [2],
      ["#2-2-1", 4, 1, [0]],
      ["#2-2-2", 4, 0, []]
    ],
    ["#2-3", 1, 0, []]
  ]
]

Because Ethereum RLP is an efficient and widely used protocol for EVM things, we use that for serializing. The result will contain all the information a decoder might need to deserialize and digest the definitions.

The function name is concatenated with the param set to produce the full CDD. For example, the fSig myFunction(uint256,bool[2]) produces full CDD:

[  "myFunction",  [    ["#1", 2, 32, []],
    ["#2", 1, 0, [2]]
  ]
]

which RLP serializes into: 0xdb8a6d7946756e6374696f6ecfc68223310220c0c78223320180c102

The serialized CDD above can be used to reconstruct the fSig and function selector. Using the same example we might see a transaction with calldata:

A wallet would receive an incoming request with both calldata and serialized CDD. The latter can be RLP-deserialized and the fSig can be reconstituted into a string using some logic that is outside of the scope of this article. The fSig is now hashed:

> keccak256(myFunction(uint256,bool[2]))
'91061af786aadc13d8e123a127b60be62170486ce5e1ba89bdff34d5be95bbb4'

The first four bytes of this should match the first four bytes of the transaction calldata. If they do, we have validated the parameter types and function name and can safely decode the calldata with our CDD.

Again, this validation does not extend to param names, since they are not covered in the FSB protocol. 🙄 So display of param names is left up to both the wallet and requester/integration.

One nice thing about our CDD encoding protocol is that the data is relatively small, largely thanks to RLP. I pulled ~10,000 fSigs from 4byte and built CDD data from each one (these use the param names “#1”, “#2”, etc). The average size is 32.18 bytes and very few signatures are >50 bytes. This makes the CDD protocol useful in a constrained memory environment such as Lattice firmware.

There were of course a few outliers (>800 bytes 😳), but anything >150 bytes is extremely uncommon.

A list of fSigs that are decodable by current Lattice firmware (v0.15.0) can be found here. There are also two parsers that may be useful for reference: fSig and Solidity JSON-ABI. There are a few notable limitations related to GridPlus’ implementation:

• We do not use fixed128x18 or ufixed128x18 types because I’ve never seen them in the wild and did not want to add the complexity to our pure-C decoder lib for them. If demand exists in the future I will work on incorporating these types into the enum.

• There remain some “extreme” edge cases that still do not decode, though I believe this is related to constraints in the C decoder implementation (FYI writing this stuff in C is not fun). These edge cases are related to multi-layered tuples that contain arrays. Needless to say, these situations are uncommon - not sure if anyone will ever need something like nestedTup(((bytes)[2],bool)), but if they do we cannot support it yet.

• Param names cannot be validated just-in-time (i.e. in the same request), though this is not related to GridPlus’ implementation specifically.

That said, the CDD decoding protocol can easily be extended and rewritten in a higher level language such a TypeScript, which would probably be more robust.

Hopefully this was useful, or at least interesting.✌️

Displaying raw, ABI-encoded calldata has been a drag on web3 user experience for years. It should be obvious that most users cannot read blocks of hex data; I wrote about this in early 2021 when GridPlus introduced a new contract readability feature. We have recently replaced this with a “just-in-time” calldata decoder, which takes advantage of self-validation in the ABI spec. With this new approach, transaction requests may include “decoder data”, which is used to mark down the calldata in a more readable way in the same request. This means users no longer need to pre-load ABI definitions, which is a significant UX improvement. It also saves a good bit of code space on our secure microcontroller, which always makes me happy.

Being more of an engineer than a theoretician, I designed this decoder-data encoding protocol by trial and error so I cannot guarantee its correctness. However, I think the design is pretty good and figured it might be useful for other wallet teams to see.

Basically, the goal is to generate a small piece of data that can be included with transaction calldata to display much more human readable information on a signing screen. Because naming things is fun, I will term this Calldata Decoder Data (CDD) and will soon outline a protocol to encode it.

As many readers will know, Ethereum and all other EVM chains build contract function calldata with a bespoke ABI encoding protocol. This protocol is interrelated with a much simpler, separate protocol which is used to build strings representing the function’s name and parameter set (a.k.a. “function signature”). Since this latter protocol doesn’t seem to have a name, I will call it “FSB” for “function signature builder”. I will also refer to a “function signature” as fSig from now on.

NOTE: The ABI docs do kind of mention the FSB, though I found the specification underwhelming and will attempt to elaborate below. For posterity, the official “spec” says: “The signature is defined as the canonical expression of the basic prototype without data location specifier, i.e. the function name with the parenthesised list of parameter types. Parameter types are split by a single comma - no spaces are used.”

FSB and ABI are closely related because:

1. FSB uses [a subset of] ABI types.

2. The FSB output is an input to the ABI-encoded calldata.

FSB builds a string (fSig) to represent the function using the following serialization:

"${functionName}(${param0Type},…,${paramNType})"

This results in something like myFunction(address,bytes32,uint256). Note the absence of spaces!

It is important to note that only a subset of ABI types can be used in building fSigs. These are called “canonical types”. To be fair, it is a very large subset, but there are a few exceptions that require conversion from their loosey-goosey Solidity types:

• int → int256

• uint → uint256

• fixed → fixed128x18

• ufixed → ufixed128x18

• function → bytes24

• byte → bytes1

NOTE: I have never used fixed or ufixed types and have never seen them in the wild, so they are not part of the GridPlus CDD encoding protocol that I will present shortly, though they could be added in a future version.

Once you have the fSig string constructed, you need to hash it with keccak256 and take the first four bytes of that hash. This is called the “function selector” and it prefixes EVM transaction calldata. Simply put, the function selector is the first four bytes of transaction calldata; the rest is param data serialized using ABI. Therefore, the only purpose of FSB is to build that function selector.

This is an important relationship because it means EVM calldata is, in a sense, self-validating. You cannot construct a function selector of e.g. myFunction(address,bool) and get away with throwing a dynamic bytes buffer into the calldata - that will not pass network consensus! It also gives wallets a lot of useful sanity checks, such as making sure that bool param value is not some large integer in the calldata. Of course there are situations where these checks do not hold, but they are still, on the whole, pretty helpful.

Now yes yes, I am aware that 4 bytes is not very large; it means there is a 0.0000000023% chance of collision on any two random fSigs. But it’s big enough to still be very useful in practice and since the param structure in calldata always needs to match the fSig, attacks in this domain are pretty limited (they generally require deploying a separate contract and changing the function name to some random colliding value).

NOTE: Lattice users are reminded to always use address tags with high value contract interactions. A full user-based sanity check would involve validating the address, function name, and param order/values. Tags make this much easier.

We seek to construct a minimal piece of data such that we can rebuild a transaction’s fSig and decode calldata parameters into individual values for better readability on the wallet interface (e.g. a Lattice1 secure screen).

At a high level, we need to find each individual parameter and describe it in the context of all other parameters. We call the individual parameter “atomic” because it cannot be further reduced. Don’t worry, this will be clearer with examples.

Each canonical param type (e.g. uint256, bytes8, etc) can be encoded with a four-item array descriptor containing:

[
  paramName,
  paramTypeIdx,
  size,
  arraySizes
]

• paramName is a string representing the parameter name. Note that this is [annoyingly] not defined in the fSig, so its usage is left to the wallet (and integrations). Speaking for GridPlus: when we only have an fSig, we use “#1”, “#2”, etc to name the parameters because, well, we don’t know what they’re called. If instead we are fetching a Solidity JSON ABI (which is what you get from Etherscan’s API), we use those param names here.

• paramTypeIdx is an enum value (uint8 type) based on the following set of ABI-ish types: [address, bool, uint, int, bytes, string, tuple]. We use these as basic types that can be expanded to build the canonical type, if necessary. address, bool, and string are canonical types already, as is bytes in the case of a dynamic buffer type (as opposed to a fixed buffer, e.g. bytes32). tuple is a “meta type” that will be discussed in the next section.

• size is a uint8 type used to further specify the canonical type if the paramTypeIdx enum value maps to uint, int, or bytes.

If a parameter can be described using only these four values, we call it an “atomic parameter”. Some types (currently only tuple) cannot be described atomically and require additional rules.

Atomic parameters are the basic building blocks of CDD but they are not sufficient because sometimes they must be nested, for example with tuple types. Fortunately, the solution for building non-atomic descriptors is pretty simple: recursively fetch atomic descriptors and concatenate them.

Take the following fSig params: (uint256,(bool,address)). This produces two descriptors:

• uint256 → [“#1”, 2, 32, []] (atomic)

• (bool,address) → [“#2”, 6, 0, []] (not atomic)

The use of enum value 6 indicates this is a tuple, so when building the CDD we would need to recurse until we describe all nested atomic params. For this example, we would build the following nested atomic descriptors:

• bool → [“#2-1”, 1, 0, []]

• address → [“#2-2”, 0, 0, []]

These would be concatenated to the tuple’s descriptor like so:

[  ["#1", 2, 4, []],
  [    "#2", 6, 0, [],
    ["#2-1", 1, 0, []],
    ["#2-2", 0, 0, []]
  ]
]

If there were additional tuples after we recursed, we would need to keep recursing. For example, fSig params ((bool, address),(bool,(bytes8[],bytes)[2],bool) would lead to the following (unserialized) CDD:

[  [    "#1", 6, 0, [],
    ["#1-1", 1, 0, []],
    ["#1-2", 0, 0, []]
  ],
  [    "#2", 6, 0, [],
    ["#2-1", 1, 0, []],
    [      "#2-2", 6, 0, [2],
      ["#2-2-1", 4, 1, [0]],
      ["#2-2-2", 4, 0, []]
    ],
    ["#2-3", 1, 0, []]
  ]
]

Because Ethereum RLP is an efficient and widely used protocol for EVM things, we use that for serializing. The result will contain all the information a decoder might need to deserialize and digest the definitions.

The function name is concatenated with the param set to produce the full CDD. For example, the fSig myFunction(uint256,bool[2]) produces full CDD:

[  "myFunction",  [    ["#1", 2, 32, []],
    ["#2", 1, 0, [2]]
  ]
]

which RLP serializes into: 0xdb8a6d7946756e6374696f6ecfc68223310220c0c78223320180c102

The serialized CDD above can be used to reconstruct the fSig and function selector. Using the same example we might see a transaction with calldata:

A wallet would receive an incoming request with both calldata and serialized CDD. The latter can be RLP-deserialized and the fSig can be reconstituted into a string using some logic that is outside of the scope of this article. The fSig is now hashed:

> keccak256(myFunction(uint256,bool[2]))
'91061af786aadc13d8e123a127b60be62170486ce5e1ba89bdff34d5be95bbb4'

The first four bytes of this should match the first four bytes of the transaction calldata. If they do, we have validated the parameter types and function name and can safely decode the calldata with our CDD.

Again, this validation does not extend to param names, since they are not covered in the FSB protocol. 🙄 So display of param names is left up to both the wallet and requester/integration.

One nice thing about our CDD encoding protocol is that the data is relatively small, largely thanks to RLP. I pulled ~10,000 fSigs from 4byte and built CDD data from each one (these use the param names “#1”, “#2”, etc). The average size is 32.18 bytes and very few signatures are >50 bytes. This makes the CDD protocol useful in a constrained memory environment such as Lattice firmware.

There were of course a few outliers (>800 bytes 😳), but anything >150 bytes is extremely uncommon.

A list of fSigs that are decodable by current Lattice firmware (v0.15.0) can be found here. There are also two parsers that may be useful for reference: fSig and Solidity JSON-ABI. There are a few notable limitations related to GridPlus’ implementation:

• We do not use fixed128x18 or ufixed128x18 types because I’ve never seen them in the wild and did not want to add the complexity to our pure-C decoder lib for them. If demand exists in the future I will work on incorporating these types into the enum.

• There remain some “extreme” edge cases that still do not decode, though I believe this is related to constraints in the C decoder implementation (FYI writing this stuff in C is not fun). These edge cases are related to multi-layered tuples that contain arrays. Needless to say, these situations are uncommon - not sure if anyone will ever need something like nestedTup(((bytes)[2],bool)), but if they do we cannot support it yet.

• Param names cannot be validated just-in-time (i.e. in the same request), though this is not related to GridPlus’ implementation specifically.

That said, the CDD decoding protocol can easily be extended and rewritten in a higher level language such a TypeScript, which would probably be more robust.

Hopefully this was useful, or at least interesting.✌️