How to: Ethereum Blobs

Article written by: Trevor Lim Yong Guan

Introduction:

Ethereum is a blockchain ecosystem widely used to power decentralised applications as it offers smart contracts. As a result of increasing use of Ethereum, the pioneer of decentralised applications has been experiencing scalability issues which increases the gas fees per transaction due to the increased computing power and resources required to verify a single block (David, 2023). With numerous transactions taking place per day (in the millions), there is no doubt that expensive gas fees will only be further exacerbated causing widespread unpopularity should the issue continue any further. Because of this, ‘Ethereum Blobs’ which work hand in hand with layer 2 solutions like Optimism will help solve this issue.

Ethereum blobs are essentially a new kind of data structure that is a collection of transactions sitting in a temporary piece of memory in cyberspace that can be verified at once; instead of verifying one block at a time (which contains one transaction), the added protocol for this: EIP-4844 will only have to verify the blob to be authentic once, unlike EIP-1559 that conventionally verifies a single block at a time and requiring additional fee payment on top of the fixed gas fee if users wish for their transactions to be prioritised first (Beck, 2021; David, 2023; EIP-4844, 2022). It then becomes clear that lesser resources are required for this since a group of transactions are being verified simultaneously, reducing the gas fees. Furthermore, Ethereum blobs being in layer 2 will ensure gas fees remain low despite peak congestion occurring since these blobs are outside of Ethereum’s layer 1 where the main bulk of transactions take place (David, 2023). This gives more leeway for transactions to be moved out of the action and put into a blob where it is verified with a bunch of other transactions in layer 2. Hence, this improves scalability without compromising the concept of blockchain: impenetrable security and decentralisation.

Need a simple analogy? Imagine a bunch of transactions flying around in the air (layer 1 Ethereum blockchain). It’s congested and there are police robots (EIP-1559) catching these transactions and checking if they are valid. Because of their hard work, they charge expensive gas fees to the people carrying out these transactions (imagine having to hand pick millions of transactions and verify them everyday!). To make life easier for both the people and police, a bunch of transactions are captured with a net (blob) and thrown up higher into the sky where it is less congested (Layer 2 e.g., Optimism). Police robots (EIP-4844) of a different variant analyse these nets/blobs and validates them. These police robots are pleased to see that they are validating multiple transactions at one go and, for the ease of their work, they charge people less fees. Back in the congested air, the number of transactions flying around begin to decrease as more are captured by nets and thrown up higher into the sky. Work becomes easier for the EIP-1559 police so they charge less gas fees too (in reality, layer 2 Ethereum users tend to experience the benefits more explicitly though, this will change as more blockchain applications begin to integrate their smart contracts to handle blobs hence, more blobs will be used, and hypothetically, that would decrease the number of transactions taking place in layer 1.)

Before continuing any further, let’s clear the air revolving around Optimism: Optimism is a type of scaling solution in L2 that assumes all transactions in a bundle/rollup is valid. The bundle can only be removed or labelled as invalid if anyone can provide evidence proving so. Hence, this reduces gas fees as less computing power and resources are required to verify this rollup of transactions since it ‘Optimistically’ assumes the transactions are true until proven otherwise (Boogaard, n.d.).

Github repository for Ethereum EIPs:

https://github.com/ethereum/EIPs

Setting up The Development Environment:

General requirements (How to install necessary dependencies, tools, and libraries):

Programming language/s: Python or JavaScript.
Libraries like web3.py (for python) and ethers.js (for JavaScript).
Ethereum client/s: Geth, Nethermind, etc. for running private chains or testnets (To test if your application is working properly with blobs).
Node.js to manage JavaScript-based tooling.
An IDE (Integrated Development Environment) like Visual Studio code.
Setup (Code written in Bash):
- Install Node.js
- Ethereum client setup
```
geth --syncmode "light"
```
- Install web3.py or ethers.js
(Ake, 2024)

1. Step-by-step development guide:

Step 1: Clone an open source rollup from Github like Optimism

Rollups are layer 2 (L2) solutions which take transactions from Ethereum’s layer 1 (L1) and bundles them together then sends the bundle back to L1 (Nightingale, 2024).
Optimism Github repository for you to clone:

https://github.com/ethereum-optimism/optimism

Step 2: dApp integration

Modify your dApp’s smart contracts to handle blob transactions. This can be done using ethers.js or web3.js libraries (See the snippet below).

2. Detailed Steps

Writing a Smart Contract that handles Blobs.

Define Storage for Blobs: Use Ethereum Virtual Machine (EVM) primitives to reference blobs.

Integrate L2 Blob APIs: Leverage APIs from your rollup of choice to store and retrieve blob data.

(Written in Solidity)

pragma solidity ^0.8.0;  

contract BlobStorage {  
    mapping(uint256 => bytes32) public blobs;  

    function storeBlob(uint256 id, bytes32 data) public {  
        blobs[id] = data;  
    }  
}

(Ake, 2024)

3. One Task at a Time: Working with Blobs:

Note that all code below was written in Python by Ake (2024)

Step 1: Create blob data

We will be sending a blob named ‘Chainstack’:

import os
from eth_abi import abi

def create_blob_data(text):
    # Encode the text using Ethereum ABI encoding for a string
    encoded_text = abi.encode(["string"], [text])
    
    # Calculate the required padding to make the blob size exactly 131072 bytes or 128 KB
    required_padding = 131072 - (len(encoded_text) % 131072)
    
    # Create the BLOB_DATA with the correct padding
    BLOB_DATA = (b"\x00" * required_padding) + encoded_text
    
    return BLOB_DATA

def main():
    text = "Chainstack" # If you change this, make sure you update the padding
    
    # Create blob data
    blob_data = create_blob_data(text)
    
    # Print the blob data in hexadecimal format
    print("Blob Data (Hex):")
    print(blob_data.hex())

if __name__ == "__main__":
    main()

You should redirect the code’s output to a file since it will be large in size (code written in Shell).
```
python create_blob_data.py > blob.txt
```

Step 2: Sending type-3 transactions

This means Putting the blob on chain:

import os
from eth_abi import abi
from eth_account import Account
from eth_utils import to_hex
from web3 import Web3, HTTPProvider
from dotenv import load_dotenv

load_dotenv()

w3 = Web3(HTTPProvider(os.getenv("EXECUTION_LAYER_URL")))

"""
Add the web3py middleware to ensure compatibility with Erigon 2
as the eth_estimateGas only expects one argument. The block number or
block hash was dropped in https://github.com/ledgerwatch/erigon/releases/tag/v2.60.1
"""
def erigon_compatibility_middleware(make_request, w3):
    def middleware(method, params):
        if method == 'eth_estimateGas' and len(params) > 1:
            # Modify the params to include only the transaction object
            params = params[:1]
        return make_request(method, params)
    return middleware
w3.middleware_onion.add(erigon_compatibility_middleware)

text = "Chainstack"
encoded_text = abi.encode(["string"], [text])

# Calculate the required padding to make the blob size exactly 131072 bytes
required_padding = 131072 - (len(encoded_text) % 131072)

# Create the BLOB_DATA with the correct padding
BLOB_DATA = (b"\x00" * required_padding) + encoded_text

pkey = os.environ.get("PRIVATE_KEY")
acct = w3.eth.account.from_key(pkey)

tx = {
    "type": 3, # Type-3 transaction
    "chainId": 11155111,  # Sepolia 11155111; Holesky 17000
    "from": acct.address,
    "to": "0x0000000000000000000000000000000000000000", # Does not matter what account you send it to
    "value": 0,
    "maxFeePerGas": 10**12,
    "maxPriorityFeePerGas": 10**12,
    "maxFeePerBlobGas": to_hex(10**12), # Note the new type-3 parameter for blobs
    "nonce": w3.eth.get_transaction_count(acct.address),
}

# Now you can estimate gas as usual
gas_estimate = w3.eth.estimate_gas(tx)
tx["gas"] = gas_estimate

# Proceed with the rest of your script
signed = acct.sign_transaction(tx, blobs=[BLOB_DATA])
tx_hash = w3.eth.send_raw_transaction(signed.raw_transaction)
tx_receipt = w3.eth.wait_for_transaction_receipt(tx_hash)
print(f"TX receipt: {tx_receipt}")

Link to transaction:

https://sepolia.etherscan.io/tx/0x5a74bd72aeeb99e874e58b927f9a5c96665278a36b61bed69a4b09597b02edce

Step 3: Retrieve Blob data

Fetching data about blob (Written in shell):

curl --request GET \
     --url CONSENSUS_LAYER_URL/eth/v1/beacon/blob_sidecars/5203463 \
     --header 'accept: application/json' | jq "."

Step 4: Computing KZG commitment

This means creating a hash value for the blob which will be used to check if the blob is identical to the blob computed by the node, hence, ensuring security as it helps prove if the blob was tampered with.

import ckzg

def bytes_from_hex(hexstring):
    return bytes.fromhex(hexstring.replace("0x", ""))

if __name__ == "__main__":
    ts = ckzg.load_trusted_setup("trusted_setup.txt")
    
    with open("blob.txt", "r") as file:
        blob_hex = file.read().strip()
        blob = bytes_from_hex(blob_hex)
    
    # Compute KZG commitment
    commitment = ckzg.blob_to_kzg_commitment(blob, ts)
    
    # Print the commitment in hexadecimal format
    print("KZG Commitment:", commitment.hex())

Step 5: Hashing the KZG commitment (Blob versioned hash)

This will add an additional layer of security by hashing the hash (Wow!):

import hashlib

# Given KZG commitment
kzg_commitment = "9493a713dd89eb7fe295efd62545bb93bca395a84d18ecfa2c6c650cddc844ad4c1935cbe7d6830967df9d33c5a2e230"

# Remove the '0x' prefix if present
if kzg_commitment.startswith("0x"):
    kzg_commitment = kzg_commitment[2:]

# Convert the KZG commitment to bytes
kzg_commitment_bytes = bytes.fromhex(kzg_commitment)

# Compute the SHA-256 hash of the KZG commitment
sha256_hash = hashlib.sha256(kzg_commitment_bytes).digest()

# Prepend the version byte (0x01) to the last 31 bytes of the SHA-256 hash
version_byte = b'\x01'
blob_versioned_hash = version_byte + sha256_hash[1:]

# Convert to hexadecimal for display
blob_versioned_hash_hex = blob_versioned_hash.hex()

# Print the result
print(f"Blob versioned hash: 0x{blob_versioned_hash_hex}")

Note that the blob versioned hash was done in the execution layer so it can be retrieved by doing a ‘Get transaction by hash’, refer here
Viewing the transaction on Sepolia (Written in Shell):

curl --request POST \
     --url EXECUTION_LAYER_URL \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "eth_getTransactionByHash",
  "params": [
    "0x5a74bd72aeeb99e874e58b927f9a5c96665278a36b61bed69a4b09597b02edce"
  ]
}' | jq '.result.blobVersionedHashes'

Step 6: Find all type 3 transactions in a block

This means finding the blob and data inside it.
Ensure that your downloaded web3.py library has the latest updates

import os
from web3 import Web3
from web3 import Web3, HTTPProvider
from dotenv import load_dotenv

load_dotenv()

w3 = Web3(HTTPProvider(os.getenv("EXECUTION_LAYER_URL")))

# Specify the block number you want to check
block_number = 6090748
block = w3.eth.get_block(block_number, full_transactions=True)

# Iterate through transactions and check for type-3 transactions
for tx in block.transactions:
    if tx.type == 3:  # Type 3 refers to blob transactions
        print("Transaction Hash:", tx.hash.hex())

Finally…

Here is all the code written together for your convenience (carries out all procedures mentioned above).

import os
import requests
from web3 import Web3, HTTPProvider
from dotenv import load_dotenv
import ckzg
import hashlib

load_dotenv()

# Connect to the Ethereum Execution Layer
w3 = Web3(HTTPProvider(os.getenv("EXECUTION_LAYER_URL")))

# Specify the block number you want to check
block_number = 6090748
block = w3.eth.get_block(block_number, full_transactions=True)

# Find type-3 transactions
type_3_tx_hashes = [tx.hash.hex() for tx in block.transactions if tx.type == 3]

# Store blob versioned hashes in a dictionary
blob_versioned_hashes_dict = {}
for tx_hash in type_3_tx_hashes:
    tx_details = w3.eth.get_transaction(tx_hash)
    blob_versioned_hashes = tx_details.get('blobVersionedHashes', [])
    if blob_versioned_hashes:
        blob_versioned_hashes_dict[tx_hash] = blob_versioned_hashes[0].hex()

# Extract the parentBeaconBlockRoot from the block data
parent_beacon_block_root = block['parentBeaconBlockRoot']

# Convert byte string to hexadecimal string
parent_beacon_block_root_hex = parent_beacon_block_root.hex()

# Ensure it starts with '0x'
if not parent_beacon_block_root_hex.startswith('0x'):
    parent_beacon_block_root_hex = '0x' + parent_beacon_block_root_hex

# Print the parentBeaconBlockRoot for visibility
print("parentBeaconBlockRoot being queried:", parent_beacon_block_root_hex)

# Use parentBeaconBlockRoot for further queries
headers_url = f"{os.getenv('CONSENSUS_LAYER_URL')}/eth/v1/beacon/headers/{parent_beacon_block_root_hex}"

header_response = requests.get(headers_url)
if header_response.status_code != 200:
    print("Failed to fetch data:", header_response.status_code)
    print(header_response.text)
    exit()

header_data = header_response.json()
if 'data' not in header_data:
    print("Unexpected response format:", header_data)
    exit()

slot_number = int(header_data['data']['header']['message']['slot']) + 1

# Retrieve blobs
blobs_url = f"{os.getenv('CONSENSUS_LAYER_URL')}/eth/v1/beacon/blob_sidecars/{slot_number}"
blobs_response = requests.get(blobs_url).json()
blobs = blobs_response['data']

# Process each blob
results = []
for i, tx_hash in enumerate(type_3_tx_hashes):
    blob = blobs[i]
    print(f"Retrieved KZG commitment for transaction {tx_hash}: {blob['kzg_commitment']}")
    
    blob_data_hex = blob['blob']
    
    # Save blob data to a file
    with open(f"blob{i}.txt", "w") as file:
        file.write(blob_data_hex)

    # Load blob data from the file and ensure it's correct
    with open(f"blob{i}.txt", "r") as file:
        blob_hex = file.read().strip()
        blob_data = bytes.fromhex(blob_hex.replace("0x", ""))  # Ensure consistent handling
        print(f"Blob data file for transaction {tx_hash}: blob{i}.txt")

    # Load trusted setup
    ts = ckzg.load_trusted_setup("trusted_setup.txt")

    # Compute KZG commitment
    commitment = ckzg.blob_to_kzg_commitment(blob_data, ts)
    print(f"Locally computed KZG commitment for transaction {tx_hash}: {commitment.hex()}")
    
    # Compute versioned hash
    sha256_hash = hashlib.sha256(commitment).digest()
    versioned_hash = b'\x01' + sha256_hash[1:]
    
    # Compare with network data, ignoring the '0x' prefix
    network_commitment = blob['kzg_commitment']
    local_commitment_hex = '0x' + commitment.hex()

    commitment_match = local_commitment_hex == network_commitment
    print(f"KZG commitment match for transaction {tx_hash}: {commitment_match}")
    
    # Use the stored blob versioned hashes during blob processing
    network_versioned_hash = blob_versioned_hashes_dict.get(tx_hash, "No blob versioned hash found")
    print(f"Network versioned hash for transaction {tx_hash}: {network_versioned_hash}")
    
    print(f"Blob data file for transaction {tx_hash}: blob{i}.txt")
    print(f"Locally computed KZG commitment for transaction {tx_hash}: {commitment.hex()}")
    print(f"Locally computed versioned hash for transaction {tx_hash}: {versioned_hash.hex()}")
    print()
    
    results.append({
        'transaction_hash': tx_hash,
        'commitment': commitment.hex(),
        'versioned_hash': versioned_hash.hex(),
        'commitment_match': commitment_match,
        'versioned_hash_match': versioned_hash.hex() == network_versioned_hash
    })

print("### SUMMARY ###")
print(f"Block {block_number}, Slot {slot_number}")
print("Type-3 transactions:", type_3_tx_hashes)
print()
for result in results:
    print(f"TX:{result['transaction_hash']}:")
    print(f"KZG: {result['commitment']}")
    print(f"Versioned hash: {result['versioned_hash']}")
    print(f"Locally computed match for the retrieved blob:")
    print(f"KZG commitment: {result['commitment_match']}")
    print(f"Versioned hash: {result['versioned_hash_match']}")
    print()

The transaction:

TX:5a74bd72aeeb99e874e58b927f9a5c96665278a36b61bed69a4b09597b02edce:
KZG: 9493a713dd89eb7fe295efd62545bb93bca395a84d18ecfa2c6c650cddc844ad4c1935cbe7d6830967df9d33c5a2e230
Versioned hash: 01afb6777db05b1376ccb3d0c1e842d437d2a8ad9c0acfb8247edab425fee61c
Locally computed match for the retrieved blob:
KZG commitment: True
Versioned hash: True

Conclusion, What’s Next?

I hope I have provided comprehensive information for you to begin leveraging off blobs in your dApps and to continue profiting from lower gas fees. Of course, there is plenty more than what meets the eye and despite me going quite deep into this iceberg, I strongly insist that you continue experimenting and exploring this rabbit hole of Ethereum Blobs. The early bird catches the worm remember?

EIP-4844 Documentation:

https://github.com/ethereum/EIPs/blob/master/EIPS/eip-4844.md

Advanced examples from Optimism’s Github:

https://github.com/ethereum-optimism/optimism

This technology is only one simple yet great solution to Ethereum’s scalability problem. While the concept of Ethereum is great, as it becomes a integral part of our lives, its illness (scalability issues) becomes worse and the dilemma of ensuring decentralisation and security at the same time will only make the problem harder to solve. But this is only one hurdle that has already been overcome as seen with the invention of Blobs. Clearly, there will be numerous more solutions that will be implemented to ensure the scalability of Ethereum remains sustainable and efficient in the near future.

With that done and dusted, there is no doubt that Ethereum will continue being the forefront of dApps and other blockchain applications. Happy Blobbing!