Checking the container logs, everything looks alright.

By using SSL Labs toolkit, it shows error “Failed to communicate with the secure server”:

Looking it up, I found

https://community.cloudflare.com/t/failed-to-communicate-with-the-secure-server/186871

which links to

https://developers.cloudflare.com/ssl/troubleshooting/version-cipher-mismatch

Then I found the root cause was I was using a multi-level subdomain:

One easy way to fix it is to merge the multiple levels from <subdomain1>.<subdomain2> to <subdomain1>-<subdomain2>.

But sometimes we do need multi-level subdomains. For example, when we deploy L2s for our clients. We want them to get resource URLs such as:

• https://myappchain.rpc.snapcha.in

• https://myappchain.explorer.snapcha.in

• …

In that case, we will use Cloudflare’s Advanced Certificates add-on feature that covers more than one level of subdomain.

One we purchased the package, we need to order those advanced certificates such as:

Then we can use the feature to issue certs when it becomes active:

Programming Model

• Global object pool: In Sui, persistent storage is supported via Sui’s global object pool rather than the account-based global storage of core Move

  • Move vs. the Sui Move

• Modules, Types and Abilities

• Objects and Ownership

  • objects are first-class citizens of the system

  • package code objects, and struct data objects

• Transactions: public tx (i.e. publish a package), call tx (i.e. call a package)

• Transaction Effects, Events, Create/Update/Wrap/Delete

The Sui System

• FastPay + Delegated Proof-of-Stake

  • requires 2/3 stake from honest authorities

  • liveness and eventual delivery: at least one honest party acts as a relay for each certificate between authorities

• object reference, transaction certificate, effect certificate

• persistent stores on authorities: 4 key-value maps

• causality & parallelism

• owned objects vs. shared objects

  execution for transactions involving read-only and owned objects requires only consistent broadcast and a single certificate to proceed; and Byzantine agreement is only required for transactions involving shared objects (using a high-throughput consensus system)

• full client (replica) and light client

Scaling and Latency

To ensure that more resources result in increased capacity quasi-linearly, the Sui design aggressively reduces bottlenecks and points of synchronization requiring global locks within authorities. Processing transactions is cleanly separated into two phases, namely (1) ensuring the transaction has exclusive access to the owned or shared objects at a specific version, and (2) then subsequently executing the transaction and committing its effects.

Discussions

In my view, the most significant innovations of Sui are:

• split the common “fire-and-forget” mode of broadcasting transactions in traditional blockchain networks to the two phases: 1) collect enough signatures from validators; 2) create the certificate and broadcast.

• object-based global storage rather than account-based

• differentiate the processes for handling single-owner objects from those for handling shared objects.

By combining these elements, the system realizes high throughput, low latency, and horizontal scalability.

However, effective utilization of the system demands careful design of the data model and operations by developers. For instance, the suitability of using the single-owner object model for creating a standard ERC-20 token is not immediately apparent.

Study Steps

1. read Introducing the OP Stack

2. watch the talks given by OP Labs’ Karl Floersch and Kelvin Fichter at Devcon 6

3. read the following specs: Introduction, Overview, Deposits, Rollup Node, Batch Submitter, System Config, L2 Chain Derivation(*)

*: for the derivation spec, I started to skip lots of the details from section Batch Queue

It should take ~2-3 days to study everything on this list.

High-level

• The OP stack is modular and minimal. It’s easy to swap with your own e.g. consensus, DA layers. The Superchain vision is able to scale horizontally.

• Example: Metis forked OP and customized the DA layer with a DA committee

• The 3 primary layers

  • Consensus

    • DA: an array of byte strings. not necessary a ledger

    • Derivation: takes the DA layer and the current state of the rollup and produces Engine API payloads

  • Execution: state transition

  • Settlement: a view that another chain has over your chain

Key Interactions

Note: step 4 is related to the concept of finalized L2 head.

Overview

The L2OutputOracle is a L1 contract to store rollup states (i.e. L2 output roots) and manage withdrawals. It will be helpful to think of it as the rollup output store and DepositFeed and BatchInbox as the rollup input store. So the L1 is simply the DA layer for storing rollup inputs and outputs, and the L2 is the computing layer.

DepositFeed is for a special transaction type (i.e. Deposited Transaction) on L2 defined by the Optimism protocol. BatchInbox is for the other normal L2 transactions (i.e. only transaction inputs) that are batch-submitted to L1.

It’s also useful to know that the BatchInbox is a regular EOA address (note: for saving gas cost by not executing any EVM code). The sequencer submits L2 transaction batches included in the calldata of the L1 transaction to the address.

Key Concepts

• Block Derivation

• Epochs and the Sequencing Window

• Deposited Transactions: This is a new transaction type on L2 defined by the protocol.

  • It’s important to understand that when tokens are deposited from L1 to L2, it’s technically “depositing the transactions to the L2“. The tokens are locked on L1 and then minted on L2.

  • deposited transactions do not need a signature and rely on other mechanisms for authorization and validation

  • L1 attributes deposited transaction and user-deposited transactions

  • L1 Attributes Depositor Account: an EOA with unknown private key.

  • L1 Attributes Predeployed Contract: it’s important to know that the contract will be called in the first transaction of every L2 block (source). Every L2 block has an L1 attributes tx.

  • Address Aliasing

• Batch Submission

  • Data Availability Provider (e.g. Ethereum calldata)

  • Sequencer Batch: each batch represents the inputs needed to build one L2 block

  • Batcher Transaction: a transaction submitted by a batcher to a data availability provider

  • Channel and Channel Frame

  • The diagram below illustrates these concepts as a whole

• L2 Chain Derivation

  • L2 Derivation Inputs

  • System Config: batcherHash stores the current authorized batcher sender(s)

  • Payload Attributes: object to be passed to the execution engine

  • L2 Chain Inception: the L1 block number where the L2 genesis starts

  • Finalized L2 Head, Safe L2 Head, Unsafe L2 Head, Unsafe Block Consolidation

    • it’s important to distinguish the difference of finalizing the L2 head with the concept of finalization period. The former form of finalization only affects inputs while the later affects Optimistic-rollup outputs.

• Fork Choice Rule: to determine the blockchain head

• Time Slot: every 2 second per L2 block

1. read https://docs.filecoin.io/about/basics/introduction/

2. read The Filecoin Virtual Machine: Everything You Need to Know

3. read https://docs.filecoin.io/about/basics/filecoin-faq/

4. read How Filecoin storage and retrieval works

5. watch Space Warp Summit 🛸 Understanding the FVM - Raul Kripalani

6. watch Space Warp Summit 🛸 Programming on the FEVM - Zak Ayesh

High-level

• Filecoin is built on top of the same software powering IPFS protocol but they are different networks

• Filecoin is different from IPFS because it has an incentive layer on top to incentivize contents to be reliably stored and accessed

• FVM has the additional ability to access verified proof of stored data within the virtual machine’s native environment

Key Concepts

• The Filecoin blockchain

  • Tipsets, Blocks, Messages

  • Actors (smart contract): 11 built-in actors

  • Nodes: chain verifier nodes, client nodes, storage provider nodes, and retrieval provider nodes. Any node participating in the Filecoin network should provide the chain verification service as a minimum.

  • Consensus

    • like proof-of-stake, Filecoin uses proof-of-storage for the leader election

    • relies on DRAND for randomness

    • Block production process

      • 30 secs each epoch/round

      • finality: 900 epochs (~7.5hr)

  • Addresses: f410 represents Ethereum addresses

  • Proofs

    • Sealing: Proof of Replication (PoRep)

    • Proof of Spacetime (PoSt)

      • WinningPoSt & WindowPoSt: be aware of the subtle differences

      • every sector is audited at least once every 24 hours

    • Collateral and Slashing

• Storage & Retrieval

  • Storage and Retrieval markets

    • Filecoin plus and verified clients

    • Storage on-ramps

      • web3.storage: automate data replication over multiple SPs

    • Basic retrieval & Saturn

  • Storage and Retrieval Providers

  • Deals

  • Sectors: basic units of provable storage

    • 32 GiB and 64 GiB sector sizes are supported.

• Compute-over-data & Compute-over-state

  • FVM is computer-over-state (i.e. metadata of the stored data such as cids)

    • It manipulates the metadata of the stored data deal. It does not manipulate the data stored in that deal.

  • FVM is built on top of Filecoin’s network. There is no additional chain.

  • FVM is WASM native and virtual machine-agnostic

  • FEVM is virtualized as a runtime (i.e. an actor) on top of FVM

    • compatible with any EVM tools

    • Filecoin Solidity libraries: to interact with built-in actors

  • delayed execution: messages/transactions included in one epoch won’t be executed until the next epoch. So it takes about ~1min before you can see the state change.

Caveats

• Storage

  • The network doesn’t guarantee the data availability. If your SP (storage provide) loses your data, they will get slashed but your data is permanently lost. So it’s recommended to make deals with multiple SPs for redundancy. But again, there is no guarantee.

  • Right now, verifiers to verify clients and allocate DataCap are chosen in a centralized way.

  • Right now, the Filecoin network does not allow modifying the data once a deal is made.

• Deals

  • Currently a deal in Filecoin has a minimum duration of 180 days

  • You might want to decide on a specific provider based on their reputation or power in the network. But reputation metrics for miners are not part of the Filecoin protocol yet.

  • to create a deal, it’s a non-trivial process (e.g. you need to download and install the Lotus client)

• Retrieval

  • Similar as storage, to avoid extortion, always ensure you store your data with a fairly decentralized set of storage providers (and note: it’s pretty difficult for a storage provider to be sure they are the only person storing a particular piece of data, especially if you encrypt the data).

  • Basic retrieval is slow (i.e. the unsealing process is non-trivial) and can take hours. The core team is working on it this year to achieve sub-millisecond speed. Right now you can use Saturn which is a CDN layer built on top of Filecoin.

  • Retrieval deals, unlike storage deals, happen mostly off-chain facilitated by payment channels. Depending on whether the miner has the data in their block-store or not, they might need to first unseal it.

• FVM & FEVM

  • No Ethereum gas accounting: metering and execution halt based on Filecoin rules

Discussions

My goal of studying Filecoin/FVM is to figure out whether I can use Filecoin as the DA layer for a rollup using the OP stack. Specifically, I am looking for a decentralized, affordable, reliable and scalable DA layer. My conclusion is although it’s pretty decentralized and affordable, it’s not reliable in regards to usability and security.

The biggest issue is data(*) stored in the Filecoin network cannot be modified. But a DA layer for a rollup needs to constantly update the data (i.e. receive deposit transactions and transaction batches, update L2 Output Root).

Security is another major concern. The DA layer for a rollup can be much more valuable economically than the collateral from the SPs. So it’s hard to protect against extortions on retrieval. Given the nature of a rollup, it’s very likely the DA layer needs to be public which implies everyone will know the SPs that are storing the data. Thus, the encryption approach mentioned here won’t work and an extortion can still happen if the SPs collude together.

Slow native retrieval data is a less important concern. But it will influence the L2 chain derivation process. Saturn can be used to workaround the problem but that also means an additional trust dependency is added.

Note that the discussion is focused on using the Filecoin network’s storage as the DA layer, mainly for it’s cheap storage cost. However, the Filecoin blockchain itself can also be the DA layer but that does not give many advantages over other L1s. The 7.5-hour finality time and 30-sec epoch are just too slow even comparing to the benchmark Ethereum network.

*: “data” here does not include the data on the blockchain.

References

• Filecoin Spec

• FVM Space Warp ETHGlobal Hackathon Cheat Sheet

• Nodes geo-distribution

• Token distribution

• FVM roadmap

Appendix: Unanswered Questions

Note: I am not going to spend more time on the list as I have already figured out that Filecoin won’t be a good DA layer for what I need.

General

1. Is the revenue of the Filecion ecosystem mainly from the storage/retrieval markets? What's the size of the revenue? Is there a dashboard for the data?

2. For SP, how much is the required collateral? There are some formulas on the spec but I'd like to know some estimated dollar values.

3. What's the current sector fault rate? Is there a dashboard to track? Where can I find any example of enforced slash transactions? Is the fault fee paid back to the sector data owner?

Technical

1. Does PoSt uses data availability sampling to prove the data?

2. In FVM, is the state storage and transaction history persistent? are there any mechanisms for state expiry and old data pruning?

3. It seems that there aren't any orders for the blocks inside one tipset. Then how to enforce transaction orders inside those parallel blocks?

4. What's the maximum allowed number of blocks in a tipset? What's the block size limit and the implies TPS?

5. Does the FVM transactions (e.g. contract call) have the same priority as other types of messages (e.g. ProveCommitSector) from the miner's perspective?

6. How to prove the stored/retrieved data is the original one and not tampered, if the owner does not keep a copy?

7. How to determine the assigned weight to a tipset?

https://community.optimism.io/docs/governance/economics/

The above page is the official documentation about the Economics of the OP ecosystem. It also has a nice diagram (see below) to demonstrate the economics.

However, it does not clearly picture where the token holders fit in. The only thing mentioned in the page is:

• Value accrues to tokenholders through the productive re-deployment of sequencer revenue. Sequencer revenue is primarily directed to fund public goods, which creates ecosystem value and drives demand for blockspace.

This part is a bit vague to me as I cannot see a link between the distribution of sequence revenue and how value accrues to token holders. In the diagram above, the revenue is distributed to the retroactive Public Good Funding (RetroPGF), which goes to the builders, not the token holders.

As far as I know, the token right now does not have any other use cases other than governance.

However, there are some discussion happening in the official discord. The screenshot below implies that when the sequencer is decentralized, the revenue could be allocated to places other than the RetroPGF. So maybe they will have a staking system by then to distribute some revenue to the token holders.

Token Distribution

https://community.optimism.io/docs/governance/allocations/

Overall, it looks healthy to me.

Valuation

https://dune.com/optimismfnd/optimism-l1-batch-submission-fees-security-costs

The above dashboard has some statistics for the Optimism fees. The sequencer fee margin section is the actual profit (i.e. revenue - cost). In the below screenshot, we can see that the 30-day average profit is 13.63 ETH.

Given a 100x PE ratio, with the current ETH price being $1605, we can do a raw estimation of the valuation of the ecosystem to be:

13.63 * 365 * 1605 * 100 ~= $800 million

The market cap implies by the current token price is $532 million while the fully diluted valuation is at $1 billion.

Discussions

The token price of OP is too high even considering my raw estimation with a very generous PE ratio.

More importantly, the token itself is just a governance token with no other utilities, at least at the moment. In general, I don’t recommend to invest in any governance tokens.

However, I am bullish on the Optimism ecosystem for the following reasons:

• Ethereum community has the consensus on the rollup-centric roadmap

• Optimism is one of the first rollup projects that are deployed and has a large market share

• The upcoming Optimism Bedrock upgrade and the OP stack release are very exciting!

• I sense a growing trend of the mind share of the OP ecosystem among developers (probably related to the OP stack!)

In summary, I think a good timing to invest in the OP token is when:

• the token has more utilities

• the sequencer model changes to decentralized and distribute revenue to token holders directly

• the token price drops to a point where the derived market cap is much lower comparing to the valuation derived from the revenue

So my advice is to look out for the governance proposals and follow closely with the community updates if you want to find some alphas in the market.

• In 2022, I entered my thirties.

• I left my job in August to pursue my passion for web3 technology.

• I explored different options to generate cash flows in DeFi sector.

• ETHSF winner and got accepted by a top web3 accelerator.

DeFi

• I developed a prototype system for hunting airdrops of crypto projects. Ultimately, I decided to abandon the project, recognizing that the pursuit of money was not something that inspired me.

• I discovered a cash-printing DeFi project and became deeply involved with its community and team. I provided meaningful input in the direction of the project. Additionally, I built a sophisticated monitoring system to maximize profits and generate earning reports.

• I was so fortunate to have narrowly avoided being impacted by the LUNA crash and the FTX collapse.

• I explored MEV with the intention of generating a cash flow, but I soon abandoned the project due to the same aforementioned reason. However, I managed to create and open source two tools for MEV as a side-effect: Ethereum-Account-Generator (for generating accounts with leading zeros to save on gas fees) and Ethereum-Private-RPC (which can give you an advantage when it comes to MEV).

• I conducted an in-depth analysis of various protocols for ETH staking and wrote a script to report earnings and track APYs of various pools to optimize profits.

• I started a private business to generate revenues from DeFi protocols through staking. However, the protocol was heavily affected by the FTX/Alameda collapse.

Career

• I started to consider leaving my job at Meta in late 2021 to start my own business. I officially resigned in August 2022.

• When I quit my job, I intended to first explore the web3 space and conduct research. I anticipated it would take roughly a year before I found a tangible project to work on. However, after seeing a tweet from a speaker at EthCC about their successful ETHGlobal hackathon project, I was inspired to apply for ETHSF. I chose an idea from my list and assembled a team to work on it. Despite having little sleep, we put in a tremendous amount of effort and, ultimately, our project was chosen as one of the 13 finalists out of 284 teams.

• After the hackathon, I conducted further research and validation and successfully applied to a top web3 accelerator (1% admission rate), raising funding at a $7M pre-seed valuation.

Key take-aways

• Looking back, I should have left Meta in May, when I had already lost my enthusiasm for my job. The last three months at Meta were difficult for me, both mentally and in terms of productivity.

• I learned a lot about myself. I came to the realization that, at my core, I am a builder. As a result, chasing after money is not something that will bring me true fulfillment.

• I have learnt an important lesson to never invest money that I cannot afford to lose in high-risk investments.

• In the web3 world, one has to dive deeper into the protocols and actively keep track of the latest developments and governance proposals to find the alphas.

• By being open to new possibilities, I experienced serendipitous moments that led me to some amazing opportunities. If I had continued to pursue my plan of independently exploring the web3 space without raising capital, my progress will be much slower than it is currently.

• Focus on the right things. At the ETHSF hackathon, I did not have any expectations of taking home a prize, and I made sure to communicate this to my teammates. We kept our attention on creating a strong project, which ultimately earned us a spot as finalists.

• A few people suggested that without a co-founder, it would be difficult to secure funding or gain acceptance into accelerator programs. Yet, finding the right co-founder is akin to finding one's perfect partner and thus, I chose not to rush the process. Instead, I applied as a solo founder and, through sheer hard work and dedication, was able to make it work.

2023 Goals

• Build MVP & showcase in demo day

• Find product-market fit

• Build founding team

However, the legal and financing terms used in the agreement can be overwhelming to founders that are not familiar with the process. I am gonna use Safe: Valuation Cap, no Discount as an example to clarify a little bit.

The first thing you need to understand is that the agreement is for future equities. Meaning that you don’t need to go through the complicated process of getting all other investors to sign documents or issuing new stocks to the new SAFE investor. This is also why SAFE is becoming the industry standard.

The second thing you should know is that there are 3 major types of events that will “activate” the agreement and lead to certain implications. Then the agreement will be terminated automatically.

Equity Financing

In human words, this refers to a formal round of fundraising in exchange of stocks. When it happens, the SAFE investor will receive some stocks as well. The amount equals to the valuation divided by the SAFE investment. The valuation will be chosen as the smaller of the formal round valuation and the SAFE valuation (i.e. Post-Money Valuation Cap).

For example, if the SAFE valuation cap is $10M and the purchase amount is $1M, then the SAFE investor will receive 10% of the company's equity if a formal round of funding is closed at a valuation of $50M. However, if the formal round of funding is closed at a valuation of $5M, the SAFE investor's equity stake will increase to 20%.

Liquidity Event

This usually happens when the company IPOs or is acquired by other companies. When such events happens, instead of receiving stocks, the SAFE investor receive some money.

The amount is determined by the valuation when the event happens. If the valuation is smaller than the SAFE valuation, the SAFE investor gets their money back, fully. Otherwise, the amount will increase in proportional to the valuation increase.

As an illustration, if the pre-money valuation cap of the SAFE is set at $10M and the purchase amount is $1M, then the SAFE investor is entitled to $5M when the company is acquired with a $50M valuation.

Dissolution Event

In the event of a company closure initiated by the founders, SAFE investors are entitled to receive a full return of their investment.

Nevertheless, the investors' return will be contingent upon the amount of funds the company has remaining. If the company lacks sufficient funds, they will receive only a fraction of their initial investment, the magnitude of which will be determined by the "Liquidation Priority" as stipulated in Section 1(d).

In essence, investors have a higher preference than founders because commonly, founders own Common Stock while investors own Preferred Stock or a SAFE.

If, for instance, the SAFE valuation cap is $10M, and the purchase amount is $1M, with an additional investor who owns 10% Preferred Stock, then the remaining funds of $100 upon dissolution should yield $50 to the SAFE investor.

Simple task

I decided to ask chatGPT to write a small helper function that has specific requirements.

A perfect example immediately came up to my mind. This is a util function I needed in the ETHSF hackathon. I assigned it to one of my teammates and it took a few back-and-forths commits to get it right. The time between the first and last commit is 4 hours.

So let’s assign the same task to chatGPT. My initial message to it:

/*
arr is an array of strings. examples:
[
  '--syncmode=full',
  '--datadir=<datadir>',
  '--nodekey=<nodekey>',
  '--port=<port>',
  '--http',
  '--http.addr=0.0.0.0',
  '--http.port=<httpport>',
  '--networkid=<networkid>',
  '--mine',
  '--miner.threads=1',
  '--miner.gasprice=1',
  '--unlock=<unlock>',
  '--password=<password>',
  '--allow-insecure-unlock'
]
[ '<httpport>:<httpport>', '<port>:<port>' ]
[ '<datadir>:<datadir>' ]
pairs is an object to specify how to do the replacement. example:
{
  '<datadir>': '/path/to/datadir',
  '<nodekey>': '/path/to/nodekey/file',
}
example return:
[ '<datadir>:<datadir>' ] => ['/path/to/datadir:/path/to/datadir']
should return a new array of strings that applies the replacements
*/
export function replaceStringArray(arr, pairs) {
}

please implement the `replaceStringArray` function based on the requirement above it

Then I got the response:

It also gave some test input and output to help me verify the correctness:

However, there is a bug and I pointed out:

chatGPT tried to fix it:

Unfortunately, chatGPT misunderstood the requirement. It’s worth noting that it’s my mistake that I didn’t specify clearly that it should “replace all” in the requirements. So I clarified it and this time it gave the right solution:

However, I noticed that it differs from my solution where replaceAll is used. But it uses regular expression. So I asked it:

chatGPT nicely explained its choice and helped me learn this new knowledge.

Then I asked it to optimize the code:

And chatGPT gave a few suggestions:

Conclusions

chatGPT is an effective tool for quickly finding solutions to coding problems, saving developers valuable time and effort.

However, if you don’t specify the requirements clearly, it might misunderstand and give solutions that have bugs in it.This also implies that you should know how to solve the coding question yourself and verify the solution carefully. It’s dangerous to blindly trust the solution given by it.

It’s also worth noting that if you find the bug and described it clearly to chatGPT, it can fix it.

It’s also interesting that if you have a different solution in mind, you can ask chatGPT why it’s not chosen. You might be surprised and learn something new.

You can also ask chatGPT to optimize the code. You can even repeat that for a few times until you feel the code is too hard to maintain.

My setup

• I created 3 droplets on digitalocean (CentOS; shared CPU; 1GB 1Core)

• install and start docker (with the compose plugin) in all droplets

Step 1: set up bootnode (droplet A)

first create boot.key

mkdir /root/devnet/ && docker run -d --rm -v /root:/root ethereum/client-go:alltools-latest bootnode --genkey=/root/devnet/boot.key

then create a compose file and run docker compose up -d:

services:
  bootnode:
    image: ethereum/client-go:alltools-latest
    container_name: bootnode
    command:
      [
        'bootnode',
        '--nodekey=/root/devnet/boot.key',
        '-addr=:30303',
        '-verbosity=5',
      ]
    ports:
      - '30303:30303/udp'
    restart: always
    volumes:
      - /root:/root

in docker compose log, you will see a encode URL like this:

enode://<REDACTED>@127.0.0.1:0?discport=30303

replace 127.0.0.1:0 with <the public IP>:30303 and take it down some where. We will refer it as encodeURL later.

Step 2: create account (droplet B and C)

echo 123456 > /root/devnet/password.txt
docker run -d --rm -v /root:/root ethereum/client-go:alltools-latest geth account new --password /root/devnet/password.txt

then you can find the address by ls /root/.ethereum/keystore/

you will see a file: UTC--2022-10-28…--<address> where you can extract the address.

Step 3: create genesis file

you can run this command in any server to create the genesis file (Note: don’t forget to export the file to /root/devnet/devnet.json)

docker run -it --rm -v /root:/root ethereum/client-go:alltools-latest puppeth

you should provide the two addresses generated in previous step for these questions:

• Which accounts are allowed to seal?

• Which accounts should be pre-funded?

then modify the "nonce" field to be a random value as recommended here

last, copy the genesis file to both droplet B and C

Step 4: init genesis block and start the node (droplet B and C)

docker run -d --rm -v /root:/root ethereum/client-go:alltools-latest geth init /root/devnet/devnet.json

then create a compose file and run docker compose up -d:

services:
  geth:
    image: ethereum/client-go
    container_name: geth
    command:
      [
        '--syncmode=full',
        '--bootnodes=<encodeURL>',
        '--port=30303',
        '--http',
        '--http.addr=0.0.0.0',
        '--networkid=<read from the genesis file>',
        '--mine',
        '--miner.threads=1',
        '--miner.gasprice=1',
        '--unlock=<step2 generated address>',
        '--password=/root/devnet/password.txt',
        '--allow-insecure-unlock',
      ]
    ports:
      - '8545:8545'
      - '30303:30303'
    restart: always
    volumes:
      - /root:/root

Step 5: verify

in the bootnode server, run docker compose log you should see:

bootnode  | TRACE[10-29|03:28:44.657] << FINDNODE/v4                           id=<REDACTED> addr=<REDACTED>  err=nil
bootnode  | TRACE[10-29|03:28:44.657] >> NEIGHBORS/v4                          id=<REDACTED> addr=<REDACTED>  err=nil

in the other servers, you should see:

geth  | INFO [10-29|00:32:41.507] Looking for peers                        peercount=1 tried=0 static=0
geth  | INFO [10-29|00:32:43.326] Successfully sealed new block            number=42 sealhash=<REDACTED> hash=<REDACTED> elapsed=14.483s
geth  | INFO [10-29|00:32:43.326] 🔨 mined potential block                  number=42 hash=<REDACTED>
geth  | INFO [10-29|00:32:43.327] Commit new sealing work                  number=43 sealhash=<REDACTED> uncles=0 txs=0 gas=0 fees=0 elapsed="269.928µs"

you can also run:

docker run -it --rm -v /root:/root ethereum/client-go:alltools-latest geth attach
> eth.blockNumber

and you will see the blockNumber increasing.

congrats on spinning up your own EVM chain!

References:

• Setup your own private Proof-of-Authority Ethereum network with Geth

• Operating a private network

• Go Ethereum: Private Networks

So I tried to increase the gas price but I kept getting this “replacement transaction underpriced” error.

Later I found out the problem was that I need to increase the “priority fee” as well. Simply increasing max gas price is not enough.

Another interesting finding when I was trying to directly interact with the contract on Etherscan (although failed with the same error) is that 0x0 and 0x are different values for bytes fields. If you want to pass in empty value for the bytes type parameter, you should input 0x instead of 0x0.

The problem is the payload can get very complicated. It’s tiring to type it yourself. Recently, I found our Chrome has a nice feature which allows you to copy the entire payload object:

The copied string is a JavaScript object so you can paste directly into your JavaScript code without any modifications.

Let’s look at an example. Say I want to parse some data from stake.rocketpool.net, simply doing an http request and parse the result with Cheerio won’t work because it needs to execute some code in a browser to fetch the data.

By examining how the page loads, I noticed that class="loaded" will be added to the <body> tag when it finishes loading.

So you can write the code as:

To select a certain element in the HTML, you can test it out in chrome first:

Then you can write the code using the Cheerio APIs:

Puppeteer also allows you perform complex tasks. Here is an example where it inputs some text in the page and lets you parse the updated HTML:

Sometimes, it’s not easy to figure out the right selector for waitForSelector. A general solution (i.e. it won’t always work) is:

await page.goto(url, { waitUntil: 'networkidle0' });

Limitations

Web scrapping isn’t the antidote for all situations. It’s slower, more complicated and less reliable (e.g. sometimes you will get the “Navigation timeout” error) than making http requests directly. Thus, in some cases, you might want to consider making http requests directly (using libraries such as axios) if it’s easy to identify the corresponding network request by analyzing the requests using the browser’s developer tools.

Some sites might also give you the HTML containing the data you want from making plain http requests. So you can use Cheerio directly to parse it. Here is an example:

Puppeteer also does not work well with docker. You can check out the guide if you want to dive into the rabbit hole.