TIL Optimism core utilities currently depend on geth-specific RPC and, if your backend / full node is not geth or does not support such RPC, your app will fail at common operations such as sending a tx:

2024-06-24 19:51:37.478+00:00 | vert.x-worker-thread-18 | TRACE | AbstractJsonRpcExecutor | {"jsonrpc":"2.0","id":22,"method":"eth_maxPriorityFeePerGas"}

2024-06-24 19:51:37.478+00:00 | vert.x-worker-thread-18 | DEBUG | JsonRpcExecutor | JSON-RPC request -> eth_maxPriorityFeePerGas null

2024-06-24 19:51:37.479+00:00 | vert.x-worker-thread-18 | TRACE | AbstractJsonRpcExecutor | {"jsonrpc":"2.0","id":22,"error":{"code":-32601,"message":"Method not found"}}

Given I don’t want to switch back to geth, and it costs too much to switch to a cloud infra provider, also I don’t want to hack the app’s Optimism dependency. What else can I do?

Due to the geth-specific RPC eth_maxPriorityFeePerGas is sparsely called. I could potentially redirect only those particular calls to a cloud infra provider and therefore keep the usage low under free tier, then send everything else to my own full node. For that I need a reverse proxy with conditional routing, which can be done with Caddy.

Assume our system already have Go installed.

To install Caddy:

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

To install xcaddy (development tools):

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-xcaddy-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-xcaddy.list
sudo apt update
sudo apt install xcaddy

Our goal is to run a reverse proxy at localhost:8545 that redirects ETH RPC calls to either of the two upstreams: (1) eth_maxPriorityFeePerGas to a cloud infra provider (assume Alchemy Ethereum mainnet), and (2) all other calls to our full node (assume running at local network 192.168.1.123:8545).

Unlike regular reverse proxies, we need to read the request body in order to determine the route. Such feature does not come built-in so we will build our own caddy with a custom plugin to match request body payloads.

In addition, reading request body may come with performance penalty given payload size is unbounded. Luckily, since we know the exact payload to look for, we can hard-code a size cap to avoid unnecessary loads.

Create and initialize the project folder:

mkdir -p /path/to/my/caddy-reqbody-matcher
cd /path/to/my/caddy-reqbody-matcher
go mod init github.com/Detoo/caddy-reqbody-matcher

Create the request body matcher plugin: reqbody-matcher.go with the following codes:

package keywordmatcher

import (
    "bytes"
    "io/ioutil"
    "net/http"
    "unicode/utf8"
    
    "github.com/caddyserver/caddy/v2"
    "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
)

// KeywordMatcher implements caddyhttp.MatchInterface
type KeywordMatcher struct {
    Keyword string `json:"keyword,omitempty"`
}

// Match implements caddyhttp.MatchInterface
func (m KeywordMatcher) Match(r *http.Request) bool {
    // Read the request body
    body, err := ioutil.ReadAll(r.Body)
    if err != nil {
        return false
    }
    
    // Restore the io.ReadCloser to its original state
    r.Body = ioutil.NopCloser(bytes.NewReader(body))
    
    // Check if the body is a string and within the character limit
    if utf8.Valid(body) && len(body) < 1000 {
        // Check if the keyword exists in the request body
        return bytes.Contains(body, []byte(m.Keyword))
    }
    
    return false
}

// CaddyModule returns the Caddy module information.
func (KeywordMatcher) CaddyModule() caddy.ModuleInfo {
    return caddy.ModuleInfo{
        ID:  "http.matchers.reqbody_keyword",
        New: func() caddy.Module { return new(KeywordMatcher) },
    }
}

// UnmarshalCaddyfile implements caddyfile.Unmarshaler.
func (m *KeywordMatcher) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
    d.Next() // consume directive name
    
    // require an argument
    if !d.NextArg() {
        return d.ArgErr()
    }
    
    // store the argument
    m.Keyword = d.Val()
    return nil
}

func init() {
    // Register your matcher with Caddy
    caddy.RegisterModule(KeywordMatcher{})
}

Build caddy executable with the custom plugin. If everything compiles, we will find the artifact under the same folder.

xcaddy build --with github.com/Detoo/caddy-reqbody-matcher=/path/to/my/caddy-reqbody-matcher

Configure the reverse proxy and our custom plugin by creating the config file Caddyfile:

# Reverse proxy for ETH JSON RPC with geth backup
http://localhost:8545 {
    # Match geth-specific requests
    @reqbodyMatcher {
        reqbody_keyword "eth_maxPriorityFeePerGas"
    }

    # Default to our own node
    reverse_proxy 192.168.1.123:8545

    # Redirect geth-specific requests
    reverse_proxy @reqbodyMatcher {
        rewrite /v2/<your-api-key>{uri}
        to "https://eth-mainnet.g.alchemy.com"
        header_up Host eth-mainnet.g.alchemy.com
    }
}

Install it as a systemctl service so it runs 24/7. Create a new file caddy-reqbody-matcher.service with the following contents:

# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy-reqbody-matcher
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=detoo
Group=detoo
WorkingDirectory=/path/to/my/caddy-reqbody-matcher
ExecStart=/path/to/my/caddy-reqbody-matcher/caddy run --environ
ExecReload=/path/to/my/caddy-reqbody-matcher/caddy reload --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

Install and start the service:

sudo systemctl enable /path/to/my/caddy-reqbody-matcher/caddy-reqbody-matcher.service
sudo systemctl restart caddy-reqbody-matcher

Now the reverse proxy should be running at localhost:8545, to verify:

# This should go to our own full node
curl --header 'Content-Type: application/json' --data-raw '{"jsonrpc":"2.0","id":22,"method":"eth_blockNumber"}' http://localhost:8545

# This should go to cloud infra provider
curl --header 'Content-Type: application/json' --data-raw '{"jsonrpc":"2.0","id":22,"method":"eth_maxPriorityFeePerGas"}' http://localhost:8545

Now we have a proper backend to handle all RPCs!

TIL Optimism core utilities currently depend on geth-specific RPC and, if your backend / full node is not geth or does not support such RPC, your app will fail at common operations such as sending a tx:

2024-06-24 19:51:37.478+00:00 | vert.x-worker-thread-18 | TRACE | AbstractJsonRpcExecutor | {"jsonrpc":"2.0","id":22,"method":"eth_maxPriorityFeePerGas"}

2024-06-24 19:51:37.478+00:00 | vert.x-worker-thread-18 | DEBUG | JsonRpcExecutor | JSON-RPC request -> eth_maxPriorityFeePerGas null

2024-06-24 19:51:37.479+00:00 | vert.x-worker-thread-18 | TRACE | AbstractJsonRpcExecutor | {"jsonrpc":"2.0","id":22,"error":{"code":-32601,"message":"Method not found"}}

Given I don’t want to switch back to geth, and it costs too much to switch to a cloud infra provider, also I don’t want to hack the app’s Optimism dependency. What else can I do?

Due to the geth-specific RPC eth_maxPriorityFeePerGas is sparsely called. I could potentially redirect only those particular calls to a cloud infra provider and therefore keep the usage low under free tier, then send everything else to my own full node. For that I need a reverse proxy with conditional routing, which can be done with Caddy.

Assume our system already have Go installed.

To install Caddy:

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

To install xcaddy (development tools):

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-xcaddy-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-xcaddy.list
sudo apt update
sudo apt install xcaddy

Our goal is to run a reverse proxy at localhost:8545 that redirects ETH RPC calls to either of the two upstreams: (1) eth_maxPriorityFeePerGas to a cloud infra provider (assume Alchemy Ethereum mainnet), and (2) all other calls to our full node (assume running at local network 192.168.1.123:8545).

Unlike regular reverse proxies, we need to read the request body in order to determine the route. Such feature does not come built-in so we will build our own caddy with a custom plugin to match request body payloads.

In addition, reading request body may come with performance penalty given payload size is unbounded. Luckily, since we know the exact payload to look for, we can hard-code a size cap to avoid unnecessary loads.

Create and initialize the project folder:

mkdir -p /path/to/my/caddy-reqbody-matcher
cd /path/to/my/caddy-reqbody-matcher
go mod init github.com/Detoo/caddy-reqbody-matcher

Create the request body matcher plugin: reqbody-matcher.go with the following codes:

package keywordmatcher

import (
    "bytes"
    "io/ioutil"
    "net/http"
    "unicode/utf8"
    
    "github.com/caddyserver/caddy/v2"
    "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
)

// KeywordMatcher implements caddyhttp.MatchInterface
type KeywordMatcher struct {
    Keyword string `json:"keyword,omitempty"`
}

// Match implements caddyhttp.MatchInterface
func (m KeywordMatcher) Match(r *http.Request) bool {
    // Read the request body
    body, err := ioutil.ReadAll(r.Body)
    if err != nil {
        return false
    }
    
    // Restore the io.ReadCloser to its original state
    r.Body = ioutil.NopCloser(bytes.NewReader(body))
    
    // Check if the body is a string and within the character limit
    if utf8.Valid(body) && len(body) < 1000 {
        // Check if the keyword exists in the request body
        return bytes.Contains(body, []byte(m.Keyword))
    }
    
    return false
}

// CaddyModule returns the Caddy module information.
func (KeywordMatcher) CaddyModule() caddy.ModuleInfo {
    return caddy.ModuleInfo{
        ID:  "http.matchers.reqbody_keyword",
        New: func() caddy.Module { return new(KeywordMatcher) },
    }
}

// UnmarshalCaddyfile implements caddyfile.Unmarshaler.
func (m *KeywordMatcher) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
    d.Next() // consume directive name
    
    // require an argument
    if !d.NextArg() {
        return d.ArgErr()
    }
    
    // store the argument
    m.Keyword = d.Val()
    return nil
}

func init() {
    // Register your matcher with Caddy
    caddy.RegisterModule(KeywordMatcher{})
}

Build caddy executable with the custom plugin. If everything compiles, we will find the artifact under the same folder.

xcaddy build --with github.com/Detoo/caddy-reqbody-matcher=/path/to/my/caddy-reqbody-matcher

Configure the reverse proxy and our custom plugin by creating the config file Caddyfile:

# Reverse proxy for ETH JSON RPC with geth backup
http://localhost:8545 {
    # Match geth-specific requests
    @reqbodyMatcher {
        reqbody_keyword "eth_maxPriorityFeePerGas"
    }

    # Default to our own node
    reverse_proxy 192.168.1.123:8545

    # Redirect geth-specific requests
    reverse_proxy @reqbodyMatcher {
        rewrite /v2/<your-api-key>{uri}
        to "https://eth-mainnet.g.alchemy.com"
        header_up Host eth-mainnet.g.alchemy.com
    }
}

Install it as a systemctl service so it runs 24/7. Create a new file caddy-reqbody-matcher.service with the following contents:

# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy-reqbody-matcher
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=detoo
Group=detoo
WorkingDirectory=/path/to/my/caddy-reqbody-matcher
ExecStart=/path/to/my/caddy-reqbody-matcher/caddy run --environ
ExecReload=/path/to/my/caddy-reqbody-matcher/caddy reload --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

Install and start the service:

sudo systemctl enable /path/to/my/caddy-reqbody-matcher/caddy-reqbody-matcher.service
sudo systemctl restart caddy-reqbody-matcher

Now the reverse proxy should be running at localhost:8545, to verify:

# This should go to our own full node
curl --header 'Content-Type: application/json' --data-raw '{"jsonrpc":"2.0","id":22,"method":"eth_blockNumber"}' http://localhost:8545

# This should go to cloud infra provider
curl --header 'Content-Type: application/json' --data-raw '{"jsonrpc":"2.0","id":22,"method":"eth_maxPriorityFeePerGas"}' http://localhost:8545

Now we have a proper backend to handle all RPCs!