# Obol Network - Bia Testnet By [GLCstaked](https://paragraph.com/@glcstaked) · 2023-02-06 --- **Easy step by step guide to running a distributed Validator cluster on Obol’s – BIA Testnet.** _(document last update:16/4/23)_ **Obol Network:** is an ecosystem for trust minimized staking that enables people to create, test, run & co-ordinate distributed validators. Charon is a distributed validator middleware client, for running a validator across a cluster of nodes across many client implementations to improve resilience. [ Infrastructure - Diving deep on Obol ------------------------------------ Decentralising the internet bond through Distributed Validator Technology https://solardefi.substack.com ![](https://storage.googleapis.com/papyrus_images/194aecf8abf124d3ed27a3f47c085db4b905d255126037ee900853819ebb057e.jpg) ](https://solardefi.substack.com/p/infrastructure-diving-deep-on-obol) **Bia Testnet:** Bia is the second official Testnet for Obol succeeding the first Testnet [Athena](https://blog.obol.tech/the-athena-testnet/), I am running a DVT Cluster with **Obol Ar líne**, and will be updating this guide as the testnet evolves. BIA Testnet - Getting started ----------------------------- **Hardware Requirements**: CPU with 4+ cores, 16 GB+ RAM, A fast SSD drive with at least 1 TB of space (storage capacity will grow over time) 25 MBit/s bandwidth **Install Linux:** Get started with Ubuntu on a local device [ Ubuntu 20.04.6 LTS (Focal Fossa) -------------------------------- CD images for Ubuntu 20.04.6 LTS (Focal Fossa) https://releases.ubuntu.com ](https://releases.ubuntu.com/focal/) 1\. Install Pre-requisite software ---------------------------------- **Update Ubuntu** sudo apt update && sudo apt upgrade -y **Install curl and git** sudo apt install curl git -y **Install Docker and Docker compose plugin** Install Docker instructions from [here](https://docs.docker.com/engine/install/). curl -fsSL https://get.docker.com -o get-docker.sh sh get-docker.sh remove the install script and add $USER to docker group, to allow use without `sudo` sudo rm -r get-docker.sh sudo usermod -aG docker $USER _Restart a new shell, this is to allow the adding $USER to docker group._ Check installed with (if correct will output version) docker --version This script should also install **docker compose plugin**, which you can check with the following command ![](https://storage.googleapis.com/papyrus_images/8036b1dc72b25470c31a24e9c5ac74a737cb6aa45f5fa5a0fcc5922e84255f26.png) If docker compose is not present, Install Docker Compose instructions from [here](https://docs.docker.com/compose/install/). **Firewall Settings** 30303 udp/tcp: Execution p2p ports 9000 udp/tcp: Consensus p2p ports 3610: Charon port sudo ufw allow 30303 sudo ufw allow 9000 sudo ufw allow 3610 sudo ufw enable NOTE: if running on a VPS you should be sure to enable SSH `ufw allow ssh`, check the status of firewall with `ufw status` **Port forwarding:** under local setup you will need to port forward these ports for the device local IP, this varies depending on internet provider and router. Can be done by logging into router settings. 2\. Cluster group formation --------------------------- **Address collection** Collect addresses of all Operators for the cluster, messages are required to be signed from the launchpad. The Cluster leader requires these addresses to continue. 3\. Create ENR for Charon ------------------------- In order to prepare for a distributed key generation ceremony, you need to create an [ENR](https://docs.obol.tech/docs/next/int/faq/errors#enrs-keys) private key for your Charon client. All operators including the leader will need to do this step. **Clone Repository for Charon** git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git ![](https://storage.googleapis.com/papyrus_images/eed0de4ee9d33b17c4987936d7fe311a5404b1b6e739d33736e6b2910776c92c.png) **Create ENR private key** cd charon-distributed-validator-node docker run --rm -v "$(pwd):/opt/charon" obolnetwork/charon:v0.13.0 create enr ![](https://storage.googleapis.com/papyrus_images/51bc2b77dface10607d67e8874f154336e6895f3e68cd733a9fdc5dc91898342.png) **Backup ENR key** Your ENR private key is stored under `.charon/charon-enr-private-key` this should be backed up. This is displayed on the output it can be copied to a text file and backed up externally. To view it you can display with `cat` or `nano` commands sudo cat .charon/charon-enr-private-key Create the DKG configuration file (LEADER ONLY) ----------------------------------------------- The cluster leader must prepare the configuration file for the distributed key generation ceremony using the launchpad. [https://bia.launchpad.obol.tech/](https://bia.launchpad.obol.tech/) 4\. Cluster Configuration ------------------------- Cluster operators will be invited to join the configuration by **invite link,** this will take you to a specific group created for the DV launchpad. All Operators will need to contribute their **ENR key** to configure the cluster-definition. You will need Metamask for this, using the same wallet provided to the Cluster leader, and connected to Goerli test network. ![It will look similar to this, and you will need to connect your wallet ](https://storage.googleapis.com/papyrus_images/cbf25e9414074494cd7580ea5f6eb85ca9d3a4c30075af70314fe7d0eac99b4f.png) It will look similar to this, and you will need to connect your wallet **Enter ENR key for configuration** Have your **ENR key** ready for adding to the configuration. **Download cluster-definition (optional)** download the `cluster-definition` file manually and move it to the hidden `.charon` folder. If using the UI on Ubuntu, copy the file to a local folder such as `$HOME/obol-files` and then move to `.charon` directory like so sudo mv $HOME/obol-files/cluster-definition.json .charon 5\. Run the DKG Ceremony ------------------------ All operators need to be running the command simultaneously, at an agreed time ![](https://storage.googleapis.com/papyrus_images/04a8bf4a0576e527466363c6872503a7e006d85cc5996449bb9d052be76109bc.png) the docker run command can be copied from here, all operators partaking in the cluster must run this simultaneously. 6\. Backup Validator Keys ------------------------- After successful DKG, a number of artefacts will be created in the `.charon` folder. `deposit-data.json`, `cluster-lock.json` & `validator_keys/` folder containing keystores of number of validators setup during DKG. **Backup the Validator Keys folder** All operators must backup their unique keystores cd mkdir charon-backups cd charon-distributed-validator-node sudo cp -r .charon/validator_keys $HOME/charon-backups ![](https://storage.googleapis.com/papyrus_images/8ba8552c27fdf1a119410a2c063d5a6c3e5c9034f48e6a9f8140faab186eb613.png) The same process above can be followed for `deposit-data.json` & `cluster-lock.json` _Note: these files are identical per operator._ To grant permissions to this folder so it can be freely moved, the `-R` will grant to every file inside. sudo chown $USER:$USER -R $HOME/charon-backups 7\. Start the Validator Node ---------------------------- To run our validator node, we must first run and fully sync and Execution layer client and consensus layer client. By default Charon is set to sync execution layer client (geth) and a consensus layer client (lighthouse). from the working folder `charon-distributed-validator-cluster` docker compose up -d this will pull several docker containers and output the logs of each on the terminal (this can be messy see ‘viewing container logs’ in useful commands). -d flag means detach the containers will start in the background, to view logs individually see ‘Useful commands’ section. At this point we are waiting for the Execution and Consensus clients (default GETH and Lighthouse) to finish syncing, before proceeding to the next step. ![Execution client not finished syncing](https://storage.googleapis.com/papyrus_images/ba5a862eb3fb4ab04d41b75392cbf21e5103274a3af8aea1b5689bfb98fca9a1.png) Execution client not finished syncing You might see something similar, the execution client is still syncing so the consensus is waiting for sync to complete, however client is connected to at least 1 peer which is required for nodes to start sync. ![Lighthouse logs once synced](https://storage.googleapis.com/papyrus_images/1abb5cacb073e6a07cf90bc0e095f28be3aebd84dd2200e5da905dbc9cd975a9.png) Lighthouse logs once synced ### Check Grafana Dashboard Open Grafana dashboard from the terminal open http://localhost:3000/d/singlenode/ Direct from the browser type into the search bar: `http://0.0.0.0:3000/d/singlenode/single-charon-node-dashboard?orgId=1&refresh=10s` _If you want to do this externally you will need to open port 3000, and use the public IP of your device._ \*\*Things to check:\*\*Your Charon client can connect to the configured beacon client and it can connect to all peers. ![](https://storage.googleapis.com/papyrus_images/2ca9c377b76ddc630ad2b9d99be3dc287e7460ea732f50d1ae85ec120fec63fb.png) 8\. Make Deposit ---------------- **When all cluster operators are synced and healthy, the cluster is ready to proceed to the deposit step.** ONE operator may process to activate this deposit data with the existing staking launchpad. [https://goerli.launchpad.ethereum.org/en/](https://goerli.launchpad.ethereum.org/en/) You will need to upload the `deposit_data.json` found in `.charon` ![](https://storage.googleapis.com/papyrus_images/637cc0ed8ef352b2cb5853360e84777575a10ef7e8048b623a092a2b4246ec3d.png) You will be prompted to go though even more checklists, eventually connect metamask and confirm the transaction. You will be given your validators public key which helps to take note of. Useful Commands --------------- **Viewing container logs** You can view running containers with docker ps -a View individual logs for selected container, from the working directory docker compose logs -f Service name will be on the `docker-compose.yml`, default these are either `geth`: execution logs, `lighthouse`: consensus or `teku`: for validator logs **Troubleshooting** [https://docs.obol.tech/docs/next/int/faq/errors](https://docs.obol.tech/docs/next/int/faq/errors) Selecting Alternative Clients ----------------------------- Charon (at least for now) is set to run GETH for execution client and Lighthouse for consensus, with Teku client for the validator client, Ethereum does particularly well in client diversity which is important for network health, you can find various client implementations [here](https://launchpad.ethereum.org/en/). ![](https://storage.googleapis.com/papyrus_images/207c04e9db987064d0d8a94c87fbee6f83c5a1b696ce428a8890baf8ff2fff81.png) To run other clients you can edit the `docker-compose.yml` file in the working directory. It would be a good idea to backup the default docker-compose file before this. cd charon-distributed-validator-node sudo cp docker-compose.yml $HOME/charon-backups nano docker-compose.yml **Replace the Service** the service needs to be replaced, with the service of the desired client. This includes the relevant docker images and arguments for that client. an example configuration can be found here, for using Nethermind as Execution client in place of Geth. [ ObolNetwork-client-scripts/bia/execution/nethermind at main · GLCNI/ObolNetwork-client-scripts ---------------------------------------------------------------------------------------------- docker-compose script deployments for alternative clients - ObolNetwork-client-scripts/bia/execution/nethermind at main · GLCNI/ObolNetwork-client-scripts https://github.com ![](https://storage.googleapis.com/papyrus_images/0f15cfa5934ae690f2f7d813cfab736f111d78c991eda19e8a8512bae7cccce8.png) ](https://github.com/GLCNI/ObolNetwork-client-scripts/blob/main/bia/execution/nethermind) _NOTE: updating this repository with working docker-compose configurations for various non default clients throughout this testnet and upcoming testnets_ Upgrade Procedure ----------------- For updating Charon, the steps can be followed for updating images for clients also. Can find on the community announcements: releases channel or on Github [here](https://github.com/ObolNetwork/charon/releases). To check current version `sudo docker ps -a` find the version currently running under the `charon` container. ![](https://storage.googleapis.com/papyrus_images/05fbe4ed610f9c9a364489af0142a66174577d5c780bd87c4d9cffa629480330.png) **Stop Node/s** cd charon-distributed-validator-node docker compose down **Pull latest Charon version** 1. Running default configuration sudo git reset --hard sudo git pull 2. Running non-default configuration if you have made specific changes to the config, then follow these steps. Change `Charon` docker image manually nano docker-compose.yml ![](https://storage.googleapis.com/papyrus_images/b01a8bc9a1372fc14fa5432f51fefb9cc94d640fefccb694f8b34d74cb5112e8.png) Navigate the `charon` service and change the highlighted text to the latest version from the git releases repository. **Restart Nodes** docker compose up -d **Check logs** docker-compose logs --tail 100 -f Migration Procedure ------------------- Should you need to migrate a node in the cluster to another server/device the following steps can be followed. **Locate backup files** These are the files that were backed up in Step 6, including the ENR key in Step 3. In total these files are required. If following this guide they should be in folder `$HOME/charon-backups` and **saved somewhere externally** in case of device failure. `deposit-data.json` `cluster-lock.json` `validator_keys` folder containing keystores and keystores.txt `charon_enr_private_key` located in `.charon/` **Setup Charon on new server** Follow setup ‘Step 1’ on the new device to prepare the server Clone charon: git clone https://github.com/ObolNetwork/charon-distributed-validator-node.git Insert the backup files into the required directories **Shut down docker containers in Old server** from Old server cd charon-distributed-validator-node docker compose down shut down old server **Start Charon on New server** cd charon-distributed-validator-node docker compose up -d Exit Procedure -------------- Should you wish to exit your validator, along with the other cluster members this process is called a voluntary exit, and a quorum of operators needs to run the same exit command for the exit to succeed. 1. **Confirm the Exit Epoch** If you want to exit as soon as possible, the default epoch of 162304 export EXIT_EPOCH=162304 otherwise, you can define a future epoch for exiting [https://beaconscan.com/epochs](https://beaconscan.com/epochs) 2\. **Run the Exit Command** This command should be broadcast from your validator client. docker exec -ti charon-distributed-validator-node-teku-1 /opt/teku/bin/teku voluntary-exit \ --beacon-node-api-endpoint="http://charon:3600/" \ --confirmation-enabled=false \ --validator-keys="/opt/charon/validator_keys:/opt/charon/validator_keys" \ --epoch=162304 --- *Originally published on [GLCstaked](https://paragraph.com/@glcstaked/obol-network-bia-testnet)*