Hi everyone! Ulad aka ulad_proto is here.

One of my responsibilities in the Protofire DevOps department is to find new and exciting projects. And today we gonna go through one of these - the Bifrost testnet from the organizational and technical sides.

Org side - it was perfect, the whole testnet was divided into 4 weeks with its primary purpose and tasks. I’ll describe these phases and details about the work I have done one by one.

As the first thing - I’ve checked the architecture of the blockchain. There are two types of nodes - basic and relayer nodes:

• basic node - stake BFC to verify and create blocks, get rewards for this work

• relayer node - all the features of the basic nodes and additionally facilitate Cross-Chain Communication Protocol or CCCP.

Few words about CCCP, because this is the main feature of the Bifrost. The main idea - the team has deployed socket smart contracts into the supported blockchains:

• The Bifrost testnet itself

• ETH Goerli, Polygon Mumbai

• BNB testnet

The relayer node is listening to the events in these blockchains using ABIs of the smart contracts and becoming a bridge between connected blockchains using CCCP. Additionally, the relayer checking prices from the Oracles: Coingeko, Upbit, Chainlink, and Bitcoin block hash.

Relayer itself is a python multi-thread app with two threads:

• monitoring - monitors CCCP events

• sender - sends transactions into connected blockchains

Cool diagram from the team that can help you to understand it better:

1. I’ve checked that the Bifrost network isn’t listed on chainlist.org - an aggregator of ETH-like blockchains, you can add a new network in one click with all information like - RPC, currency, explorer, and so on in one click.

   There was a straightforward way to help the team, I opened PR with all information about Bifrost, and, the team thanked me for this and started to use it in chats:

   https://chainlist.org/chain/49088

2. Created a docker-compose configuration for both validator and relayer nodes to simplify and speed up the deployment process for the Bifrost community. I’ve shared this idea with devs and it was added to the re

3. Internal monitoring. As per above, participation in the Bifrost network involves direct communication with several other networks. To make our connection reliable I have configured internal monitoring that tracks down the status and uptime of RPC endpoints. For BTC networks I even spun up our node because I haven't found any reliable external RPC provider.

   I was thinking about how I can add this, haven’t found opened RPC and just spin up our own using the kylemanna/bitcoind image, also added it to the monitoring.

4. I set up our in-house monitoring and find out that I was under-using our resources so I moved to AWS:

   from m5zn.2xlarge (8 CPU, 32 GB) - 0.4162$ per hour for reserved

   to m5zn.large (2 CPU, 8 GB) - $0.10 per hour for reserved

   Total savings: 228$ per month

   Сonclusion: always double-check the real resource usage of your setup - it can save you a lot of money. Please also consider using reserved instances or analog in your VPC provider it is also a way to save your costs.

5. And I was helping the community with its problems, the team even granted me a Testnet support a role for me.

6. I've found that there was no .dockeringnore file in the repo - I’ve added one and shrunk the docker build context from 178MB to 1.7MB and speed up the build time.

There were tasks for testing Intra-chain Transactions and Bridges.

It was a great time to learn a little bit about the web3 library and how to communicate with blockchain from code. Really good article for beginners:

https://docs.moonbeam.network/builders/build/eth-api/libraries/web3js/

The most interesting part was to temporarily shut down nodes and check how the blockchain will behave during this time frame.

• There was an interesting issue with the relayer - the monitoring thread had died, but the sender was still there. - I’ve suggested that the application should be restarted if one of the treads is unhealthy and devs have implemented this.

• Also, I’ve suggested showing somehow how the validators’ election round is going and it was a real lack of this information, it was important because this is how you can know when you will go from the Candidates to the Validators list.

  And guess what - devs heard me and added this cool visualization on the main page - props to devs for listening community:

The main task was adding a new external chain - Avalanche Fuji chain. It was interesting how smoothly the addition of the chain went - a single PR into the repo did the job well. Again all the heil to the dev team!

It was also an election task where validators can vote for new community proposals like setup a commission rate for CCCP transactions, it can help to community rule the Bifrost network together.

I’ve faced again with died thread issue that was restarting every 60 seconds thus the relayer was unhealthy, but I couldn’t catch this on the monitoring because the python app was alive. I’ve created an issue for the team about this, I believe the full app should be crashed exit code 1 if something goes bad.

https://github.com/bifrost-platform/bifrost-relayer/issues/11

• Additionally, I’ve rewritten Dockerfile with ENTRYPOINT and CMD commands with JSON form - now the app is started as one PID and will listen to the Linux signals like SIGHTERM and can stop gracefully

https://github.com/bifrost-platform/bifrost-relayer/pull/9

I’ve enjoyed the level of the Bifrost testnet organization and the idea behind it. I like how tasks were described and helped with technical and organizational problems.

However, I believe it can be several improvements for the Bifrost from the technical side:

• create a docker-compose file for the full nodes validators - to simplify the nodes deployment. Yes, there is a setup for the relayer and node.

  It can be enlarged and cover Matic, Ethereum, Avax, and BNB local nodes setup as well as preparation for the mainnet.

• BFC node and relayer health monitoring.

  Right now this is painful for me as a validator to monitor my node. The best idea here is to create a default Node and Relayer monitoring setup based on a popular Grafana stack, e.x. start to share Prometheus metrics, include them in docker-compose, and deploy everything as one stack.

Nevertheless, I believe that team will head the community as it was before and add these ideas to the Validators setup stack.

See you all in the Bifrost mainnet, dudes, it was a cool month, thanks to everybody - devs and support team, and other full node validators!

If you need help with your infrastructure architecture and its setup in any environment, CI/CD, monitoring or you need to consult, please ping me or drop a mail to devops@protofire.io and we will be glad to help.

Best wishes,

Ulad aka ulad_proto aka 0x4CAd798bdeAabD117687E28C4b868769C81ecA62