Arweave TX
HashBrown Research
Table of Contents
Why Decentralized Data Storage
Some Data Regarding Decentralized Storage Solutions
Types of Decentralized Storage Solutions
IPFS, Pinata & Filecoin
4.1 Filecoin Fees, Storage Process & Interface
4.2 Deals; The Core of Filecoin Storage
4.2.1 Proof-of-Replication:
4.2.2 Proof-of-Storage:
4.2.3 Proof-of-Spacetime:
Arweave
5.1 The Blockweave
5.2 Arweave Interface
5.3 The “Backbone” of Arweave; Storage Endowment
5.4 Arweave Fees
5.5 Summary of Key Features of Arweave
Adoption of Decentralized Storage Platforms
Food For Thought
7.1 Filecoin Sentiment
7.2 Arweave SentimentWhy Decentralized Data Storage
Having complete control over your data is something that is often overlooked, but important. There have been many who have been hurt by placing too much trust on Centralized Entities.
Of course, Decentralized Platforms has its own problems… such as smart contracts exploits etc., but we rather entrust our data to immutable code which we can verify if it is malicious, with the code being open source, rather than centralized entities which can go rogue anytime.
An example would be the recent 3Commas API Key Leak //
Anonymous Twitter User Leaks 3Commas API Database
https://filscan.io/#/tipset/chain
https://dashboard.starboard.ventures/market-deals
https://storjstats.info/d/storj/storj-network-statistics?orgId=1
Built by Protocol Labs, Filecoin and IPFS are separate protocols.
InterPlanetary File System or otherwise more commonly known as IPFS, is a file sharing technology, when you add a file to IPFS you will get a CID (Content Identifier).
Although IPFS is free, there is no guarantee it's hosted on some node forever. Eventually, the node cache becomes full, and then garbage collection happens. This means that the node empties its cache to make room for more content which could cause your file to “disappear” if they are not accessed for a while.
Other than hosting your own node, people use Pinata to "pin" a file on IPFS, or use Filecoin to incentivise people to host files.
Pinning means that you are telling the node that the file is important and to store it. Pinata is a pinning service, they will keep your files pinned and anyone with the CID can obtain the file.
To store data, we need to find “Storage Providers” to make a deal.
The Filecoin network allows storage providers to compete by offering different terms for pricing, acceptable data sizes, and other important deal parameters.
It’s also important to consider the storage provider’s location; the closer the storage provider is to you, the faster the storage and retrieval process will be.
Source: Store data - Filecoin Docs
Before going into deals, we will introduce some concepts of Filecoin.
A novel Proof-of-Storage which allows a server (i.e. the prover P) to convince a user (i.e. the verifier V) that some data D has been replicated to its own uniquely dedicated physical storage.
Proof-of-Storage schemes allow a user to check if a storage provider is storing the outsourced data at the time of the challenge.
Where a verifier can check if a prover is storing her/his outsourced data for a range of time.
Sources:
Filecoin: A Decentralized Storage Network, Protocol Labs, July 19, 2017
Below is a simplified explanation of Storage Deals and Retrieval Deals.
The Blockweave is ran by miners who are providing disk space and replicating the data stored in the network. They are incentivised by earning AR Tokens.
Up till Block 633720, Arweave was on the Proof of Access Consensus Mechanism, but since then have changed to Succinct Proofs of Random Access to align mining profitability with speed of data access, and to reduce the energy required to maintain the network.
Proof of Access was until Block 633719:
Basically Proof of Access: Proof of Work + Includes Reference to past data.
PoA achieved Arweave’s primary goal of permanent storage and data access, but it imposed no significant incentives for miners to retrieve data quickly. Without a competitive edge to data retrieval, miners found an opportunity to benefit from using a remote storage pool rather than maintaining separate, decentralized nodes.
Source:
Succinct Proofs of Random Access (SPoRA) started from Block 633720 approx Feb 24 2021:
SPoRA doesn’t depend on the previous block to validate transactions but rather uses the previous block and a random block in the chain as the challenge to miners.
Source:
SPoRA requires that they include a random byte of data from a past block with each hash cycle. If they don't have the byte they have to retrieve it from other nodes on the network and potentially miss out on mining the block to a node that has the data stored. This incentivizes miners to store as many replicas of the data as they can and keep them close to the CPU.
Source:
Arweave Dev Discord
The SPoRA consensus algorithm swaps much of the compute that used to be needed for mining blocks for disk reads. This means miners will be incentivised to keep many more replicas of data in the network, and it will consume less electricity.
Because it is "random" miners will be incentivised to keep many more replicas of data in the network, and it will consume less electricity.
// The protocol generates random challenges for chunks of data from the entire Arweave data set.
// The more of the dataset the miners store the better their mining performance
the randomly selected "chunks" are 256kb chunks of pictures/videos/encrypted content etc.
The more you store = the better the mining performance.
Furthermore, with the Blockshadowing System, which decouples transactions from blocks, and only sending a minimal ’blockshadow’ between nodes which allows peers to reconstruct a full block, rather than transmitting the full block itself, allows transactions to be mined into a block as fast as they can be distributed around the network, and consensus about blocks to be achieved at near network speed.
Source: Arweave Yellow Paper
Users pay a one time payment initial cost to upload data to the Arweave network, which covers the first 200 years of storage.
The idea is based on the assumed decreasing storage prices over time.
Over the past 50 years, data storage has seen its costs decrease by an average of 30.5% per year.
Source: https://ardrive.io/topic/arweave-economics/
Most of the transaction fee is contributed towards a storage endowment, which is distributed to the wallets of miners over time to incentivise this perpetual storage.
Miners earn tokens by replicating the data in the network and constantly testing its validity.
You can read more in detail about the Endowment Simulation here: https://arwiki.wiki/#/en/endowment-simulation
As at: UTC 2023-01-13 05:06:00, an estimated amount of 0.1346% of the current circulating supply is in the endowment.
Dataas at: // UTC 2023-01-13 05:06:00 //
You can view the real time fees to store data permanently on Arweave here:
Arweave Fees
Filecoin also has an initiative known as Filecoin Green, which aims to measure the environmental impacts of Filecoin and verifiably drive them below zero, building infrastructure along the way that allows anyone to make transparent and substantive environmental claims.
(Source: https://green.filecoin.io/)
You can view the energy consumption rate here: Filecoin Energy
As at 11 Jan 2023, we can see that the energy consumption rate has been steady since Q3 of 2021, with no sustainable increase. We feel that this attempt at trying to acknowledge the concerns of users who might be concerned about the environmental impacts shows that they have a clear and well thought out plan.
The customization with Filecoin Deals also enables users with access to more competitive rates to store their data.
However, we note that Arweave is more suited for Web3 Platforms, in the sense that it is based on a blockchain “model”, rather than based on a decentralized “hard disk”, such as Filecoin.
Due to the permanent nature of Arweave, it also works as a double-edged sword, in the sense that illegal content could be hosted there and could cause regulators to “take necessary actions” to shut it down if it becomes too rampant.
However, Arweave seems to have come up with a way to address this issue quite effectively.
Arweave's blockweave data structure ensures that miners are not required to store the entire dataset, and miners are able to run arbitrary computation (from simple keyword searches, to computer vision image and video analysis) upon all data they store, so that they can remove content they are not comfortable with hosting.
Source: https://arwiki.wiki/#/en/content-policies
Hence, we at HashBrown Research feel that Arweave has a good edge for Web3 Projects who are in it for the long term, to build on.
Have you thought about which platform you would choose to store your data?
Feel free to connect with us on our Socials as we would like to hear your thoughts!