Thanks to @trish for initiating a lively discussion, and thanks to @nounishprof for the hat tip .
I’m one of the 5-15 and this stresses me out. People might not care now, but nobody knows how valuable a fully archived Farcaster will be. Undoubtedly very valuable and possibly most to people who aren’t thinking of it. I’m pretty much in this space for watching history unfold. https://warpcast.com/dwr.eth/0xe06c1d17
As of now, you save everything w/ @farchiver — maybe @artlu can elaborate?
TL;DR
casts are kept live in Hubs
casts are stored in databases
casts are exportable as JSONs
threads are savable as Farchives
Farchiver hosts public and semi-private Farchives
Casts are kept live in Hubs
The Farcaster protocol achieves eventually-consistent consensus on live state. Each FID has a storage allocation, and activities that go over the allocation lead to pruning of older activities. More details elsewhere.
A few of things: 1. You can keep paying for storage and as long as someone is running a valid Hub your content will be preserved. You could even make a smart contract that drips out payments for 100 years. 2. Anyone can pay for someone else's storage ($3 in OP ETH) 3. Back up off protocol on Arweave or onchain https://warpcast.com/dwr.eth/0xa7eb1dbb
Casts are stored in databases
A handful of individuals maintain public databases of cast history. Some will prune, some will not.
The best known example is searchcaster.xyz by @greg. Somebody else (pixel?) received early Purple grants to make their database snapshots public.
Neynar pushes their private custom Replicator dataset to Dune, where it is available to the public via Dune's gating. Airstack provides access to Farcaster data via their own private APIs. Etc.
Casts are exportable as JSONs
It is fast and low-cost (often free) to get your own cast history on your local machine! It is entirely yours, and you can do anything with it, including back it up.
Check out fcau.xyz by @iammathias for an MIT-licensed FOSS Node CLI utility.
Check out fario by @vrypan for a powerful CLI tool in python, also MIT-licensed.
Check out @pixel's guide. Neynar is not free, but the one-month cost is less than one year of Farchiver.
All of these are really fast. The actual process to download full history takes less than ~2 minutes for most users. The setup may be intimidating to some, but is not super difficult for experienced devs.
To find free, open Hub endpoints at which to point these utilities, check out the list we maintain at foss.farchiver.xyz.
Threads are savable as Farchives
Farchiver lets you download every thread you've ever touched, with all interactions, in a delightful package. We suggest Obsidian for browsing; it can be any Markdown-aware knowledge tool.
We believe this is more interesting than just one's own casts. We charge 0.03 ETH for one year of this unique service.
You may purchase a snapshot of any FID.
Farchiver hosts public Farchives
In the public interest, Farchiver hosts Farchives, pinned on IPFS and available for the community to preserve collectively.
Examples:
the proto channels thread, NBA game 5
an epic @keccers.eth thread, requested by @shazow.eth
a month of @dwr.eth interactions, around the wowow meme
Farchiver also enables private, sharable Farchives of threads of interest. Each user has access to a section of private Farchives (possibly encrypted) behind a wallet login. This includes Farchives that others have paid for, to share with others.
As an example, one caster wanted to own the toxic interaction they had with a powerful account. It had been deleted, and only pieces of it had been screen-shotted without the full context. Now this user has a copy of the deleted abusive casts, and this Farchive can be shared with others as a semi-private, but transparent and permissionless record.
I’m one of the 5-15 and this stresses me out. People might not care now, but nobody knows how valuable a fully archived Farcaster will be. Undoubtedly very valuable and possibly most to people who aren’t thinking of it. I’m pretty much in this space for watching history unfold.
You can use @farchiver to download everything if you’d like. Could do this periodically to ensure everything is archived.
The thing about archiving is it’s gotta be everything. Who knows what will be important in 20 years. I don’t know if that’s affordable or not but certainly willing to pay if I can. If you look at web.archive, it’s a public good. Seems like something a separate DAO should be doing and keeping it accessible
As of now, you save everything w/ @farchiver — maybe @artlu can elaborate?
It's far from perfect, but this free, permanent archive concept is one of the main reasons we built this. Would love any thoughts on how we could make this a more official public good For instance, adding columns for `pruned_at` is on our short term list https://warpcast.com/runninyeti.eth/0xd02f9bc1
Me too!
also in the 5 - 15% and am willing to pay more for storage tbh
My statement isn’t about an unwillingness to pay for it (depending on price). Archives from a significant digital town square will be of public interest some day. Web.archive is a public good because what’s happening here will be of public interest
You probably already know this but my understanding is that as long as you pay for hub storage it should be kept indefinitely. You can check your usage here if needed caststorage.com
🙋♂️ 5-15% reporting for duty. Shoe box 📦 in tow.
I've said a lot of dumb stuff in the past. My greatest fear… blackmail.
Would archive nodes solve this for you? Even if they're not decentralized (could very easily make them so)
I don’t know. I’m not sure what that involves/ how it would work
Would basically be third parties storing cast history separate from the primary network and clients can choose to show those historical casts
Would you want to store just your information or have n number of degrees of interaction as well?
The nerd in me would love daily or weekly or monthly snapshots of everything. Because you don’t know what will be important down the line. I think web archive takes user archive snapshots of individual tweets or accounts going back to a certain date.
“Everything” as in the whole network?
I'm also in the 5-15%, and I think the real percentage is much higher than that.
Also in the 5-15%. Like @tomato.eth I think the precent is much higher. There will come a time when people will care deeply about digital legacy. Something will help each of us curate that. It is what will live on beyond our physical form. I'm a digital hoarder and I approve this message.
Digital hoarders unite. I have every single text conversation I’ve ever had archived.