# Create Farcaster Postgres mirror

*Script to replicate Farcaster raw data into a new database. Stream data from a Linux server to a Postgres DB*

By [hellno](https://paragraph.com/@hellno) · 2023-07-11

web3, dev, farcaster

---

I needed a copy of Farcaster raw data for a new project building on top of the network. There were some hiccups for me to get started, so I've compiled a script.

I replicated the Farcaster data into a new DB in AWS and am running this script in an EC2 instance. It can stream data from any Linux server to any Postgres DB ([fly.io](http://fly.io), Supabase, etc).

This is _not_ a script to run a hub to support the Farcaster decentralization, because it downloads data and does not sync anything back into the network.

**the script**

    # install basic tools
    sudo yum install git tmux htop
    
    # install nvm 
    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
    
    # activate nvm
    export NVM_DIR="$HOME/.nvm"
    [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
    
    # install node 
    nvm install 20
    npm install --global yarn
    
    # get farcaster repo
    git clone https://github.com/farcasterxyz/hub-monorepo.git
    
    # configure script to use DB
    export POSTGRES_URL=postgres://USER:PASSWORD@URL/DATABASE
    
    # go to directory
    cd hub-monorepo/packages/hub-nodejs/examples/replicate-data-postgres
    
    # install
    yarn install 
    
    # run
    yarn start

If you want to restart the sync after stopping or rebooting:

    ## go here after new session or restart 
    
    # configure script to use DB
    export POSTGRES_URL=postgres://USER:PASSWORD@URL/DATABASE
    
    # go to directory
    cd hub-monorepo/packages/hub-nodejs/examples/replicate-data-postgres
    
    # get changes to repo
    git pull
    
    # run
    yarn start

**Notes**

*   the Supabase free tier has not enough storage to get a full data copy
    
*   AWS details: EC2 instance is `t2.small` and smallest possible RDS postgres instance

---

*Originally published on [hellno](https://paragraph.com/@hellno/farcaster-db-mirror)*