Author: Michael Gingras (@0xmcg)

Modern software development moves quickly, and the ways we conceptualize, design, and deploy software applications have undergone significant change. Among the most revolutionary advancements has been the emergence of "Infrastructure as Code" (IaC), a method in which a singular configuration file delineates the infrastructure necessary for a deployment. Yet, as monumental as this development is, there emerges an even more innovative evolution: Framework Defined Infrastructure (FDI).

At its core, FDI stems from the observation that modern applications, in many instances, adhere to discernible and consistent patterns. This realization permits us to translate framework concepts directly into the appropriate infrastructure without manually describing explicit configuration that comes with the likes of IaC. This abstraction offers unparalleled advantages. For novice developers, this translates to a world where they can prioritize the functionality of applications (the 'what'), rather than becoming entangled in the complexities of infrastructure operations (the 'how').

An example of FDI is Vercel’s flagship framework, Next.js. What makes Next.js a quintessential example of FDI is its ability to automate the deployment of infrastructure needed for many common web related tasks. For example, developers often need to deploy API endpoints. Traditionally a developer might have deployed an entirely separate service to host an API. However, in Next.js, this process is reduced to simply creating a new file in an `api` directory, which the next converts into a dedicated API endpoint that is callable from the application. Next also uses FDI to offer performance gains that might otherwise have been too challenging for normal developers or smaller start-ups to realize. For example, using the <Image /> component that next provides automatically pushes these images through a complex image optimization pipeline that improves performance for free. Independently setting up such a pipeline would likely be unfeasible for smaller companies, but with Next.js this comes for free.

While Next.js utilizes FDI to improve the experience of developing traditional web apps, there is potential for framework authors to create frameworks that are purpose built for specific domains that have specific infrastructure needs. One example of a domain that has specific infrastructure needs is web3.

In web3, most data is stored on-chain and lives outside of a database that an application developer controls. Developers are able to fetch this data, but often choose to deploy “off-chain” databases alongside their application to complement or duplicate data that lives on-chain. They might do this for a few reasons:

• Speed limitations: Reading on-chain data can be slow.

• Data completeness: An application might rely on data that does not make sense to store on-chain.

• Lack of structured data: Direct on-chain querying isn't straightforward due to the unstructured nature of the data.

Hosting a database is a big win. However, syncing on-chain data to a managed database presents two main challenges:

• Transaction confirmation delays: On-chain data can possibly take minutes to process. Writing to the database before data is finalized on-chain could lead to discrepancies if the on-chain transaction fails.

• External interactions: If an on-chain transaction is triggered outside of the application, there is no way for our local database to pick up on this new data. While this challenge is less frequent than the first, its implications are profound.

To solve this problem, it is typical to seek out the help of an indexer. An indexer listens for events as they happen on-chain, and often duplicate the data in a more convenient store like a traditional database where the data is easily available and queryable. There are even indexers like Spec which can listen for on-chain events and directly write to an application’s database.

Indexers like Spec work well. However, we might consider that using tools like Spec is akin to setting up our own API server in a pre-FDI world. Using an indexer is a common, predictable solution that relies on dedicated infrastructure to solve the problem. Using third party indexers is cumbersome and doesn’t allow for an application’s type system to transcend the boundary between application code and indexer code. Might it be possible that a framework designed specifically for the web3 industry exists to remedy some of these problems? Imagine a framework that offers an `/index` directory that lets developers define an event they want to respond to.

For example, imagine might have a contract event Transfer(address,address,uint256) that represents transferring an ERC721 between accounts. We could create a function at /index/transfer(address,address,uint256) with a callback function that automatically runs whenever this event is fired on-chain. Since this is contained within our framework, we get our type system, our database, and we don’t have to deal with managing two separate projects. Behind the scenes, our framework would compile the code and translate each file in the `index` directory into a callback that is registered to an indexer that is hosted by the framework provider. Doing so creates a much more seamless developer environment for folks wanting to build in web3.

An index directory mapping routes to on-chain event callbacks is just one example of framework defined infrastructure applied to web3. Web3 is full of idiosyncrasies that make development challenging and of which a framework could likely solve for. In a hypothetical Web3-oriented framework, directories such as /contracts/ would handle contract deployments, with automated infrastructure in place for ABI management and deployment scripts. As described above, directories such as /index/ could be responsible for contract indexing through event handlers. Other directories like /relay/ and /sign/ would seamlessly integrate with on-chain transaction execution and EVM-compatible key management services. Furthermore, any files under /web3/public/ would be automatically directed to IPFS and pinned, ensuring decentralized access. This is a glimpse into how framework-defined infrastructure can significantly simplify and automate the Web3 development process, allowing developers to focus on innovation rather than infrastructural intricacies.

Station Labs is building GroupOS, a smart account toolkit to activate users, distribute rewards, and power growth programmatically. If you’re interested in following along our journey and building with us, you can follow our Twitter, check out our documentation, or join our Discord.

Author: Michael Gingras (@0xmcg)

Modern software development moves quickly, and the ways we conceptualize, design, and deploy software applications have undergone significant change. Among the most revolutionary advancements has been the emergence of "Infrastructure as Code" (IaC), a method in which a singular configuration file delineates the infrastructure necessary for a deployment. Yet, as monumental as this development is, there emerges an even more innovative evolution: Framework Defined Infrastructure (FDI).

At its core, FDI stems from the observation that modern applications, in many instances, adhere to discernible and consistent patterns. This realization permits us to translate framework concepts directly into the appropriate infrastructure without manually describing explicit configuration that comes with the likes of IaC. This abstraction offers unparalleled advantages. For novice developers, this translates to a world where they can prioritize the functionality of applications (the 'what'), rather than becoming entangled in the complexities of infrastructure operations (the 'how').

An example of FDI is Vercel’s flagship framework, Next.js. What makes Next.js a quintessential example of FDI is its ability to automate the deployment of infrastructure needed for many common web related tasks. For example, developers often need to deploy API endpoints. Traditionally a developer might have deployed an entirely separate service to host an API. However, in Next.js, this process is reduced to simply creating a new file in an `api` directory, which the next converts into a dedicated API endpoint that is callable from the application. Next also uses FDI to offer performance gains that might otherwise have been too challenging for normal developers or smaller start-ups to realize. For example, using the <Image /> component that next provides automatically pushes these images through a complex image optimization pipeline that improves performance for free. Independently setting up such a pipeline would likely be unfeasible for smaller companies, but with Next.js this comes for free.

While Next.js utilizes FDI to improve the experience of developing traditional web apps, there is potential for framework authors to create frameworks that are purpose built for specific domains that have specific infrastructure needs. One example of a domain that has specific infrastructure needs is web3.

In web3, most data is stored on-chain and lives outside of a database that an application developer controls. Developers are able to fetch this data, but often choose to deploy “off-chain” databases alongside their application to complement or duplicate data that lives on-chain. They might do this for a few reasons:

• Speed limitations: Reading on-chain data can be slow.

• Data completeness: An application might rely on data that does not make sense to store on-chain.

• Lack of structured data: Direct on-chain querying isn't straightforward due to the unstructured nature of the data.

Hosting a database is a big win. However, syncing on-chain data to a managed database presents two main challenges:

• Transaction confirmation delays: On-chain data can possibly take minutes to process. Writing to the database before data is finalized on-chain could lead to discrepancies if the on-chain transaction fails.

• External interactions: If an on-chain transaction is triggered outside of the application, there is no way for our local database to pick up on this new data. While this challenge is less frequent than the first, its implications are profound.

To solve this problem, it is typical to seek out the help of an indexer. An indexer listens for events as they happen on-chain, and often duplicate the data in a more convenient store like a traditional database where the data is easily available and queryable. There are even indexers like Spec which can listen for on-chain events and directly write to an application’s database.

Indexers like Spec work well. However, we might consider that using tools like Spec is akin to setting up our own API server in a pre-FDI world. Using an indexer is a common, predictable solution that relies on dedicated infrastructure to solve the problem. Using third party indexers is cumbersome and doesn’t allow for an application’s type system to transcend the boundary between application code and indexer code. Might it be possible that a framework designed specifically for the web3 industry exists to remedy some of these problems? Imagine a framework that offers an `/index` directory that lets developers define an event they want to respond to.

For example, imagine might have a contract event Transfer(address,address,uint256) that represents transferring an ERC721 between accounts. We could create a function at /index/transfer(address,address,uint256) with a callback function that automatically runs whenever this event is fired on-chain. Since this is contained within our framework, we get our type system, our database, and we don’t have to deal with managing two separate projects. Behind the scenes, our framework would compile the code and translate each file in the `index` directory into a callback that is registered to an indexer that is hosted by the framework provider. Doing so creates a much more seamless developer environment for folks wanting to build in web3.

An index directory mapping routes to on-chain event callbacks is just one example of framework defined infrastructure applied to web3. Web3 is full of idiosyncrasies that make development challenging and of which a framework could likely solve for. In a hypothetical Web3-oriented framework, directories such as /contracts/ would handle contract deployments, with automated infrastructure in place for ABI management and deployment scripts. As described above, directories such as /index/ could be responsible for contract indexing through event handlers. Other directories like /relay/ and /sign/ would seamlessly integrate with on-chain transaction execution and EVM-compatible key management services. Furthermore, any files under /web3/public/ would be automatically directed to IPFS and pinned, ensuring decentralized access. This is a glimpse into how framework-defined infrastructure can significantly simplify and automate the Web3 development process, allowing developers to focus on innovation rather than infrastructural intricacies.

Station Labs is building GroupOS, a smart account toolkit to activate users, distribute rewards, and power growth programmatically. If you’re interested in following along our journey and building with us, you can follow our Twitter, check out our documentation, or join our Discord.