Recently, Vitalik Buterin wrote an insightful article, exploring the intersections between AI and Web3, along with the challenges inherent in merging these two domains. It is encouraging to see a thoughtful and complete approach to these often buzzword-heavy topics.
In this post, we will distill some of the key observations from the article and explore what they might mean for the future of AI and Web3 integration.
In contemporary society, there’s a prevalent reliance on centralized services, a phenomenon I like to refer to as centralization dependence. This means that users are generally comfortable trusting centralized entities for various core aspects of their lives. Consider that the majority of media content consumed online is recommended by centralized AI systems. While this centralization isn’t inherently negative, it highlights the lack of widespread, open, and decentralized alternatives, particularly in recommendation systems. The extent of this dependence becomes even more evident when considering services like healthcare, financial services, chatbots, image generation, and social media.
It would be great to complement such reliance on centralization with more open and decentralized protocols to handle such important AI systems. I believe significant changes towards decentralization are unlikely in the (very) near term, but should be achievable in a couple of years. Centralized services often offer lower costs compared to decentralized alternatives, contributing to their wide and spread use.
Those familiar with the Web3 space might wonder why we can’t run AI models on blockchain networks like Ethereum. Unfortunately, this is currently impractical. For example, training a simple MNIST neural network, the most basic AI model of interest, on Ethereum would take about 80 days and cost approximately $32 million. In contrast, this task takes around 8 minutes on an average laptop with negligible cost. Even more efficient blockchains like Solana or Sui can’t handle such computationally intensive tasks on-chain. The need for consensus among a majority of nodes and the requirement for specialized hardware, like interconnected GPUs for larger AI models, are major obstacles.
So, if on-chain AI compute is unfeasible in the near future, what prospects exist for making AI open and decentralized? The answer lies in exploring the concept of verifiable compute, a key notion at the heart of this challenge.
Verifiable compute refers to the ability to run a computation on a machine and obtain not only the output but also a highly reliable attestation (e.g., with 99.9999% confidence) confirming the computation was executed correctly. This assurance is crucial to ensure the integrity of the process, particularly in scenarios where accuracy is paramount.
Consider a recommendation AI model tasked with predicting and recommending the next Netflix movie for your enjoyment based on your viewing history. You want to be certain that the model uses your data accurately and isn’t influenced by irrelevant information, like someone else’s history or data intended for a YouTube recommendation model. However, most centralized systems do not provide verifiable compute attestations. We often rely solely on trust in these centralized entities. This is fine, if the centralized entities interests are aligned with their customer base, which is, to be honest, the majority of cases.
In contrast, blockchains inherently provide verifiable compute as part of their consensus and execution layers, offering a high degree of certainty about the computations. However, this level of verifiability comes at a significant cost, particularly for large-scale computations. Fortunately, simpler operations like value transactions on blockchains are relatively less resource-intensive and more cost-effective.
Centralization, while not inherently negative, allows for significant cost reductions in compute compared to blockchain-based systems. This leads to an important question: Can we strike a balance between the low-cost, non-verifiable compute of centralized systems and the high-cost, highly verifiable compute of blockchains?
The answer is yes, but it involves trade-offs, as is often the case in technology.
Verifiability in computing can be adjusted based on user needs and willingness to pay. Ideally, users should have the flexibility to decide how much they’re willing to invest in verifiable compute for each transaction. For example, when transferring a million dollars via a blockchain like Bitcoin, paying a few dollars for the transaction’s security and verifiability is reasonable. In contrast, paying thousands of dollars for a verifiable guarantee that an AI model, like the one powering ChatGPT, ran correctly for a simple query about Vitalik Buterin biography is excessive.
The challenge lies in the lack of a middle ground. Currently, as Vitalik Buterin points out, users face a stark choice: either opt for low-cost AI compute without verifiable attestations or pay a high price for verifiable compute. There’s no intermediate option offering both affordability and trustworthiness.
Technologies like Zero Knowledge cryptography (ZK) and Multi-party computation (MPC) offer exciting possibilities, but their implementation is still prohibitively expensive. Generating a non-optimized Zero Knowledge proof for a one-second Twitter feed recommendation, for example, could cost around $88,704 using cloud compute hardware. After the amazing work on TensorPlonk, the authors reduce this cost to about $30. However, this is still too expensive for most users seeking to have verifiable Twitter feeds (even though this sounds pretty cool, if you ask me).
So, what’s the solution? The hope lies in the continued improvement and cost reduction of these technologies by scientists and engineers. In the meantime, one approach is to use a subset of nodes in a decentralized network for AI inference or training. For instance, in a network of 1,000 nodes, selecting a smaller number to run the computation can reduce costs. The more nodes involved, the higher the cost. However, this introduces significant challenges in network coordination and consensus, making it hard to envision such decentralized networks operating as efficiently as current blockchain systems.
While we’ve extensively discussed verifiability, it’s important to note that verifiability alone isn’t sufficient to bridge AI and Web3. AI compute often lacks accountability, which contrasts with Web3’s emphasis on verifiable processes. However, verifiability is still a crucial element in integrating these two domains.
Vitalik Buterin, in the referenced article, highlights a significant risk in incorporating AI into Web3: the opaque nature of AI models. Unlike traditional logic programs, where experienced coders can decipher the underlying logic, AI models often act as black boxes. Their outputs are based on complex matrix multiplications and non-linear functions, making it challenging to predict outcomes based on the model’s internal workings alone.
This obscurity presents security concerns, especially when AI models are used in critical decision-making processes. For example, using a Large Language Model like ChatGPT to assist a DAO in making token allocation decisions could be risky. It’s plausible to craft prompts that might mislead participants, leading to adverse social and financial outcomes. The inherent opacity of AI models makes them susceptible to various attack vectors. Even if the AI model considered is open source and has been through a broad auditing process, it is still imaginable that such model is subject to potential attack vectors. Also open sourcing these models, allow everyone to deploy it locally and try to find ways to exploit it.
Some might argue that the complexity and unpredictability of AI models could actually be a defense against exploitation. However, I contend that it’s possible, through prompt engineering, to manipulate LLMs like ChatGPT to undermine entire protocols and security systems. This is just the beginning; fully mitigating these risks is nearly impossible, even with advanced techniques like Reinforcement Learning from Human Feedback (RLHF). To be secure, training data would need to encompass every conceivable vulnerability of each system, which is an impractical task, particularly in the context of DeFi.
The conclusion is that relying solely on large AI models in open and collaborative environments presents significant reliability concerns.
I’m glad you’ve read this far, as the discussion now turns more optimistic and interesting. Given all the points above, I still firmly believe in the value of building open and decentralized AI systems for several reasons.
Firstly, the costs associated with verifiable AI computation can be offset by the value they provide. In scenarios where verifiability is crucial for critical decision-making, the costs can be justified, especially if the decisions have the potential to significantly benefit an entire community. Additionally, advancements in technology, particularly in cryptography, will likely reduce these costs over time. As Vitalik mentioned in his blog post, adding permissions to use AI models can also be a strategy to mitigate some potential critical risks mentioned above. This is a pretty cool idea, especially if it can be used alongside with decentralized digital identity solutions. If you are interested in decentralized digital id and the potential future ahead of us, I strongly recommend taking a look at this article by DC.
On another note, I advocate for the open-sourcing of AI models in the AI and Web3 ‘convergence metaverse’. Contrary to Vitalik’s view, I believe making the internal weights of AI models publicly available aligns with the open-source ethos of Web3. While there’s a risk of model replication for malicious purposes, I argue that AI models relevant to Web3 applications should be of most value to protocols and user communities. Permission mechanisms can effectively mitigate potential risks of malicious usage. Moreover, open sourcing will permit a throughout process of AI safety to be lead by the open source AI and Web3 communities. This synergy between two distinct communities, the AI and Web3 open source communities, will be a win for the entirety of the AI and Web3 space. In my opinion, this is how you foment innovation on a large scale, by bringing specialists on different topics to collaborate on shared purpose use cases.
Vitalik’s article doesn’t address the potential of Retrieved Augmented Generation (RAG) in enhancing state-of-the-art Large Language Models (LLMs) and their possible valuable applications to the Web3 space. RAG allows for the integration of verified factual data into LLM responses, broadening the possibilities for decentralized systems in Web3 applications, DAOs, and even nation-states.
Imagine a DAO that collectively develops a verified knowledge base, accessible only to a selected community of users. Such knowledge bases could be built by various community members, specialized in a variety of topics. Such enterpreneurship could revolutionize fields like scientifical research, social media, on-chain governance, etc. By combining expert human input with advanced AI models it is possible to leverage both human and machine expertise. A popular example in the mathematical community is the Stacks Project, which is an open source project for enhancing and compiling mathematical knowledge on very advanced topics such as algebraic geometry and category theory. Such projects can know be lead on large scale by bridging transparency of trust experts, the right economic incentives and LLMs. In the future, RAG-enhanced LLMs could provide coherent and secure responses within a community, significantly reducing the risks associated with unverifiable AI model training.
Beyond RAG, formal logic or probabilistic programming language can further be used to enhance security across LLM generation. For instance, prompting an LLM to express answers expressed as a probabilistic programming language code and running Bayesian inference on the output can build reliable world models. These models, when accurately validated, can provide a basis to run Bayesian inference on a wide range of possible scenarios allowing communities to derive a range of scenarios given their choices on some immediate or future issues (think of governance, financial decision making, etc). Such Bayesian models can then be compiled and composed across DAOs and build a next era of intelligent composability. This will possibly open the doors for a truly decentralized and broad intelligent system, leveraging both human and machine cooperation, while maintaining a high level of security across the entirety of the stack. These world models, if they have been coherently assessed can provide a true intelligence layer into smart contracts and communities across Web3.
A more down to earth description on how we can enforce security guarantees on LLM generation concerns the use of formal logic to assess and improve AI generated outputs. As an example, LLMs usually do not generate correct code on a single go (trust me, I know), however you can enhance these LLMs with suitable compilers that are able to generate compilation errors on the nose and feed it back to the LLM, creating a feedback loop for improvement. Leveraging this idea to great depths, researchers at Nvidia have shown that you can build AI smart agents to achieve the best performance to date on playing Minecraft.
AI and Web3 represent two distinct realms, each with its own foundational assumptions and problem-solving focuses. However, the intersection of these fields holds immense promise. Web3 can introduce verifiability into AI systems, enhancing trust and transparency, while AI can augment Web3 protocols with a variety of advanced functionalities. This synergy generates substantial excitement about the future possibilities.
Currently, the integration of AI and Web3 faces challenges in terms of efficiency and cost. However, as the technology evolves, these hurdles are likely to be overcome, paving the way for a more seamless merging of these two domains. Additionally, there are promising strategies to mitigate the risks associated with deploying AI systems within the Web3 framework. One such approach involves supplementing LLM inferences with logic programs, thereby enhancing the overall functionality and reliability of these systems.
As we progress, the convergence of AI and Web3 is set to unlock new horizons, bringing transformative changes to the way we interact with technology and each other.