The Treasury operates based on two core principles:

1. Funding Public Goods: Proposals must demonstrate ecosystem-wide benefits rather than focusing on individual gains.

2. Special Purpose Entities (SPEs): These entities manage and distribute funds, ensuring resources are allocated efficiently and with domain-specific expertise.

How the Treasury Works

The Livepeer Treasury empowers LPT holders to actively participate in governance by proposing and voting on funding allocations. The governance framework emphasizes accountability and transparency, ensuring that every LPT allocated contributes to the network's growth and decentralization.

Special Purpose Entities (SPEs) play a crucial role by focusing on specific areas such as video AI, large language models (LLMs), and decentralized cloud infrastructure. Each SPE:

• Manages funds allocated by the Treasury.

• Drives innovation in its focus area.

• Reports progress and ensures accountability to the community.

Community Treasury: 2024 in Review

2024 has been a transformative year for the Livepeer ecosystem, driven by the strategic use of the Onchain Treasury to fund innovative initiatives through Special Purpose Entities (SPEs). These SPEs have played a pivotal role in advancing the network’s mission by focusing on specific areas such as AI-powered video processing, decentralized cloud infrastructure, and cutting-edge large language models.

Let’s review the key SPEs funded by the Treasury, highlight their progress, and provide updates on their milestones. These insights showcase the tangible impact of decentralized governance and how community-driven funding supports public goods, fosters innovation, and shapes the future of Livepeer.

AI SPE: Driving Livepeer’s AI Innovation in 2024

In 2024, the AI Special Purpose Entity made significant strides in establishing Livepeer as a leader in AI-powered video compute. Key achievements included implementing the core network upgrades needed to enable AI jobs on the Livepeer network and introducing the first generative, batch, and real-time AI pipelines, such as T2I, Whisper, Segment Anything 2, and real-time artistic video animation. These advancements, combined with clear metrics, comprehensive documentation, and developer-friendly SDKs, validated the potential of these new capabilities through successful demos and real-world adoption by early builders and developers. Ecosystem programs such as the Software Bounty Program, AI Startup Program, and the inaugural AI Hackathon fostered community engagement, open innovation, and impactful use cases.

The AI SPE leverages Livepeer’s unique strengths in decentralized media compute by adopting a demand-driven strategy and prioritizing real-time AI video workflows. While challenges such as container inefficiencies persist, the SPE’s focus on refining real-time services and expanding both generative and batch AI capabilities provides a solid foundation for growth in 2025. Through strategic financial management, a commitment to overcoming technical challenges, and close collaboration with the community, the AI SPE has positioned Livepeer for broader adoption and sustained innovation within its ecosystem.

Cloud SPE: Enhancing Accessibility and AI Integration

The Cloud Special Purpose Entity made notable progress in increasing accessibility to the Livepeer network through two major proposals. The first focused on expanding the Livepeer.Cloud Gateway, integrating with Owncast, and simplifying gateway operations. This led to an increase in Gateway Nodes from 8 to over 35 and the transcoding of 853,000 minutes, with approximately $2,250 in fees paid. While user adoption presented challenges, this initiative laid the foundation for improved analytics and reporting.

The second proposal prioritized advancing AI capabilities within the Livepeer network. Key achievements included processing 155,000 AI jobs, generating over $9,250 in fees, and introducing tools like the AI Performance Leaderboard, Performance Inspector, and AI Job Tester. These tools, alongside enhanced metrics and developer resources, strengthened Livepeer’s ability to support scalable and innovative applications, solidifying the network's position in the decentralized media landscape.

LLM SPE: Pioneering AI Solutions for Seamless Developer Experiences

Launched in September 2024, the LLM Special Purpose Entity (SPE) has achieved significant milestones as the first SPE dedicated to a specific AI pipeline. Within a few months, it delivered a highly performant LLM pipeline comparable to centralized incumbents, featuring an API identical to OpenAI's SDK. This familiar interface simplifies onboarding for developers and makes it ideal for RAG applications and AI agents.

With core engineering goals accomplished, the LLM SPE is now focused on driving demand through its AI gateway. Efforts will center on education, open-source tooling, and subsidizing costs for app and agent developers. These initiatives aim to foster adoption and position Livepeer's LLM capabilities for growth in 2025.

Aquareum: Advancing Open Source Video Technology

Aquareum has made significant strides over the past several months, shipping a range of open-source tools to revolutionize video technology. Key achievements include the Aquareum Stream Segment Primitive, enabling secure and trustless video replication via C2PA-signed MP4 files, and the versatile Aquareum Node, which supports multiple streaming formats like WebRTC, HLS, and Progressive MP4. Available across major platforms, the node leverages GStreamer for efficient video processing and simplifies adoption through its statically linked binaries.

Additionally, Aquareum delivered a robust suite of applications, including the Aquareum App for web and mobile platforms and the Aquareum Desktop bundle, which combines node and app functionalities for seamless livestream signing and local development. Supporting tools like libraries for C2PA signing and OAuth integration with Bluesky were also released. Moving forward, Aquareum aims to finalize livestreaming on the AT Protocol and showcase its product launch at nwHacks 2025, driving adoption and innovation in decentralized video technology.

Live Pioneers: Bridging Delegators and the Livepeer Ecosystem

Live Pioneers cemented its position as a vital hub for the Livepeer community, enhancing engagement through the launch of the Live Pioneers App and bolstering global community growth. The app, introduced in July and now in its third version, provides delegators with advanced tools for tracking rewards and exploring the Livepeer ecosystem. Continuous updates have refined its functionality, making it a cornerstone for participant engagement.

Beyond the app, Live Pioneers expanded its global footprint, driving educational initiatives and creating multilingual content to reach diverse audiences. With a 4x increase in Telegram activity, over 83K impressions on X, and growing content viewership across YouTube and the blog, the platform has deepened community connections worldwide. Regional moderators and translated guides have played a key role in fostering inclusivity and accessibility, setting the stage for even broader reach in 2025.

Your Role in Treasury Governance

Whether you’re an LPT holder, developer, or enthusiast, you can contribute to shaping Livepeer’s future:

• Propose New Initiatives: Bring your ideas to life by drafting proposals for Treasury funding.

• Participate in Treasury Talks: Join discussions, provide feedback, and inspire innovation.

• Vote on Proposals: Help decide which initiatives receive funding and support.

For more resources, visit:

Livepeer Forum: Engage with the community and discuss proposals.

Livepeer Voting Portal: Vote on active proposals and track progress.

Livepeer Discord Governance Channel: Collaborate and connect with other participants.

Looking Ahead

The Livepeer Onchain Treasury exemplifies how decentralized governance can fuel innovation, support public goods, and drive the network’s growth. The first Treasury Talk is just the beginning—an opportunity to engage with the community, explore new ideas, and set the stage for the next wave of Livepeer’s evolution.

Join us in building a decentralized future for video streaming and beyond. Together, we can make an impact!

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

I’m excited to introduce a vision for the next phase of the Livepeer Network - Cascade. This post will aim to:

• Provide a brief background on the 7+ year history of Livepeer from a product perspective.

• Introduce Cascade, a vision for how Livepeer can be the leading platform and community for the future of real-time AI video.

• Explain why this is a unique opportunity for Livepeer that is well suited to the project’s strengths.

• Outline the focus this can bring to the community and lay out practical next steps.

A Brief Journey Through A 7 Year Livepeer History

In the nearly 7 years that the Livepeer Network has been live, it’s gone through many successive phases that have increased the capabilities of the project, scaled the project, furthered decentralization of the project, and enabled sustainable public goods funding for the project. Through this iteration, Livepeer has delivered many core utilities to build the world’s open video infrastructure.  This has been evidenced by the millions of minutes/week of video transcoding and AI media inference it performs across hundreds of applications built by developers building toward the future of AI media. It generates fees in both ETH and LPT split across hundreds of hardware contributors who run nodes and GPUs on the network. It is one of the most widely used practical projects that pioneered the DePIN category.

While the raw utility, strong community, and demonstrated use cases on top of the network are all positive signals relative to many other early decentralized tech projects, it is still fair to say that the Livepeer Network as a product hasn’t yet found that clear breakout fit that drives growth momentum and inbound interest at an exponential rate. Transcoding and generic AI inference alone, with their value props of cost-effectiveness and open access, remain critical elements of the video infrastructure of the future but haven’t yet been bundled into the must-have-hair-on-fire-solutions that send hundreds of potential users at once clamoring for direct integration with the network. As such, it’s critically important that the Livepeer community continue to iterate and experiment with various productization theses that ensure that the network delivers must-have-utility for a broad and fast-growing market - one that can provide orders-of-magnitude fee growth on the network and a thriving community coming together to build the project toward the future to support this vision.

I’m excited to introduce this proposal for the next phase of the Livepeer Network: Livepeer Cascade. At its core, Cascade proposes that during this phase, the project focuses on building the leading platform and community for the future of real-time AI video pipelines.

Livepeer Cascade Vision - The Leading Platform and Community for Real-Time AI Video Pipelines

For those paying attention, it’s no surprise that the rise of artificial intelligence impacts every vertical across technology, and media is no exception. ChatGPT showcased the power of text-based LLMs two years ago, and it wasn’t long before richer forms of media in the form of image generation models like Dall-E, Midjourney, and Stable Diffusion became consumer accessible and became built into some of the fastest-growing consumer products and open models of all time. In 2024, the world is beginning to experience the power of AI as it relates to video, with the Sora demonstrations and new models emerging each week showing how videos can be generated, enhanced, and analyzed using AI.

While these sorts of batch media generation are handy for creating files or analyzing data asynchronously, a new, exciting, cutting-edge wave of technologies is coming that operate not just on prompts or files but also on livestreams of media or data. They can generate or modify outputs in real-time to keep up with the speed of livestreams, and they can perform analyses that output metadata that can be used in real-time decision-making.

Cascade - any situation where something flows or occurs in succession, often building in intensity or effect.

This area of real-time AI video is at the forefront of innovation. It requires a very different technical stack and approach than the very saturated batch AI processing market that is so prevalent today. Fortunately, one of Livepeer’s key strengths and assets is its real-time streaming stack. Every week between 78% and 92% of the usage on the Livepeer network is powering livestream transcoding, which can be broadly applied to other forms of computing on livestreams, including running AI pipelines.

What is a real-time AI video pipeline? Many AI models perform a single processing function on media. A chained together set of AI models and processing steps that operate on video, and execute many different steps to accomplish a use case or business goal can be called a Pipeline. A pipeline consisting of many steps can be far more powerful than just a single model.

What sorts of things are possible via real-time AI video? Some exciting new use cases range from creator- and entertainment-focused to real-time analysis of investment decisions. See the following examples

Content Creation

See this example using a livestream video input to animate and direct an avatar within live content.

Video Agents

AI-powered video agents can be deployed to identify objects, assess environments, or generate insights directly from live video streams, reducing the need for manual labor and on-site assessments. The diagram below illustrates a range of applications for video agents. Each use case showcases the versatility and efficiency gains possible through AI-driven video agents operating in real-time, providing scalable, dynamic solutions to industries reliant on live data analysis and decision-making.

Real-Time Analysis

Real-time analysis of streams can be applied across countless business use cases. See this sports example, which can feed real-time data to coaches making decisions. Previously, this data was only accessible to top-level professionals with big budgets, but it can now be accessible via open-source models to any high school coach with access to a camera.

Analysis use cases can be extended to any sort of monitoring use case, be it traffic, safety, security, business intelligence leading to real-time trading or investment actions, or more.

Livepeer as the Leading Platform AND Community

As mentioned, Livepeer’s stack is already very well suited to perform real-time processing on video streams. However, as we’ve seen with transcoding and generic AI inference, it’s not enough just to have technical capabilities on a network. That also needs to be coupled with a solid go-to-market motion to introduce demand to the network, and in those particular areas in crowded market segments, demand generation took the form of B2B sales and BD. While leading to growth over time, it was competitive and a bit slow going.

In cutting-edge areas of niche AI experimentation, the early momentum is typically far more community-oriented. Before knowing exactly how all of these technologies will be used at scale within applications, the most vibrant activity is within communities of enthusiasts - tinkerers, hackers, creatives, and engineers who are playing with new models and pipelines, sharing and showing off their outputs, discussing techniques for optimization and performance all evening in chat rooms, and building clout and reputation through sharing the latest innovations. The momentum often follows things that may look on the surface like fun toys to play with. We see this through examples like Civitai and Huggingface spaces, where the browsable galleries of potential outputs are just as inspiring to the communities and users as the underlying technical innovations are.

These communities and popular tools-based communities like ComfyUI are thriving but not built and designed around real-time AI video. The tools accept static inputs like prompts or files. The outputs are similarly static. They don’t offer a playground and browse experience to hook up existing livestreams or let tinkerers go live from their webcam. And perhaps more importantly, they aren’t coupled with infrastructure stacks that can keep up with the real-time latency and scale demands for ingesting, computing, and serving back AI-enabled video in real-time.

Livepeer’s community is already an excellent seed for what could be the center of innovation in this new real-time AI area. The tech stack, Livepeer network, and the incredibly knowledgeable base of node operators and video developers present a powerful starting point for a community to rally around showcasing, sharing, playing around with, and innovating on real-time specific AI models. This community growth is where momentum can be built and picked up significantly for the Livepeer project.

As the technology matures and developers are already innovating and playing with these inspirational real-time AI technologies within the community, it’s only natural that many will begin to bring them into more scalable business and application use cases down the line. Because real-time AI video needs to be paired with an infrastructure that can handle the scale and latency, unlike text and image batch processing that can be done closer to the edge, the Livepeer network itself will stand to benefit and grow from the fees paid in not only through the initial community playground experience, but eventually from the integrations into these scaled applications.

A relentless focus on providing this infrastructure and cultivating and growing the leading community product and platform for playing with these technologies will be central to Livepeer’s growth and success.

The Unique Superpowers of Livepeer and Opportunities This Creates

There are a few potential reasons why pursuing the goal of being the leader in real-time AI video would produce excellent outcomes for the Livepeer project.

The Superpowers that make Livepeer well suited to succeed at this effort

1. Live video is a differentiator that aligns with Livepeer’s tech stack and expertise. Most generic AI platforms don’t have the tech, expertise, or inclination to spend years building a global video ingestion, serving, and processing infrastructure.

2. Cryptoeconomic incentives and public goods funding present opportunities to make AI developers stakeholders in the network and its future use. Centralized platforms may be able to use fundraising capital to build out capabilities, but they can’t easily make the contributors stakeholders in the future growth of the networks the way token-coordinated networks can.

3. Community-Oriented. As an open and community-oriented project, Livepeer is well suited to pursue an effort that requires and benefits from community engagement, support, and growth. To date, this has been very effective on the supply side of the network, but with this real-time AI video community approach, the network's usage will also be reinforced by a demand-side community of developers.

4. Cost-effective infrastructure. Video-specific AI computing requires 10-1000x the compute of text or images, and GPUs come at a cost. Livepeer has already shown the ability to provide cost-effective GPUs through crypto-economic bootstrapping incentives and an open marketplace.

The winning outcomes for Livepeer

1. Opportunity to be the market leader. Going after real-time AI video pipelines is a bet on the future. Not many other tech platforms are trying to be this, and it’s still unclear what killer use cases will emerge. By leaning in early, and leveraging the above strengths, Livepeer can be a market leader rather than a competitor in a crowded space where it may struggle to catch up.

2. A network effect. Each new real-time AI video pipeline or plugin added to the network is open and composable with other pipelines also on the network. This means that every new pipeline makes the network MORE useful than before, and the more useful this network is, the better alternative it is to DIY cloud platforms, which should attract more developers. Unlike transcoding, where each user doesn’t add any more utility to the network, for the first time, this presents an opportunity for an actual network effect that can create a defensible moat around Livepeer as the most useful real-time AI platform. This is exciting.

3. Fee growth, community growth, decentralization. Ultimately, a growing community that is adding and playing with real-time AI pipelines leads to increased utilization of the Livepeer network. Those pipelines inspire application developers and businesses, leading to even more scale. Fees into the network accrue value to the stakeholders powering the network, and ultimately, Livepeer sees the demand growth that keeps those GPUs humming and the live video flowing across a decentralized network of node operators serving diverse AI video pipelines and use cases.

Focus

Previous network phase summaries have often listed out a number of protocol changes to support scaling, decentralization, or new capabilities. We’ve laid the groundwork with a robust and decentralized video infrastructure network, various core teams and SPEs that contribute development to different vital areas of the stack, and public goods funding sources.

The Cascade vision in this proposal is far more about focusing the entire ecosystem on one significant opportunity. While every stakeholder can use and build on the open Livepeer network as they see fit according to the protocol, we’ll be far more likely to succeed if we can marshal the above stakeholders, node operators, SPEs, community admins, and more to lean into this opportunity and spend the next 6 months validating that Livepeer can become the leading platform and community in real-time AI video pipelines. This may manifest in some dramatic changes to how the community messages Livepeer to the world, the tools we use, the metrics and stats we look at, and the product interfaces for both node operators and users of the network.

What sort of things does the community need to come together to build?

Practical Next Steps and Roadmap

At the moment, there are some early prototypes and demos that show real-time AI workflows being run in the context of Livepeer software. See this StreamDiffusion demo, which takes prompts and converts mouse movements in real-time to visual outputs.

Several elements must be built to deliver an initial product that makes these types of pipelines usable and contributable to the network. Here’s a quick glimpse at some of the components and stakeholders who will be focused on this in the coming months.

Phase 1 - Initial Real-Time AI Video Pipeline Playground - December 2024

The initial phase of delivering on this vision includes demonstrating and inspiring the world with the early possibilities of real-time AI video pipelines. To do this, the Livepeer community will enable developers to discover, test, and deploy AI-enhanced real-time video experiences. The first milestone will feature:

• A playground to experiment with real-time AI video pipelines in your browser, letting creators and developers transform live video streams with just a few clicks and share the results with collaborators.

• Livepeer network support for these pipelines, including payments, on our orchestrator Livepeer nodes.

• Self-serve live Pipeline deployment for ComfyUI workflow creators. This will allow you to convert your ComfyUI workflows into live video pipelines and deploy them on the Livepeer network.

• A community experience designed around onboarding, engaging, and retaining real-time AI pipeline developers. Let’s start cultivating the home for this thriving community, which may require a bit of a revamp of Livepeer’s existing community platforms.

Phase 2 - Standardized Pipeline Contributor Platform - Q1 2025

The second phase goes from provable and inspirational demonstration to permissionless contribution of custom pipelines. It allows unbridled community growth and the creative use cases enabled by real-time AI pipelines to flourish. It includes potentially:

• Standardized pipeline contribution framework or SDK.

• Curation, reputation, and social elements amongst pipeline developer community in the AI pipeline playground and community.

• Growth hacking to accelerate the community growth

Phase 3 - Scalable real-time AI Developer platform - Post-community growth

When Livepeer is a thriving and growing community of real-time AI pipeline contributors and a growing number of inspirational pipelines, it becomes time to focus more on the tools that enable developers to integrate these pipelines into their applications and businesses at scale. This means there’s an increased focus on:

• Network reliability and cost

• Developer SDKs and product interfaces for integration

• Community-driven support and resources for application builders

While phase 3 represents significant growth and scaled usage, phase 2 alone can dramatically increase the usage and fees on the Livepeer network. All of these cutting-edge technologies have begun to be adopted by communities tinkering with them, and the pipeline showcase and playground allow significant usage amongst a growing community. If there is growth and momentum there, those developers will inevitably take the pipelines into applications at scale, the same way we’ve seen image and text-based AI models be adopted into countless products directly out of the experimentation playgrounds.

In Summary

This post contained lots of information about the potential of Cascade and this new opportunity. But in summary, if the Livepeer community can come together to do the following things in the next 6 months, then we’ll collectively be off and running with a shot at achieving this exciting goal:

1. Add real-time AI video pipeline support to the Livepeer Network and surrounding tools.

2. Build a great pipeline showcase and playground UI that lets people show up to discover these pipelines, run them, and view and share the outputs to inspire others.

3. Update our community experience to be an excellent home for those looking to experiment and push the boundaries of what’s possible with real-time AI video pipelines.

4. Leverage our public goods funding programs to create the right incentives to make builders in this space stakeholders in the Livepeer network.

This new and exciting opportunity energizes me. For all those in the community or looking to join the community to get involved in this mission, the best starting points are:

1. Get early access to real-time AI video pipelines.

2. Stay up to speed with development by joining the Livepeer’s Discord community.

Whether you're an independent filmmaker or a large-scale production team, understanding how AI can streamline workflows is essential to stay competitive.

This article explores some of the most impactful ways AI is enhancing video production. You’ll learn about AI's role in automating editing processes such as auto-reframing, color correction, and scene detection. We'll delve into how AI is transforming scene design and offering new opportunities for content creators through AI-generated video. Additionally, we'll highlight the power of AI in analytics—providing insights into audience behavior that can refine your creative strategy and improve audience engagement.

With the potential to save time, improve quality, and make video production more accessible, AI is not just a trend—it's the future of the industry. Read on to discover how these innovations can elevate your production process and drive the next level of creative expression.

Content Creation

Tools like Runway, Sora, and Stable Diffusion use machine learning models to create entirely new videos based on text prompts or visual data inputs. For example, Runway enables creators to generate unique video content without the need for traditional video shoots.

Here’s how a workflow could look for creating an animated scene that would normally need a big budget and teams of specialists:

Input a text description (e.g., “a space station orbiting Mars”).

Generate a video sequence that aligns with the input description.

Fine-tune the result by regenerating scenes or modifying visual elements to better suit the project’s needs​.

This workflow eliminates the need for live filming or complex visual effects, making video production faster, more flexible, and accessible to creators with limited resources.

Editing

Auto Reframe

AI-driven auto-reframing tools like Adobe Premiere Pro automatically adjust video aspect ratios, ensuring that key elements stay in focus, whether the content is for TikTok, YouTube, or Instagram. This saves hours of manual cropping, making sure that the critical parts of the video fit multiple platforms, a task that would typically take much longer​ when completed manually.

Scene Edit Detection

Using AI-powered tools like DaVinci Resolve’s 'Detect Scene Cuts,' editors can quickly break down raw footage into individual scenes. This reduces the need to scrub through hours of content manually and allows the editor to focus on storytelling instead of laborious, technical edits. It's particularly useful for editors working on documentaries or long-format content​.

Color Correction and grading

AI helps automate color grading, ensuring visual consistency across footage. Machine learning algorithms match color tones between different clips, making adjustments for white balance, exposure, and color correction. For filmmakers working with mixed camera formats, AI can seamlessly blend footage, ensuring a uniform aesthetic throughout​.

Scene Design

Tools like Runway allow users to design entire video scenes by generating content based on text prompts or existing footage. This can vary from designing sets for shoot locations to creating background graphics. This approach is particularly helpful for content creators working on visual projects like music videos, where custom scenes can be generated without physically shooting them.

Analytics

AI analytics tools offer actionable insights into how audiences engage with video content. These tools analyze view counts, engagement rates, and audience retention to help creators refine their content for better performance.

Audience Segmentation

Automatically group viewers based on demographics, behaviors, and preferences, allowing creators to tailor future videos to specific audience segments.

Predictive Analytics

AI predicts which video elements will perform best based on historical data and current trends.

Optimization

AI tools can suggest improvements to video titles, thumbnails, and metadata to boost engagement and click-through rates.

What the future holds for AI video production

AI is already streamlining the video industry from pre-production to enhancing marketing strategies. However, the future holds even more possibilities for AI in video creation. Entire production processes could eventually be automated in real time, allowing for the creation of next-generation content at unprecedented speeds.

Marketing

AI applications could revolutionize video marketing by automatically identifying the best parts of longer clips and assembling fully formed, targeted videos based on current audience trends. Marketers could use the same application to track engagement across channels, refining their strategies for even greater impact in future campaigns.

Instant translation

In the future, AI tools could offer real-time translation and transcription, matching the cadence and tone of speakers in any language, eliminating the need for costly translation services. This would make video content globally accessible with minimal effort.

Live AI-Powered Virtual News Reporting

AI could enable personalized, real-time virtual newsrooms, providing round-the-clock news broadcasts tailored to individual preferences. This could include everything from live financial updates to custom documentaries, all driven by AI-generated content.

Immersive VR Experiences

AI could transform traditional video footage into fully immersive VR experiences, allowing audiences to engage with creators in interactive, real-time environments. From exploring vibrant street markets to attending private concerts, viewers could experience a deeper connection with their favorite content creators than ever before.

Limitless creativity

Imagine the possibilities: marketers, influencers, and creators of every stripe empowered to craft breathtaking videos on a massive scale. The only obstacle? A shortage of tools that make video creation both seamless and affordable.

That’s why Livepeer AI is leading the charge by making AI video tool development more accessible and affordable than ever. With a comprehensive suite of AI capabilities—text-to-image, image-to-image, and image-to-video—powered by the cost-efficient GPU compute of the Livepeer Network, developers have everything they need at their fingertips. Beyond generative video creation, Livepeer AI also empowers developers with AI-driven editing, upscaling, subtitling, and more, streamlining the entire video development process.

Are you a builder that wants to enable the future of video with outstanding AI tools? Learn more about Livepeer AI here.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

This was many internet users’ first interaction with a convincing deepfake. The ensuing furore highlighted the growing threat posed by deepfakes and AI-generated content. Deepfakes leverage advanced machine learning techniques to create hyper-realistic but entirely fabricated videos, images, and audio. As these technologies evolve, distinguishing between authentic and fake content becomes increasingly challenging. Since then, content authenticity has become a central part of the debate surrounding AI.

Let’s dive in.

Challenges Posed by Deepfakes and AI-Generated Content

The term deepfake was coined in 2017 after a Reddit user created a portmanteau of “deep learning” and “fake”.

While the term is relatively new, this kind of process has been possible since the late 90s. Momentum only really picked up when Ian Goodfellow created Generative Adversarial Networks (GANs) in 2014. GANs consist of two neural networks—the generator and the discriminator—that work together to produce increasingly realistic content. Since then, the technology has advanced rapidly, making it easier to produce convincing deepfakes with minimal technical expertise.

Vulnerabilities in the AI Generation Process

Misinformation online doesn’t just come from shocking deepfakes or polarising social media content. It can also sneak into the very AI generation process itself. The creation of AI-generated content involves a complex process that, while innovative, introduces several vulnerabilities at different stages. These weaknesses can be exploited to manipulate or distort the information that AI systems produce, leading to the dissemination of false or misleading content.

Let’s take a look at some of the vulnerable stages:

• Data Collection: The initial stage where data is gathered can introduce the first set of vulnerabilities. Biased or fabricated data can skew AI learning algorithms, resulting in outputs that may perpetuate inaccuracies or false narratives.

• Output Generation: Finally, the actual content produced by the AI can be altered. Manipulating the final output can be particularly harmful as it directly affects the content being consumed by the public.

Combating Content Inauthenticity with Blockchain Technology

Blockchain technology is one viable solution for tackling the internet content crisis because its decentralized nature offers an immutable audit trail for each piece of content, ensuring a verifiable and unalterable record of its origins and any modifications. Every piece of data that is recorded on a blockchain is timestamped, documenting the exact time of creation and any further changes that are made. Timestamps are created via a digital signature verified by hash functions, which create a sort of digital fingerprint for the original data. This information is publicly available and verifiable, allowing users to certify the authenticity and provenance of data rather than simply having to trust a centralized authority.

Blockchain is already being used to support content authenticity, from data gathering to distribution:

1. Transparent training and data collection: Blockchain can address AI biases at the dataset level by tracking data provenance, adding transparency to how models are trained. Blockchains also allow for sharing of data between parties, avoiding biases from centralized datasets. In this sense, blockchain is not a magic fix, but rather a useful lens to observe, track and correct biases.

2. Content authenticity: Blockchain can create provable attestations for content that document its history, taking in creation, editing and distribution, making it easier for viewers to know if they can trust what they are seeing. It’s even possible to note the camera, creator and encoding details, all of which can be checked independently.

3. Built-in Verification: These attestations can be integrated into the stack of the existing media ecosystem. Browsers, media platforms and verification systems can then use this to both tackle fake content and build the trustworthiness of their brand. This can be done through cryptographically secured identities and domain names, such as ENS handles.

Blockchain can help stem the rising tide of deepfakes online too. U.S. media company Fox partnered with Polygon to create Verify, a protocol to combat deepfakes and AI-generated stories. According to Fox, “publishers can register content in order to prove origination. Individual pieces of content are cryptographically signed onchain, allowing consumers to identify content from trusted sources using the Verify Tool.”

Joining the Coalition for Content Provenance and Authenticity

These stages underscore the importance of robust frameworks like the Coalition for Content Provenance and Authenticity (C2PA), which was established to create standards that ensure digital content remains genuine and reliable from creation to consumption.

The C2PA, initiated by industry leaders such as Adobe, Microsoft, and the BBC, focuses on embedding provenance metadata within the content. This metadata, secured through cryptographic means, offers a comprehensive record of the content’s origin, creation process, and any modifications, fostering transparency and accountability. In practice, this means that every piece of content could come with a C2PA ‘nutrition label’, in which it’s possible to check:

• The creator’s identity

• The tool used to make the content (Adobe Photoshop, DALL-E 3, camera)

• Time of creation

• Assets used and actions performed on them during creation

This so-called ‘label’ would travel with the content across all platforms and could be checked at any time to see if any alterations have been made.

Livepeer is committed to building the world’s open video infrastructure, so ensuring that video content is authentic and clearly labeled when generated by AI is an important part of the firm’s vision. Livepeer intends to follow the standards outlined by the C2PA to ensure the integrity of digital media, embedding crucial metadata that tracks content from its origin through all modifications. This approach not only combats misinformation by verifying content authenticity but also supports creators by protecting their work in a landscape increasingly influenced by AI technologies.

Establishing the authenticity of content online is fast becoming one of the most pressing issues of our time. The sheer scale of the challenge means that companies, governments and organizations will need to work together to make any kind of meaningful impact. The technology behind deepfakes will unquestionably improve with every passing day, but so will both the tech and resolve of those trying to stop them. As a company that helps creators with their video content, we are very intentional about tackling authenticity. The road ahead is long, but with C2PA and blockchain we’re already making a strong start.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

Livepeer AI SDK

To accelerate your development process, we've introduced a comprehensive, open-source SDK for Livepeer AI. Key features include:

• Developer-Centric Design: Crafted with best practices and optimal developer experience in mind, ensuring seamless integration of Livepeer AI into your projects.

• Intuitive Interface: Developers familiar with client libraries will find our conventions immediately accessible.

• Multilingual Support: Available in TypeScript/JavaScript, Python, and Go, powered by OpenAPI and generated with Speakeasy.

• Code Quality Assurance: Benefit from strong type safety, intelligent hinting, and comprehensive documentation, all with minimal overhead.

Whether you're launching a new application or enhancing an existing one, our SDK equips you with the tools to effectively leverage Livepeer AI with confidence.

Current Pipeline Capabilities

Livepeer AI offers a diverse array of cutting-edge pipelines, empowering you to create sophisticated AI-driven applications:

• Text to Image: Breathe life into written descriptions with vivid visuals.

• Image to Image: Transform existing images using AI-powered enhancements.

• Image to Video: Convert still images into dynamic video sequences.

• Image Upscaling: Significantly boost image quality and resolution.

• Audio to Text: Generate accurate written transcripts from spoken content.

• Segment Anything 2: Achieve precise object segmentation for advanced computer vision tasks.

• Language Model (LLM): Harness powerful language models for natural language processing and generation.

Each of these pipelines are already supported by our SDK, allowing efficient and cost-effective use of these advanced AI capabilities. From creative tools to content management systems and data analysis applications, Livepeer AI provides the robust foundation needed to innovate and excel.

Upcoming Pipeline Developments

At Livepeer, we continuously push the boundaries of AI technology. Our roadmap includes exciting new capabilities to expand your creative and analytical potential:

• Text to Audio: Convert written content into natural-sounding speech, opening new avenues for content creation and accessibility.

• Image to Text: Transform visual information into descriptive text, enabling advanced image analysis and improving content accessibility.

• Lipsync: Synchronize audio with visual lip movements, enhancing realism in animated characters or dubbed video content.

These forthcoming features will be supported by new versions of the SDK, providing an even more comprehensive toolkit for AI-powered development. With Livepeer AI, you'll remain at the forefront of AI technology, poised to create the next generation of intelligent applications.

Start Building Today

Livepeer AI supplies developers with powerful tools to integrate advanced AI capabilities into their applications. We invite you to download our SDK and explore its potential for your projects. For ongoing support and collaboration, join our Discord community, where you'll find fellow developers and our team ready to assist with implementation challenges and discuss use cases.

The SDK documentation and our active community serve as invaluable resources as you develop AI-enhanced applications. Take the next step in your AI journey—download the Livepeer AI SDK and connect with us on Discord to fully harness our platform's capabilities.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

Decentralized AI services offer a compelling alternative. By distributing computational resources across a network of individual contributors, a decentralized approach can dramatically reduce costs, increase accessibility, and foster a more open and innovative ecosystem. Here, we explore how developers can create a completely decentralized generative AI tech stack, and the benefits of doing so.

The Decentralized Gen AI Tech Stack

• Infrastructure and Compute Resources

  • Livepeer AI

  • Akash Network

• Foundation Models

  • EleutherAI’s GPT-J

  • BLOOM (BigScience)

• Development Frameworks

  • PySyft

  • FATE

• Fine-Tuning and Adaptation Tools

  • Hugging Face’s Adapters

  • PET (Pattern-Exploiting Training

• Data Processing and Management

  • Livepeer AI

  • Filecoin

Infrastructure and Compute Resources

Generative AI is dependent on the computational infrastructure it runs on. In a decentralized ecosystem, this infrastructure is distributed across numerous nodes, offering advantages in scalability, cost-effectiveness, and resilience.

Livepeer AI

Livepeer AI is a decentralized network optimized for AI video processing. It leverages its widely distributed, established hardware providers, who are already transcoding millions of minutes of traditional video every week with plenty of GPU capacity available, to also support affordable generative AI tasks.

Benefits:

• AI video focus optimizes text-to-image, image-to-image, and image-to-video conversions

• Pay-per-task model more cost-effective than reserving expensive centralized capacity

• Eliminates single-point-of-failure risk inherent to highly centralized cloud servers

Akash Network

Akash is a decentralized cloud computing marketplace focused on container deployments, with unique capabilities in hosting machine learning models for inference and running validators and nodes for various blockchains, often at prices significantly lower than centralized providers.

Benefits:

• Container-native structure is optimized for deploying containerized applications

• High performance by leveraging unutilized data center capacity

• Interoperability allows it to support various container orchestration tools

Summary

• Livepeer AI is a tailored choice if your primary focus is video processing, streaming, or media-related applications, as it is specifically optimized for video transcoding, AI-driven video enhancements, and streaming scalability.

• Akash Network may be better suited for broader compute use cases, including deploying AI/ML models, decentralized applications, or blockchain nodes.

Foundation Models

Foundation models, also known as large AI models, are advanced AI systems trained on large amounts of data to perform a wide range of tasks, such as generating output from prompts. They serve as the bedrock upon which generative AI applications are built. While centralized models like GPT-3 or DALL-E have garnered significant attention, their decentralized counterparts offer unique advantages.

EleutherAI

The open-source language models developed by EleutherAI, including GPT-J and newer GPT-NeoX, are designed to rival the capabilities of OpenAI's GPT-3 — so much so that Jack Clark, the author of the Import AI newsletter, has called it an “attack on the political economy of AI.”

Benefits:

• Open-source nature allows for transparency and community-driven improvements

• Can be run locally, reducing dependency on cloud services

• No usage restrictions or API costs associated with proprietary models

BLOOM (BigScience)

BLOOM is a multilingual large language model, created through a collaborative open source effort involving more than 1000 researchers worldwide.

Benefits:

• Supports 46+ languages, and can be fine-tuned for specific domains or languages

• Enables research and innovation without commercial constraints

• Community-driven development ensures diverse perspectives

Summary:

• EleutherAI may be your choice if you prioritize greater open-source flexibility, stability, and a more grassroots researcher community.

• BLOOM is a better option If multilingual support, a globally diverse development approach, and a deliberative focus on ethical AI are important to you.

Development Frameworks

Development frameworks play a crucial role in the creation and deployment of AI models, supporting distributed computing, ensuring data privacy, and facilitating collaborative development. They play a role throughout the entire development process, from data preparation and training to testing and deployment.

PySyft

PySyft is an open-source library for secure and private Deep Learning, developed by OpenMined. It preserves privacy by enabling computation on encrypted data without decrypting it first.

Benefits:

• Multi-party computation allows collaborative AI development without sharing raw data

• Interoperabile with popular deep learning frameworks like PyTorch and TensorFlow

• Extensible, supporting custom privacy-preserving operations and protocols

Federated AI Technology Enabler (FATE)

FATE is an open-source project initiated by Webank's AI Department to provide a secure computing framework for federated AI, with a modular design for flexible deploymenet and customization.

Benefits:

• Comprehensive support for various federated learning algorithms and architectures

• Scalable design optimized for large-scale industrial applications

• Works cross-platform with support throughout heterogeneous computing environments

Summary

• PySyft’s decentralized model can empower many privacy-preserving use cases in particular, such as collaborative health care research on sensitive patient data or fraud detection across multiple banks without sharing customer data.

• FATE’s cross-organizational features can help aid collaborative model training across different companies or institutions, as well as offer federated learning for distributed sensor networks in fields like IoT or Edge Computing.

Fine-tuning and Adaptation Tools

While foundation models provide powerful general-purpose capabilities, fine-tuning and adaptation tools allow developers to tailor these models for specific tasks or domains. In a decentralized AI ecosystem, these tools play a crucial role in democratizing AI development and enabling customization without the need for extensive computational resources.

Hugging Face's Adapters

Hugging Face Adapters are lightweight, trainable modules that can be added to pre-trained models to adapt them for specific tasks without fine-tuning the entire model, using significantly less computational resources in the process.

Benefits:

• Multiple adapters can be trained for different tasks and swapped out as needed

• Can be trained on sensitive data without exposing the entire model

• Easier to manage and version control compared to full model checkpoints

PET (Pattern-Exploiting Training)

PET is a few-shot learning technique that leverages task-specific patterns to improve model performance with limited labeled data (a scenario in which there is a scarcity of annotated information available for training machine learning models).

Benefits:

• Data efficient, achieving good performance with very few labeled examples

• Enables AI applications in domains where large labeled datasets are scarce or expensive to obtain

• Facilitates distributed learning scenarios where data cannot be centralized

Summary:

• Hugging Face Adapters are ideal if you need parameter-efficient fine-tuning, modularity, and want to benefit from the ecosystem provided by Hugging Face.

• Pattern-Exploiting Training (PET) may be preferable if you need label-efficient learning, are operating in a low-data environment, and prefer a prompt-based learning approach for tasks like classification or few-shot learning.

Data Processing and Management

Efficient and secure data management is crucial for AI development, especially in a decentralized ecosystem. This component of the stack focuses on how data is stored, shared, and processed in a distributed manner.

Filecoin

Filecoin is a decentralized storage network that turns cloud storage into an algorithmic market, enabling distributed long-term storage of large AI data sets and archiving of AI research data, among other use cases.

Benefits:

• Distributed storage leverages unused storage capacity around the world

• More resilient data, distributed across multiple nodes for redundancy

• Verifiable through cryptographic proofs showing that data is being stored correctly

Livepeer AI

While primarily an infrastructure provider, Livepeer AI also plays a crucial role in data processing, particularly for video-related generative AI tasks. What’s more, it is the first open infrastructure network to join the Coalition for Content Provenance and Authenticity (CP2a), an open technical standard working to provide publishers, creators, and consumers the ability to trace the origin of different types of media.

Benefits:

• Supports video processing tasks: upscaling, frame interpolation, and subtitle generation

• Greater resiliency and censorship resistance reduces risk of data loss or unavailability

• Working to provide verification for the origin and authenticity of AI video content

Summary

• Filecoin is best used when your focus is on general decentralized data storage, particularly for large-scale data that needs long-term security and redundancy.

• Livepeer AI is ideal when you need to process, manage or enhance video content in real-time, especially when AI/ML video analysis or cost-efficient streaming is important for your platform.

You can get started testing your decentralized AI tech stack with Livepeer AI, which has been optimized by our community to streamline generative AI tasks for developers and creators.

Certainly, some larger enterprises may be fine dealing with these obstacles. But smaller creators and independent artists? Most simply can’t afford to incur the high costs of experimenting with generative AI, plus creating on these centralized platforms can pose serious data privacy and censorship concerns. Worst of all, many contractually state that artists can’t claim copyright over their AI-generated pieces, robbing them of ownership over the works they create.

“The creative process today is quite complex and expensive. The costs can be astronomical, and they inhibit many from doing this type of art at all,” says Malcolm Levy, a co-founder of Refraction DAO, an artist-owned community working at the intersection of digital art, music and culture.

If digital art is to thrive in the age of AI, it must be supported by a system that is fairer to users, one that helps them more affordably create memorable works for the world to enjoy.

A generative AI video created by Jahmel Reynolds using the Regenerate app on Livepeer AI

How decentralization can help AI creators

Using token incentives to support collaboration, so artists can build on networks that are more cost-effective and more accurately represent their beliefs.

Decentralized systems can transfer control of the generative AI tech stack from one centralized entity to a network of individuals working together to create a platform that better reflects their needs and values. This is especially critical to creatives, who may have strong ethical beliefs about how AI should be built, and also may want to more directly have a say over the tools they create with.

Blockchain-based platforms can use cryptoeconomics to bootstrap these decentralized AI communities. By rewarding positive contributions with a token, the network is able to incentivize good behavior and properly credit those who do the most to build and grow it. Creators can choose to align themselves with networks that reward the values they want to see more of, and they can also provide better feedback to improve these decentralized AI tools, since most are open source.

One example of this is Livepeer AI, which is lowering the cost of entry for accessing generative AI. By using its LPT token, it is able to organize and incentivize individual hardware providers to offer compute for a litany of on-demand generative AI tasks, fueling a decentralized pay-per-task model that can be much more cost-effective for AI developers and creators.

Democratizing generative AI art with Regenerate

Blending traditional art techniques and cutting-edge AI technology to shape a more robust, and collaborative, canvas for creators.

Regenerate is an application built with Livepeer AI used for creating new artistic realities with generative AI, including text-to-image services. Built by the Refraction DAO community, it is supported by Livepeer’s decentralized compute network which is already used to transcode millions of hours of video on a weekly basis.

While centralized generative AI tools often have to address many competing audiences with their product, Regenerate is able to focus intently on the specific needs that generative AI creators within its own community face. For example: When surveyed, many of Refraction’s AI artists said they wanted more capabilities for complex collaborative works, a simpler UX/UI experience, and, of course, a more affordable compute landscape.

“The current process can be quite stagnant. To create interesting, creative work that goes beyond just entering a prompt, it takes using a number of applications, a lot of time, and significant cost and knowledge barriers that we’re trying to alleviate,” Levy says.

Regenerate currently integrates AI tools — starting with a simple interface for text-to-image, which will soon expand to include image-to-video — offering a blend of both traditional digital art techniques and cutting-edge AI technology like “shader art,” in which GPU programs originally designed for processing the appearance of 3D models are creatively repurposed to create visually stunning images and animations.

What’s more, it allows AI creators to share and collaborate on generative AI pieces.

"We're using different ways of affecting those tools that are much simpler and easier to understand, so people can augment and create around each other’s work," Levy says.

When artists create AI art on Regenerate, they can check the “This prompt is public” box and then press “Create Prompt” to have their work be shared with the entire community on the app’s dashboard.

Other Regenerate community members can then click on their image and iterate on both the prompt itself or on the different filters and permutations by adjusting various sliders, as shown in the screenshot below:

Empowering AI artists with ownership and community

Improving provenance, feedback loops and community contributions to artistic endeavors, all made programmatic through decentralization.

Refraction hopes to create a model that better serves artists by granting them complete ownership over the works, possibly using blockchain capabilities like on-chain attestation to provide verifiable proof of authorship and provenance. The community also believes it can better respond to feedback from its artists than centralized AI platforms that may make compromises for their leadership teams or investors at the expense of creators.

That focus on collaborative input goes beyond just AI image and video creation, but is woven into the very fabric of the project. Its very code is a work of art for the community, since countless developers from a number of organizations within the Refraction DAO network can creatively contribute to its open source development.

“Decentralization gives the opportunity for all of these different creatives to add on and grow and build AI technology in real time,” Levy says.

By using Livepeer AI, Refraction is able to not just reflect the values of its decentralized community, but also access resources it needs to experiment in the quickly evolving generative AI space.

“There is an ethical value to us using a decentralized compute provider like Livepeer. Plus, there is certain developer help, including tools and functionality, that just wouldn’t be possible to access without their support,” Levy says.

Rather than renting expensive compute space, Regenerate will also benefit from Livepeer AI’s decentralized pay-per-task model, savings that can make the app more cost-effective for its creators than centralized operators.

Embracing the decentralized future of generative AI

Regenerate recently launched on the Livepeer AI testnet, and is actively seeking creators to provide feedback for its text-to-image features.

Learn more about projects building on Livepeer AI, and join the network either as an infrastructure provider or as a developer.

AI and Crypto: Pushing the Frontiers of Innovation

One of the biggest areas of focus in August was the continued development and recognition of Livepeer’s role in AI and decentralized media.

Grayscale Report

Livepeer's inclusion in the Grayscale report on the intersection of AI and crypto was a significant milestone. The report highlighted how Livepeer is providing decentralized video processing and AI compute solutions, positioning us as a key player in tokenized AI infrastructure. This marks an important step in showcasing Livepeer’s capabilities to a broader audience of investors and developers.

The Launch of the AI Video Startup Program

August also marked the official launch of the AI Video Startup Program, aimed at supporting the development of innovative AI-powered video applications. Through grants, mentorship, and early access to Livepeer’s AI video infrastructure, this program fosters the next generation of AI-driven media startups. Participants in the program are working on cutting-edge projects that utilize Livepeer's decentralized video infrastructure to enhance AI applications.

Supporting the Generative AI Video Hackathon

During Korea Blockchain Week (KBW2024), Livepeer sponsored the Generative AI Video Hackathon at Story House. The event allowed developers to leverage Livepeer’s decentralized AI infrastructure to build innovative media applications. This hackathon showcased Livepeer AI infrastructure's capabilities and demonstrated Livepeer’s commitment to supporting the AI developer ecosystem.

Key Collaborations: Strengthening the Ecosystem

August was also marked by important collaborations that extended Livepeer's reach across multiple industries.

Livepeer Powering the Burning Man Webcast

In one of the standout moments of August, Livepeer played a pivotal role in powering the Burning Man webcast. The event, held in the Nevada desert, is known for its innovation and creativity, and this year, it embraced decentralized technology to broadcast the festival to a global audience. By leveraging Livepeer Studio’s decentralized video infrastructure, the webcast team delivered seamless, uninterrupted, and cost-effective coverage, ensuring that viewers worldwide could experience the magic of Burning Man without any disruptions.

Collaboration with Encode Club for AI & GPT Bootcamp

Livepeer continued to invest in AI education by teaming up with Encode Club for their AI & GPT Bootcamp, which began on August 19th. This collaboration is designed to give participants hands-on experience with AI technologies, teaching them how to leverage Livepeer’s decentralized infrastructure for AI video applications. The bootcamp is yet another testament to Livepeer’s role in nurturing the next wave of AI talent.

Enhancing Web3 Engagement with Backdrop V6

In addition to partnerships in the AI space, Livepeer joined forces with Backdrop for the launch of Backdrop V6, a community-driven platform designed to enhance Web3 engagement. The collaboration reflects Livepeer's ongoing commitment to supporting innovative applications across Web3 ecosystems by offering decentralized, scalable video infrastructure.

Ample Protocol Goes Live

Ample Protocol, a Web3 app that enables creators to monetize their onchain intellectual property (IP) and NFTs, went live in August. Leveraging Livepeer Studio for seamless content hosting and token gating, Ample Protocol provides creators with new opportunities to manage and monetize their digital assets.

Educational Content and Community Engagement

August also saw Livepeer's leadership step forward with significant contributions to public discourse on decentralized AI and crypto.

Doug Petkanics on CCN and the DePIN Hub Podcast

Livepeer’s co-founder and CEO, Doug Petkanics, shared his thoughts on a CCN article discussing the dangers of Big Tech monopolies over AI models. Doug emphasized the importance of open-access AI networks, like Livepeer, which empower developers and innovators to sidestep the biases, high costs, and restricted access typically associated with proprietary AI models. Doug also appeared on the DePIN Hub Podcast, where he further elaborated on how Livepeer can empower creators and tokenized AI can revolutionize industries.

Multiple Blog Articles

Several influential articles were published this month, advancing Livepeer’s thought leadership in the decentralized AI space:

• "The Limitations of Centralized Cloud Compute in the Age of AI" discussed how centralized cloud infrastructure stifles innovation and how decentralized solutions offer a compelling alternative.

• "Understanding Diffusion Models in Generative AI" provided a deep dive into the diffusion models that are transforming video production through AI.

• "Adding Multimedia Engagement to Messaging" explored how Livepeer AI can enhance messaging apps by enabling real-time generative media capabilities.

• "How Crypto Incentives Unlock the Magic of AI" examined how tokenized models can drive innovation in AI-powered applications.

Flipguard’s Launch of Gen, Powered by Livepeer AI

Flipguard, a startup in Livepeer’s AI Video Startup Program, launched its Gen tool this month. Gen enables users to generate AI-powered media in real-time, offering a glimpse into the future of personalized, interactive media creation. By utilizing Livepeer AI, Flipguard is making generative AI tools accessible to everyone.

Community Growth and Governance

This month was a significant period for community initiatives and governance proposals within the Livepeer ecosystem.

Passing of Key Proposals

Two important governance proposals passed this month:

• The Live Pioneers proposal aims to build a global Livepeer community by expanding outreach into additional languages, improving educational resources, and offering support channels for delegators.

• The LLM SPE proposal, which will integrate Large Language Models (LLMs) into the Livepeer network, passed and promises to open new AI-driven opportunities for developers and users alike.

Onchain Bear Cage Partnership with Base

Livepeer teamed up with Base to launch Onchain Bear Cage, a new platform designed to bring more immersive and interactive experiences to users by utilizing Livepeer's decentralized video infrastructure.

Bi-Weekly Development Coordination Call

August introduced an updated cadence for bi-weekly core development calls, now aligned with the newly funded SPEs and treasury-coordinated initiatives. These calls are designed to coordinate efforts among independent development teams, streamline workstreams, and ensure that the Livepeer network continues to evolve efficiently and in alignment with community needs.

Looking Forward: The Road Ahead

As we step into September, the stage is set for even more groundbreaking developments. Livepeer will be hosting Limitless, an exclusive event during Token2049 in collaboration with DCG. This event will focus on tokenized AI and feature leaders from across the AI and crypto sectors. It’s a key opportunity for investors, developers, and founders to network and learn more about how tokens are driving AI innovation.Don’t forget to RSVP here.

With these significant achievements behind us, August has been a transformative month for Livepeer. As we move forward, we remain committed to building the world’s open media infrastructure and pioneering the future of decentralized AI.

Stay tuned for more exciting updates from Livepeer, and thank you for being part of this incredible journey!

Consider just one segment: the AI-generated video sector. AI has already transformed video creation, reducing barriers to access and replacing content that once took entire teams to produce. However, that promise is limited by a serious challenge: It is powered by a largely centralized $49 billion GPU market that is controlled by a few global web2 monopolies (from NVIDIA hardware to Microsoft Azure and Amazon Web Services compute) that drives up prices and creates a global AI compute bottleneck.

Just a few months ago, our community launched Livepeer AI to take on this challenge. Through blockchain incentives and tokenomics, we can now leverage our decentralized network of hardware providers — a community that is already transcoding millions of hours of video weekly — to provide resources for AI-generated video as well. With decentralization, we can offer developers a much more pay-per-task model that is much more cost-effective than renting expensive GPU capacity from centralized operators.

Unlike traditional cloud services, our model creates an infinitely scalable network of GPUs as demand increases. This open, affordable compute network can power limitless creation, invention and scaling for AI tools and the creators who use them — and ours is just one example of the innovative communities using token-enabled decentralization to catalyze the advancement of AI.

That is why the Livepeer community is proud to announce Limitless, an exclusive event to network and learn from the leading voices in crypto and AI. Submit your RSVP below to join the top tech investors, executives, and AI trailblazers at Limitless.

RSVP

From Limited to Limitless

Limitless is a unique opportunity to share, network and find new venues for collaboration around token-powered AI. We are bringing together speakers and leaders from some of the top web3 and AI companies in the world, including DCG, Akash, Render, Coinfund, Pantera, OKX and more. The audience will be a curated group of venture capitalists, institutional investors, asset managers, and executives from the top tokenized AI projects.

Join us on Sept. 17 at the world-famous ArtScience Museum in Singapore, where we will have a welcome luncheon and speeches from visionary innovators, plus time for entertainment, networking and collaboration.

Limitless is about more than just knowledge sharing - it’s about participation and engagement with AI tools. That’s why Livepeer is launching the AI-generated media contest: a call to all AI builders and creators to experiment with these cutting edge tools for the chance to win two suite tickets to the Formula 1 Singapore Grand Prix event — more details on that below.

SPEAKER LIST | AGENDA | RSVP

The Contest

Are you ready to break boundaries and redefine what’s possible? Livepeer invites you to participate in Unbounded Creativity: The Limitless Challenge!

Create an image, GIF, or short video that pushes the limits of AI-powered media and showcases a future where creativity knows no bounds. Top entries will be showcased at Limitless, with a chance to win two suite-level tickets to the F1 Grand Prix Event on the 20th of September.  The top submissions will be selected by the Livepeer team and presented at Limitless, with investors casting a live vote to select the final winner.

Theme: Participants are invited to use generative AI tools to create content —whether an image, GIF, or short video—that embodies the theme of “Unbounded Creativity.” This challenge encourages you to visualize a future where AI, decentralization, and innovation redefine media, pushing the boundaries of what’s possible.

Requirements: You may use any generative AI tool of your choosing. Individuals can submit a maximum of three submissions. You must be present at the Limitless event on September 17th when the winner is announced in order to claim the contest prize.

Deadline: Submissions will be accepted until 11:59 PM EST on September 12.

Unleash your creativity, embrace the limitless, and be part of shaping the future of media. We can’t wait to see what you create. Enter your submission below.

ENTER YOUR SUBMISSION

Developing AI tools faster and better is critical to realizing those many opportunities. Yet, most AI development today occurs in a closed system that stifles innovation. Only a few centralized companies, such as Open AI or Anthropic, have significant access to the specialized hardware needed to run AI models. The discoveries that come from the billions of prompts made in tools like Chat-GPT and Claude are walled off, siloing critical learnings away from the public and behind corporate doors.

Decentralization represents a major opportunity to scale this generational opportunity more effectively, using blockchain technology to connect global teams and push the entire AI ecosystem forward together. This is often achieved on blockchains by using tokens to create trust and incentivize good behavior among disparate actors that, otherwise, couldn’t be trusted to work together in a permissionless way.

In this piece, we explore how five networks use crypto-incentives to create new public models for better developing, deploying, and accessing novel AI capabilities.

1) Bittensor: Incentivizing Open Collaboration to Advance AI Benchmarking

Peer-to-peer marketplace for transforming machine intelligence into a tradable commodity.

Token Ticker: TAO
Market Cap*: $1.98 billion

What it is:
Bittensor creates a decentralized mechanism for evaluating and iterating AI models across the web. Peers rank each other by training neural networks that are able to learn the value of their neighbors. High ranks are given additional weight in the network, signifying their stronger performance. The result: A collectively run machine intelligence market that continually produces, and improves on, benchmarking for strengthening AI models.

How it works:
Bittensor uses its native TAO token to incentivize users that contribute information to the collective intelligence of the network. In order to fairly distribute these incentives, the Bittensor network leverages a staked weighted trust through consensus. Peers are incentivized to rank each other using their unique and private cryptographic keys. Those who earn higher ranks receive additional awards, thus incentivizing them to provide more valuable and trustworthy benchmarking over the network. Additionally, the network also uses incentives to provide security on the network, providing additional rewards for peers who validate honestly.

Impact:
Imagine what progress would have been lost to humanity if, with each generation, we had to completely start over from a place of zero knowledge. This is what is happening to AI, with benchmarking programs that are siloed in centralized systems. Rather than building on the progress of others, researchers are forced to start from the beginning with their own models instead. Through its TAO token, Bittensor incentivizes the continuous improvement and fine-tuning of models in public. It creates a dynamic intelligence market that accelerates the pace of AI development in a more open and collaborative manner.

2) The Graph: Incentivizing Compute That Arms AI Agents With Blockchain Data

Open-source protocol for collecting, storing, and processing blockchain data.

Token Ticker: GRT
Market Cap*: $1.35 billion

What it is:
The Graph is a decentralized protocol for indexing and querying blockchain data. Leveraging its existing crypto-economic incentives for data indexing, it aims to create an additional decentralized marketplace for AI Inference and Agent services that can expand blockchain-based AI use cases.

How it works:
The Graph’s AI expansion integrates seamlessly with the Graph’s existing network of Indexers. It rewards them with the protocol’s native GRT for investing in AI-capable hardware like GPUs and for opting in to provide computational resources for AI model deployment and querying. It also discourages Indexers from using subpar neural networks by creating multiple mechanisms for developers to verify AI Inference. “Cheaters” — those who try to fool the system by using lesser networks or other means — get slashed if there isn’t consensus, leading to them losing some or all of their staked assets.

Impact:
By incentivizing its established Indexer base to offer compute services for AI Inference, The Graph gives the developers of AI-driven dApps the tools they need to perform significantly more complex operations (for example, autonomous AI trading bots or NFT market trend analysis agents that could be trained on blockchain data). It also protects developers against the centralization risks of using traditional providers, such as censorship, service instability, and restrictive agreements.

3) Livepeer AI: Incentivizing Pay-Per-Task Architecture for AI Video Creation

Developer platform creating globally accessible, ultra-low cost infrastructure for AI video

Token Ticker: LPT
Market Cap*: $381 million

What it is:
Livepeer AI is a decentralized set of AI capabilities available to developers for generative video apps, lowering the barriers to entry for developers with its commitment to open innovation and access. Currently, access to essential AI tools are mostly controlled by a few monopolies like Amazon Web Services or Microsoft Azure, which force developers to sign extractive agreements and pay high costs to rent, run and manage compute on their GPU servers. Livepeer uses its LPT token to incentivize its established network of compute providers — which are already transcoding millions of minutes of traditional video every week with plenty of GPU capacity to spare — to now provide that same support for developers and creatives using generative AI.

How it works:
The LPT token coordinates, bootstraps, and incentivizes participants to make sure the Livepeer network is as cheap, effective, secure, reliable and useful as possible. GPU providers, called “Orchestrators,” must hold LPT to transcode and distribute video on the network — the more they have, the more work they can do and the more fees they can earn in ETH. “Delegators” manage the flow of tasks by staking their LPT tokens, helping provide security and quality assurance to the network by directing tokens toward more reliable Orchestrators, based on capability and current load.

Impact:
Livepeer AI’s cryptoincentives enable a novel “pay-per-task” model that is fairer and more efficient for AI video developers and creators, who no longer have to pre-reserve expensive compute capacity. Centralized competitors have high employee overhead and GPU upkeep costs, which require them to charge renting models to remain cost efficient. In distributed networks like Livepeer, providers are oftentimes providing excess or latent resources, allowing them to be more flexible in offering better economics to end users. Through tokenization that enables a true peer-to-peer network, Livepeer is infinitely scalable and permissionless— as demand increases, the network returns even more value to the Delegators and Orchestrator nodes, making it even easier to scale.

4) SingularityNET: Incentivizing Shared AI Development To Achieve “AGI”

Decentralized platform creating a global public marketplace for AI research and services

Token Ticker: AGIX
Market Cap*: $768 million

What it is:
SingularityNet’s goal is simple: to speed up the development of “AGI,” or AI that is as smart or smarter than humans, in globally inclusive fashion. Most AI research is fragmented and done in closed-development corporate environments that slow the pace of innovation, and developers often face geographic and financial barriers to access or advance AI. Through decentralization, SingularityNet uses its token to create a free market for a wide range of AI services — everything from gene sequencing and speech recognition to image manipulation tools — accessible to anyone on the internet, incentivizing developers to produce AI services in public and at a fair market rate for AGIX token rewards.

How it works:
The AGIX token is used for value exchange and governance, with developers using the token to charge users for AI services on SingularityNet’s “AI curation market.” This market is essentially a discovery ranking mechanism. Token holders stake their tokens to a specific Agent and specific query in order to increase that AI Agents’ stake-rank, receiving AGIX rewards in proportion to how much that AI Agent’s services improve its reputation. If the Agent fails to process tasks correctly or otherwise has its reputation diminished, their stake is confiscated. As the volume of processing and clients for any single AI Agent grows, staking delivers diminishing rewards, with the largest rewards going to the earliest stakers.

Impact:
This tokenization model significantly incentivizes stakers to back higher-quality AI services and to always be on the hunt for new AI agents to stake and promote. This creates a more meritocratic and democratized AI research model, evening the playing field for smaller AI developers and researchers whose contributions otherwise can’t compete with large centralized operators in the global marketplace. It also allows for more interoperable research, as global AI teams are incentivized to work together in public rather than behind centralized company silos.

5) Injective: Incentivizing Developers to Build AI-enabled dApps

Network fueling the growth of AI-powered trading apps across decentralized marketplaces.

Token Ticker: INJ
Market Cap*: $1.84B

What it is:
There are certain core financial primitives that serve as the necessary building blocks for healthy markets and financial systems, yet centralized TradFi operators often restrict access and charge extractive fees for using them. Injective is working to create a freer and more inclusive financial system by making these primitives plug-and-play for DeFi apps, increasing settlement and trade execution in a highly decentralized, permissionless, and censorship-resistant fashion. One of these decentralized primitives is an MEV-resistant on-chain order book, which makes it easier for buyers and sellers to make decisions by improving market transparency. Another is a smart contract layer that provides shared on-chain liquidity, or access to capital, and oracles, which provide real-time, secure, and precise price data that makes it easier to have confidence while trading assets.

How it works:
None of this would be possible without the incentives of the INJ token, which plays a multifaceted role in governance, security, and value accrual. Market makers earn INJ tokens in 28-day cycles, based on a total score that rewards them based on their dual-sided liquidity, liquidity-depth, bid-ask spread, market maker uptime, volume, and participation in multiple markets. Injective also utilizes a Tendermint-based Proof-of-Stake consensus mechanism, allowing INJ token holders to earn rewards by staking their tokens to provide network security. Finally, all transactions programmatically return 40% of the protocol fees in INJ tokens back to the developer, through a “relayer fee share,” providing a major incentive for them to build dApps on the network.

Impact:
INJ’s token incentives create a positive feedback loop: as more AI-enhanced DeFi applications are built and used on Injective, more developers are incentivized with INJ tokens to join the ecosystem. This accelerates further innovation of AI apps, compared to centralized ecosystems like the Apple Store or Google Play store, which charge high extractive fees — as high as 30% on in-app purchases — without returning them to the developers who create those apps.

*Market capitalization figures as of August 2024 from CoinMarketCap.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

The answer is that this is already happening. FlipGuard is leveraging Livepeer AI to introduce dynamic text-to-image capabilities within messaging apps. FlipGuard is an interesting example of how forward-thinking companies can anticipate the potential for generative AI in messaging apps to boost engagement, creativity, and communication. With AI, communities can give their members free reign to create and communicate limitlessly with text-to-image, text-to-video or even text-to-audio.

The next step in the internet’s communication arc

From the early days of internet forums to the current era of social media, online expression has consistently evolved to become more visual and interactive.

Emails and text messaging brought quick, instant communication methods compared to traditional mail. Soon after, online forums and chatrooms allowed unconnected people from anywhere in the world to communicate instantly in real time. Social platforms like MySpace and Facebook then took center stage, bringing multimedia communication using photos and videos to billions of people for the first time.

Emojis first brought a new layer of emotional expression to texts, but GIFs and memes truly transformed digital interaction by allowing users to communicate complex emotions and cultural references quickly and effectively. These visual snippets, easy to create, share and consume, became a language of their own, permeating everyday communication and becoming essential tools for personal expression on the internet.

In recent years, still images and text have been overshadowed by video. Skyrocketing internet speed and the proliferation of smartphones made video the preferred medium for sharing content and engaging with others online. Instagram and TikTok elbowed their way into the heart of meme creation and now the short video format pioneered by these platforms is usually the most emotionally, conceptually and informationally dense medium of expression available to users today.

Video is dominant because it is universally accessible, easy to consume and share. While the platforms that exist today have attracted billions of users through harnessing the interactive capability of video, their offerings still lag behind the need for unique, personalized forms of visual communication. Users often find themselves searching for the perfect gif, emoji or video which, while useful, can feel impersonal and restrictive. But with generative media, the next iteration of digital tools will help users to encapsulate mood, nuance and thoughts more precisely than ever before.

Flipping the script on messaging apps with AI

Flipguard made its name through bringing Web3 to the Web2 world. They saw how messaging platforms like Discord had so much potential but were limited by a lack of tools, engagement and incentives. Their ability to work out what communities need to grow and express themselves better earned them the trust of Roblox and Drill Club, as well as a presence in 800 servers, 16,000 users and nearly a million transactions to date. The Flipguard team has been active in the Livepeer community as node developers and token holders since 2018, so when the opportunity came along to build on the network’s powerful AI ecosystem, they jumped at the chance to bring real-time text-to-image generation to messaging apps.

This feature allows users to craft unique images that perfectly match their message, making conversations more engaging and personalized. Imagine being able to type a description of your mood, an idea, or a scenario and instantly generating a custom image that encapsulates it—this is the new reality that FlipGuard is bringing to life. Companies can use Flipguard to crowdsource meaningful contributions, such as company logos or the visual direction for a new marketing campaign. The concept is simple, but the opportunities that generative AI tools bring to communities are not to be overlooked. The day-to-day running of a community is a job in itself, and that’s before taking into account the marketing, branding or design elements that need taking care of. With AI, these communities can sidestep expensive agencies and use the same communication tools to crowdsource a community logo, campaign images or design assets for social media platforms. It’s no secret that creatives have voiced concern about the impact of AI on their craft, which is why the Flipguard team is determined to create tools that support the creative process and add value to the culture that communities hold so dear - as opposed to trying to replace them.

Flipguard will soon launch its first set of AI tools, all of which provide interactive ways to help Discord users leverage AI to communicate more creatively. The first is a generative tool that enables users to create a one-of-a-kind gif that encapsulates the current feeling of the community. Creators earn upvotes for clips that resonate with the community and the winners are rewarded with digital assets.

The second is a playful reinterpretation of the first. An AI generated gif is posted and the community then competes to guess the prompt. Members are incentivised to create unique, culturally relevant content for an audience they understand and connect with on a deep level.

The third tool being launched is a Discord AI gallery, in which users can save images and transport them to any other server. This straightforward feature not only serves as a portfolio of each community’s most treasured creations, but also cross-pollinates its unique culture into other communities that can collaborate and grow together.

Flipguard’s first steps into integrating AI into messaging apps show just how much potential there is to make using messaging apps a more rewarding and engaging experience. Not only does AI give a low-cost and largely autonomous way of boosting engagement but it can also open up new revenue streams, both for companies and communities, if they have access to an affordable and open environment like Livepeer AI for experimentation.

How AI can disrupt messaging apps

Flipguard’s tools are just one way of using text-to-image for enhancing communication within messaging apps. The reality is that, now that Livepeer AI is available for builders to innovate regardless of their stage or financial backing, the only limitation is imagination. But here are a few ways that text-to-image could shake up communication in messaging apps to get the creative juices flowing:

1. Personalized Visual Responses: Instead of being restricted to static text or pre-existing clips, users can now generate custom images on demand, creating a response that could only ever come from them. Messaging apps can add another deeply personal layer to their offering making them more immersive and expressive. Apps that have an emotional and communicational edge will stand out in an increasingly crowded market.

2. Creative collaboration: In professional settings, particularly in creative industries, teams often collaborate through messaging apps. Text-to-image AI can supercharge this by enabling real-time generation of concept art or design ideas directly within the chat.

3. Interactive Storytelling: Messaging apps are increasingly being used for storytelling in both personal and professional contexts. Text-to-image AI enhances this by allowing users to generate images that bring their narratives to life, making stories more engaging and visually compelling.

From the personal to the professional, text-to-image to image has huge potential to boost the utility of messaging apps. The internet is a swirling maelstrom of words, images and sounds and what really counts more than ever is a way to cut through the noise and communicate more effectively and in a way that resonates with people.

Building the future of AI with Livepeer AI

FlipGuard’s integration of Livepeer AI is a glimpse into this exciting future. The best part is that this is just the beginning and the most imaginative builders can put their stamp on the technology that will create the media of the future.

Are you a builder that wants to empower creators to create limitlessly? Check out Livepeer AI here. Want to breathe life into your Discord community? Learn more about Flipguard here.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

Much of that progress is thanks to the humble diffusion model. But what are these models and how have they helped whip up a generative AI storm?

After reading this you’ll not only be able to impress your friends with your knowledge of what diffusion models are but also understand how they came to have such a big role in generative AI, along with some real world applications of this remarkable technology.

What Are Diffusion Models?

Diffusion models are a class of probabilistic models used in machine learning to generate data by iteratively adding and removing noise.

Imagine trying to restore a blurry photograph: the process involves gradually refining the image until it becomes clear. Similarly, diffusion models start with a noise-filled version of the data and refine it step by step to produce clean, high-quality output.

These models have gained prominence in generative AI due to their ability to generate remarkably realistic data, outperforming traditional generative models in certain tasks. They operate through a two-step process: forward diffusion, where noise (variations and disturbances in the data) is progressively added to the data, and reverse diffusion, where the model learns to denoise the data back to its original form.

The Evolution of Diffusion Models

Like much technology that comes to have an outsized impact, diffusion models began in relative obscurity, characterized by limited computational power and relatively simple algorithms.

They could generate data but often lacked the fine details and high fidelity seen in real-world data. Significant breakthroughs came with the introduction of denoising diffusion probabilistic models (DDPMs), making it possible to generate high-fidelity images and other data forms with remarkable precision.These advancements marked a turning point, as they drastically improved the quality and efficiency of diffusion models.

Today, diffusion models are at the forefront of generative AI, capable of producing high-resolution images, coherent text, and even complex biological data. This progress has been driven by continuous improvements in computational power, more sophisticated algorithms, and advanced training techniques. Researchers have fine-tuned these models to handle diverse and intricate data types, pushing the boundaries of what is possible in AI-generated content.

Types of Diffusion Models

Several types of diffusion models have been developed, each with its own respective strengths and unique applications. Here, we explore two of the most prominent ones and their specific pros and cons:

Denoising Diffusion Probabilistic Models **(**DDPMs) excel in generating high-fidelity images with intricate details. They are particularly good at creating realistic textures and structures. DDPMs have been used in projects like OpenAI's Sora which generates highly detailed images from textual descriptions. The primary drawback of DDPMs is their computational intensity. The iterative nature of the denoising process requires a huge amount of compute, making it slower and more expensive compared to some other models.

Latent Variable Diffusion Models **(**LDMs) can capture the underlying structure of the data, allowing for more efficient generation of complex distributions. Stable Diffusion is an open source, free to use latent diffusion model that produces photorealistic images from text and image prompts. Stable Diffusion differs from other models not only in that it is open source, but also that it uses reduced-definition latent space as opposed to the pixel space of images in image generation. This difference means that Stable Diffusion has much less intensive processing requirements and can be used on desktop computers. The free, open and affordable nature of Stable Diffusion played a big role in accelerating the AI video generation boom and demonstrated that open source models can rise as serious challengers to the top models like OpenAI’s Sora.

Real-World Applications

Diffusion models have found applications across various fields, showcasing their versatility and power. Here are some notable examples:

The most prominent use case for diffusion models is image generation. DDPMs and LDMs can produce high-resolution images, but are also able to fill in empty or missing parts of images. These AI models can improve existing image quality through Image-Super Resolution, in which a higher quality output is generated from lower quality input. These use cases mean that diffusion models are already being used in professional contexts, such as graphic design, advertising, fashion and entertainment to lower costs and boost high quality output.

Diffusion models are earning a name for themselves in the scientific world. A number of models are used in generating realistic biological data, such as protein structures or genomic sequences, aiding in research and diagnostics. Projects like AlphaFold by DeepMind leverage these models to predict protein folding, significantly advancing biological research. These models can predict complex structures computationally, saving time and resources in medical research.

In the gaming industry, companies like NVIDIA use diffusion models to enhance graphics and create realistic textures. Additionally, diffusion models are employed in movie special effects, generating lifelike animations and effects that were previously challenging to create.

Challenges and Future Directions

Despite their impressive capabilities, diffusion models face several challenges that researchers are actively working to address:

1. Computational Cost and Efficiency: The iterative nature of diffusion models can be computationally intensive, requiring significant resources for training and inference. Optimizing these processes to reduce costs and improve efficiency remains a key area of focus.

2. Scalability: As data complexity and size increase, scaling diffusion models to handle large datasets efficiently becomes crucial. Researchers are exploring techniques to enhance scalability without compromising performance.

3. Bias and Diversity: Ensuring that diffusion models generate diverse and unbiased data is essential, particularly in sensitive applications like healthcare and law enforcement. Developing frameworks to mitigate biases and promote diversity in generated data is an ongoing challenge.

Diffusion models have emerged as a cornerstone of generative AI, offering unparalleled capabilities in data generation and manipulation. From their early development to the cutting-edge advancements of today, these models have continually pushed the boundaries of what is possible in AI. These models are already having an outsized impact on AI video, but are held back by the lack of accessible tools for creators. That’s why Livepeer AI is supporting builders to develop the next generation of applications at the intersection of decentralized AI and generative media.

Are you a builder that wants to enable the next great wave of creativity and shape the future of media with your exceptional applications? Check out Livepeer AI and see just how quickly you can have an impact.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

Kicking off on September 23rd, this four-week event will bring together top developers and AI enthusiasts around Livepeer’s cutting-edge protocol to build the best new generative media applications.

Empower the next generation of creators

With AI, anyone can now create high-quality video content. Whether it’s using generative AI to create new content, or leveraging AI models for editing, upscaling, or subtitling, artificial intelligence will change the world of media as we know it.

Picture the impact: marketing execs, influencers, and creators of all kinds empowered to produce stunning videos at scale. All that stands in the way of this coming wave is the lack of tools to make creating video content effortless and affordable.

That’s where you come in. With incredible technology from Livepeer AI, Zora, Story Protocol, and Swarm Zero at your fingertips, you have the opportunity to shape this new frontier. Build something extraordinary—applications that will define the next era of video content creation.

About Livepeer

Livepeer’s mission is to build the world’s open media infrastructure. The Livepeer network is a decentralized compute network that has already been used to transcribe millions of minutes of video on a weekly basis. With the launch of Livepeer AI, video developers can now leverage low-cost compute to power the resource-intensive process of AI inference for generative video creation.

The future of media is AI. Livepeer AI is enabling that future by making building AI video tools easier and cheaper than ever before. Livepeer AI offers developers a full range of AI functions —text-to-image, image-to-image, and image-to-video capabilities—backed by the cost-effective GPU compute power of the Livepeer Network. In addition to generative video creation, Livepeer AI also provides developers with AI-powered video editing, upscaling, subtitling and more to support the end-to-end video development experience.

How It Works

Hackers will submit a project in one of three categories:

1. Social Media App: Create a fun and engaging web app that utilizes Livepeer AI pipelines to provide a novel user experience with viral potential.

2. Developer Tools: Create a tool that makes it easier for developers to leverage generative AI in their own applications.

3. Creator Tools: Build a generative AI tool that helps content creators meaningfully engage with their audience.

To get your imagination running, we have provided an example for each category.

1. AI Photo Booth: React-based web app that takes a visitor’s webcam output and uses Livepeer’s image-to-image pipeline to generate stylized portraits that can easily be saved and shared on social media.

2. Web-Based AI Workbench: Web app interface for research & development with Livepeer AI pipelines.

3. AI Monster Maker: Application that gives viewers the ability to generate artwork of monsters for a tabletop RPG live stream.

You won’t be going it alone. Our teams will be available for 1-1 calls, mentor workshops, and deep technical overviews to ensure your project hits the mark. In addition, we have amazing bounties from Zora, Story Protocol, and Swarm Zero that will be announced soon.

Support and Structure

All developers will have access to a full support team, including a Program Manager, Marketing Manager, and Operations Manager, to guide you through each step.

• 1-1 mentorship: For tailored advice and insights.

• Technical deep dives: To master the tech stack and tools you’ll be using.

• Custom marketing: To help you promote your project with a personalized plan, dedicated emails, and a tailored hackathon homepage.

• Logistics: From scouting participants to team-building, we’ve got you covered.

Join Us

This is more than a hackathon; it’s a launchpad for the next wave of AI-generated media. Livepeer is leading the charge into decentralized AI, and we’re inviting you to shape the future with us.

Find out more here.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.Twitter | Discord | Website

The risks of centralization became all too clear in July, when a faulty Crowdstrike update sparked a worldwide outage for Microsoft clients worldwide. The cost to enterprises was stunning: for example, Delta reported at least a half-billion dollars in losses after having to cancel more than 6,000 flights and reroute hundreds of thousands of passengers — and that doesn’t even take into account the major hits to its reputation and the pending liabilities it faces from angry customers and federal regulatory groups in the coming days.

That failure crippled more than just major companies.  Small businesses from locksmiths in Australia to restaurant owners in New York City were left struggling to access critical files or pay employees. And centralization risk will only get worse if businesses continue to rely on traditional solutions in the age of generative AI.

“We’re reaching an age where a lot of functions that were typically done by humans … are going to be delegated by AI agents. And so these types of issues are going to become more prevalent,” Ifeoma Ajunwa, an Emory University law professor, recently told reporters.

The danger of exposing your business to a single point of failure is just one of the many risks of centralization. The reliance on just a few centralized cloud providers for compute has led to other significant downsides for businesses, including artificially high costs, restrictive lock-in requirements, and crippling bottlenecks — all concerns that are especially pertinent to generative AI companies and developers as they build the applications of the future.

The drawbacks of the web2 cloud for generative AI

Centralized cloud systems, in which just a few companies control access to key compute tools or capacity, are prone to significant drawbacks including …

Centralization risk

• Currently, NVIDIA holds 92% of the market share in the GPU compute used in AI data centers space, while Microsoft and OpenAI hold 69% of the market for foundational AI models and platforms.

• Industry monopolization doesn’t just expose companies to a single point of technological failure, as Microsoft users faced in the wake of the Crowdstrike outage. It also poses significant regulatory and business risks as well: If just one of these major companies fails, or is closed down by a government, all those who rely on it will collapse too.

Extractive agreements

• Because they face little competition, the leaders of centralized industries are able to force vendors to accept unfavorable terms that may lock them into long service agreements with fixed capacity, or may require them to rent servers and reserve space even if they don’t use them.

• The combination of high costs and unideal terms results in significantly less access for startups, academics, or individuals who do not have the buying power of more established competitors.

High costs

• Regulations favor incumbents, limiting access for competitors that become even worse amid supply chain shortages that favor established cloud operators (like AWS or Microsoft Azure) and hardware providers (including NVIDIA and AMD). This leads to consolidated compute in the hands of a few industry actors that can then set higher prices as a result.

Restricted access

• The closed loop systems created by centralized compute can prohibit certain people from accessing generative AI and other resources.

• These permissioned systems mean that legacy cloud giants like Microsoft or market-controlling hardware providers like NVIDIA decide who can use these transformative technologies.

These drawbacks were clear in the past. However, the advent of generative AI has exacerbated them. With such high demand and limited supply, centralized gatekeepers are able to charge even more exorbitant fees while having even less incentive to improve compute quality or agreement terms. And that effect is particularly felt by non-legacy clients and smaller generative AI companies, developers, and researchers.

How decentralization can help generative AI applications

The pitfalls of centralized compute demands a novel solution. A decentralized compute model, one in which generative AI needs are distributed across a broad variety of cloud compute providers, provides a new possibility for open access inference that is more cost effective and reliable, as well as infinitely scalable on demand.

Less risk with decentralized compute providers

• There is no single point of failure, with cloud compute capacity spread out between a large network of providers. This makes it harder for businesses to lose access to services due to a cyber attack, software crash, regulatory crackdown, or any other unforeseen event that can damage a centralized operator’s integrity.

Less fee extraction with more competitive services

• Providers that charge exorbitant fees or prices will be outbid by others, leading to a more efficient market for cloud compute services. A decentralized system gets rid of monopolized services and extractive middlemen, leading to less costly overhead and more savings that can be passed onto users.

Reduced costs with pay-per-task marketplaces

• While there is still a huge demand for generative AI services, there is also an ocean of small, medium, and large-scale cloud providers who are capable of meeting it in a decentralized system.

• Rather than pay for compute capacity they don’t need, companies can operate on a pay-per-task basis, greatly increasing cost efficiency and reducing risk by allowing them to scale or downsize production as they see fit.

More equitable access and contract terms

• Because there is no centralized provider, a decentralized system can offer permissionless cloud compute services on demand, without any long-term service agreement or any single entity dictating who can access services and who cannot.

• That greater flexibility helps level the playing field and expand access for less-established companies, including individuals, startups, academics, and other non-enterprise institutions looking to access generative AI compute services.

Case Study: Decentralized compute for generative AI videos

Many businesses believed cloud computing prices would drop over the years, as new technologies emerged to lower costs. Instead, cloud inflation has been the norm since the arrival of generative AI, with a reported year-over-year increase of 3.7% from early 2023 to 2024 and an upward trend since September 2022.

A number of industries are facing serious ramifications: consider video creation, a sector that AI has transformed by taking a process that used to require dedicated crews of workers and hours of editing and shifting it into tasks that can be performed with just a few text prompts. Although OpenAI’s Sora demos and the top open-source AI video model Stable Diffusion have showcased the industry’s potential, it remains inaccessible to many due to the high cost and limited compute offered by centralized market leaders like Microsoft Azue and Amazon Web Services.

Using a decentralized compute network similar to what Livepeer has built, video developers can start addressing these challenges by leveraging permissionless networks to access low-cost, high-performance processing that is globally available and affordable.

On Livepeer AI, video app developers are able to integrate gen AI features — including text-to-image, image-to-image, and image-to-video conversion — on a per-task basis rather than having to reserve expensive compute capacity they might not use, as they currently have to do with centralized cloud providers.

It is able to do this by tapping into Livepeer’s already-established network transcoding millions of minutes of video weekly, making it one of the largest and most reliable decentralized sources of affordable AI-ready compute.

Developers can scale these AI efforts infinitely, since all cloud compute capacities are performed through a permissionless decentralized system: there’s no need to negotiate new agreements with a single centralized provider, since compute is spread out among as many competing providers that are needed to meet their demand.

Emerging decentralized compute technologies and networks like these will be critical to producers going forward — and if adopted, they could enable a future where AI generative video is the de facto norm for content creators across the globe.

Learn more about Livepeer AI here.

Sign up for the waitlist to stay updated.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

Continuous Innovation: Advancing Livepeer AI

As Livepeer keeps building its AI video capabilities, more and more node operators and app builders are onboard and using the alpha. The latest release brings several exciting enhancements:

• Speech-to-Text via Whisper. Seamlessly convert audio to text.

• Stable Diffusion 3 Support. Integration of the SD3 model into Livepeer AI.

• Customization Options. New metrics and customizable inference steps for generative AI media outputs.

These updates provide more tools for developers and orchestrators to create and optimize AI-driven video applications.

Key Stats

The on-chain statistics from the early node operator and app builder groups running the alpha show continuous growth and adoption:

• 27 AI node operators performing jobs.

• Approximately 41,000 Text-to-Image jobs performed.

• Around 6,000 Image-to-Image jobs performed.

• Nearly 20,000 Image-to-Video jobs performed.

• About 2,500 Upscale jobs performed.

• 845 winning AI tickets paid out.

DALL-3: Decentralized AI Demo

DALL -3, a decentralized AI demo built on Livepeer, showcases the innovative applications of Livepeer AI. This demo illustrates how decentralized AI can provide robust, scalable, and cost-effective solutions for video processing and generation.

Recognitions

Livepeer's contributions to the intersection of AI and blockchain continue to gain recognition, leading to significant partnerships and mentions in industry reports.

Inclusion in Grayscale Decentralized AI Fund

Livepeer is proud to be included in Grayscale's new Decentralized AI Fund. This recognition underscores Livepeer's role as a key enabler for the future of AI media through its decentralized GPU network, which performs thousands of AI video compute jobs.

Grayscale Report on AI and Crypto

Grayscale's report on the intersection of AI and Crypto highlights Livepeer as a prime example of a provider of AI compute resources. This acknowledgment reflects Livepeer's commitment to leveraging blockchain technology to advance AI capabilities.

Grayscale

@Grayscale

Resources Needed for #AI Development

155

9:45 AM • Aug 1, 2024

Empowering Innovators: Livepeer Grantees Building the Future

Livepeer continues to support innovative projects through its grants program, fostering creativity and technological advancements in the Web3 space.

ViewFi Grantee

ViewFi has been announced as a recent Livepeer grant recipient. ViewFi is pioneering a video marketing tool for Web3, enabling users to add rewards for their audience and use the ViewFi network to reach Web3 users. Their integration with Livepeer Studio for live streaming promises to drive significant volume and traffic to the network, benefiting the entire ecosystem.

Origin Støries Feature in MusicTech

Origin Støries, another Livepeer grant recipient, was featured in MusicTech for their innovative approach to fan-sourced, AI-generated visuals at live shows. This project allows fans to contribute to the visuals of live performances from anywhere in the world, showcasing the potential of Livepeer's open video infrastructure in creating engaging, interactive experiences.

AI Video Startup Program

The AI Video Startup Program continues to accelerate the development of innovative startups at the intersection of AI and media. Selected teams receive grants, mentorship, and early access to Livepeer's AI video infrastructure, fostering the next generation of AI-powered video applications.Join the AI Video Startup Program now.

Articles

July saw the publication of two impactful articles:

**AI in Media: How Generative AI is Transforming Video Production**This article explores how generative AI is transforming video production, highlighting the capabilities of leading AI video generation models and the role of decentralized networks in democratizing access to AI tools.

**How Open Job Marketplaces are Simplifying AI Compute for Developers**Discusses how decentralized, open job marketplaces can slash the costs of AI compute and shift complexity from developers to the network, making AI more accessible and affordable.

ETH CC in Brussels

Livepeer had a strong presence at ETH CC in Brussels, engaging with the vibrant crypto community and showcasing the potential of decentralized AI and media. The event was an incredible opportunity to connect with key stakeholders, including node operators, partners, and the broader blockchain community.

Highlights from ETH CC include:

• Panel Discussions. Livepeer core team members participated in insightful panel discussions about the future of AI media enabled by open and cost-effective networks like Livepeer.

• Networking. ETH CC was an opportunity to interact and get valuable feedback from Livepeer users streaming the conference and various side events.

• Community Engagement. Meeting orchestrators provided valuable insights and fostered stronger community ties.

Looking Ahead

As we look forward to August and beyond, Livepeer remains committed to pushing the boundaries of decentralized AI and video technology. Stay tuned for more updates, and join Livepeer at our upcoming events, including Token 2049, where there will be a dedicated event to showcase the latest advancements.

Thank you for being part of the Livepeer community and supporting its journey towards building the world's open video infrastructure. Let's continue to innovate and shape the future of decentralized video together!

For more information and to stay updated, visit:Twitter | Discord | Website

Generative AI models like Sora, Runway, and Stable Diffusion have fundamentally transformed video production, making it more accessible than ever. To harness this potential, Livepeer welcomes the first cohort of AI video startups, ushering in the next generation of applications at the intersection of decentralized AI and generative media.

What is the AI Video Startup Program?

The Livepeer AI Video Startup Program is a bespoke three-month initiative designed to accelerate the development of decentralized AI video technologies and applications.

The inaugural cohort of eight startups was hand-picked by the Livepeer tech team based on their commitment to open-source and decentralized technology, a focus on creating nimble, user-centric products, and a strong interest in using the Livepeer network for AI inference tasks.

Livepeer AI enhances video capabilities on the network. Startups will focus on integrating generative AI features into their applications, such as text-to-image, image-to-image, image-to-video, upscaling, and speech-to-text. Developers can use established pipelines or build and deploy their own, working closely with the Livepeer AI engineering team.

How Livepeer Supports Startups

The program offers selected startups $20,000 in grant funding, infrastructure credits for AI inference and transcoding, and expert mentorship from the Livepeer tech team.

Participants will also gain early access to Livepeer’s cutting-edge AI video infrastructure, with opportunities for co-marketing and partnership activation. The program culminates in early October.

Meet Livepeer’s First AI Cohort

Livepeer is proud to introduce the startups joining the AI Video Startup Program:

Flipguard: Transforming community engagement on Discord with AI-powered features like image creation, event management, and interactive games. Flipguard aims to enhance its offerings with new quest features and an enriched user experience.

Katana: Revolutionizes content creation by auto-generating engaging short clips and summaries from long-form videos. Katana leverages Livepeer’s distributed transcription API to train its own models for superior video content creation.

Newcoin: Creates dynamic video content by converting text prompts into video files using Livepeer’s API. This tool is set to redefine video interaction on the Newcoin OS platform.

Operator: Specializes in meme creation, using custom AI models to produce themed backgrounds and distinctive visual styles. This autonomous agent generates memes with a unique aesthetic and viral appeal.

Origin: Enhances its VPM application to offer co-created video experiences with improved user experience and efficiency. New features, including advanced video editing tools and integrations, will broaden its market reach and capabilities.

Refraction:  Integrates Livepeer AI into its Regenerate video filter tool to provide powerful generative AI functionalities like text-to-image and image-to-video transformations. This enhances real-time visual effects for streaming and live performances.

Supermodel: Leverages Livepeer’s infrastructure to deliver real-time trend insights and forecasts for the fashion industry. By applying AI models to video feeds, Supermodel provides brands with valuable consumer insights and demand forecasts. (Currently in stealth).

StreamEth: Implements a suite of AI-powered features to revolutionize video content management and viewing experience. Their software unlocks video clipping workflows and multilingual video alteration.

Why Livepeer?

Livepeer disrupts the centralized grip of traditional compute providers by leveraging thousands of GPUs already on its network. This significantly reduces costs, enabling on-demand AI video generation without the need for expensive, dedicated servers. With Livepeer’s established network transcribing millions of minutes of video weekly, startups can access high-reliability AI compute affordably.

Livepeer AI an open-source, decentralized marketplace of GPU power, ensuring that advancements in AI video technology remain transparent, resilient, and accessible. By fostering an open-source ecosystem, Livepeer not only supports collaborative development but also provides a fertile ground for groundbreaking innovations in AI and video.

You can expect regular updates from the AI video program, both from Livepeer and the teams building the next generation of applications.

Watch this space for the latest news and read more about the Livepeer AI here.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

The AI video craze was kicked off by Runway back in 2023 with the launch of Gen-2, its pioneering video generation model. It was a huge leap forward from the clunky, warped images that the public were used to. The real watershed moment was the launch of OpenAI’s Sora. Sora not only created astoundingly high-quality videos, it was also capable of generating clips that could feasibly pass for real-world footage. Things would never go back to the way they were and video production was changed forever.

But all this innovation is private, closed source and often only available on a subscription-basis. That’s why free-to-use, open-source options like Stable Diffusion are rising as serious challengers to the top generation models. Combined with affordable, decentralized access to AI compute, there is a clear path towards an open AI video infrastructure for everyone.

This post will look at how the leading AI video generation models have changed video production, what projects they’ve cooked up, as well as how open-source models are disrupting this hotly contested market.

Sora sets a new standard

Sora is a video-generation model from OpenAI, the company behind the agenda-setting chatGPT. Sora can currently create photorealistic videos up to a minute long, based on user prompts in either image-to-video or text-to-video format.

The launch of Sora was a big break from everything else on the market. Instead of the dream-like previous incarnations of text-to-video generators, complete with images that ooze and flow like a surreal low-fi tapestry brought to life, Sora could produce videos that, while not instantly convincing that they could be real footage, might make the average internet user double take.

OpenAI

@OpenAI

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions.

openai.com/sora

Prompt: “Beautiful, snowy

132.4K

12:14 PM • Feb 15, 2024

Sora’s launch videos created waves as they showed a huge variety of the model’s capabilities, from a herd of wooly mammoths shuffling through a snow-covered mountain plateau to a happy young couple walking through a bustling, wintry city-scape.

Despite the millions of overawed internet creatives, Sora remains off-limits to almost everyone. As of now, the model is only available to a select few testers, filmmakers, visual artists and designers. But that is soon set to change, with OpenAI CTO Mira Murati reporting that the video generation model will be available in 2024.

But Sora isn’t the only video generation model on the market. OpenAI’s model has a growing list of competitors, most notably Runway and Pika Labs.

Runway

When the company released its Gen 2 system, it was a huge leap forwards in the public consciousness about what AI could create.

Gen 2 was both image-to-video and text-to-video and would duly create something approximately similar to whatever users wanted, up to 3 seconds in length. Now, creators can generate 4 second videos as standard, with the option to extend the shot up to a total of 16 seconds, for both image and text to video prompts.

In February 2024, Runway released some details about Gen 3, highlighting fine-grained temporal control for dramatic transitions and photorealistic humans with a diverse range of actions, gestures and emotions.

Runway is really leaning into the cultural and serious filmmaking element of generative AI. It even created its own AI film festival that takes place every year. Creators from around the world can submit films up to 10 minutes in length that include AI-powered tooling, via a Runway link. The top prize is $15,000 and 1 million Runway credits.

Last year, filmmaker Daniel Antebi won the top prize with his unnerving film Get Me Out which sees a man struggle against nightmarish supernatural forces that trap him in a seemingly normal, suburban American house. It might not be an Oscar winner, but it shows that AI can be effectively integrated into serious, narrative filmmaking.

Pika

Pika is in some ways an offshoot of Runway’s original generative video model.

Pika co-founder Demi Guo was determined to make a film using Runway. Guo labored over her film for months, but she ended up frustrated with what the combined efforts of a group of highly technical people could produce using the generator.

So, she dropped out of Stanford University to launch Pika, with the dream of developing a more user-friendly AI video generator for amateur creators. Since then, then Pika has raised millions in both funding and users.

Pika also offers text, image and video-to-video conversions and gives users the chance to mix up their videos by changing dimensions, altering characters, environments and content length.

Pika isn’t intended to be used by professional filmmakers, but it’s still more than capable of creating compelling images that can craft a narrative. This sci-fi trailer made using Pika shows that you don’t need photorealistic quality to make something that can instantly create a powerful atmosphere and hold a viewer’s attention:

Pika founder Guo told Bloomberg that Pika will be able to create high-quality videos up to a minute in length and that within two years it could be used to create a feature film. But that doesn’t mean there won’t be a significant impact on production and filmmaking before then, many impactful clips and ads are much less than 10 seconds long.

Corporate partners look for AI’s next top model

AI video isn’t just some kind of boredom-killing experiment. It’s already serious business. For that reason, there are already some prominent partners lining up to implement these models into their business offering.

In April 2024, U.S. software maker Adobe said that it was in the early stages of allowing tools like OpenAI’s Sora into its popular Premiere Pro app. While this sounds promising, it’s worth noting that Adobe’s bullish statements on AI might be more to do with boosting its lagging share price than building actual products.

Meanwhile, recently-revived Toys ‘R’ Us became the first big brand to use Sora in its advertising. The ad premiered at the Cannes Lions Festival, an indulgent get-together for the international advertising glitterati, and received mixed reviews. Some praised the innovation from Native Foreign, the agency behind the ad, while others criticized the lack of visual consistency, particularly the appearance of the main character. Regardless of the advert’s finer technical or artistic points, it’s a watershed moment for the use of AI in advertising.

It’s not just Sora getting some high profile attention. Madonna’s Content Director Sasha Kasiuha said they used Runway to create state visuals for the Queen of Pop’s Celebration Tour.

Source: https://runwayml.com/customers/generating-new-worlds-for-madonnas-celebration-tour/

Hollywood hides behinds its hands

Many commentators have described AI video generation as an existential threat to the TV and film industry. This is almost certainly an exaggeration but it is true that AI has leveled the playing field for video production.

Scenes that would require a team of experts to create can now be conjured up with just a few short prompts. This not only reduces the potential payroll of production teams, but also the overall length of shoots. AI can also help with color correction, scene transitions and other time consuming editing jobs.

AI video generation is significant because it puts professional-quality tools in the hands of the wider public, without requiring detailed know-how. This is a game changer for independent creators and even smaller studios. Professional studios will need to adapt to stay competitive and integrate AI into their workflows.

While AI video might not be set to replace Hollywood production studios overnight just yet, that doesn’t mean the impact isn’t already being felt. Tyler Perry, a US TV and film mogul, said that he halted an $800m expansion of his Atlanta studio after the launch of Sora. Perry stated that although he is impressed at how the technology will improve film production, he foresees the loss of a lot of jobs across the industry.

Open source models

There’s one thing these models all have in common: they’re private and closed source.

Open vs closed source is a divide in the tech world as old as the industry itself. There are those who think tech should be freely available and those who think the source code behind innovative products should remain behind closed doors. In the past, the closed-source option has usually won out because for-profit products have better earning potential for investors, meaning these products are easier to fund.

But AI video could be set to buck this long-established trend. Open-source AI video models like Stable Diffusion are putting up some stiff competition.

Stable Diffusion is widely available to anyone who has the technical capability to implement and run it. It can be deployed on both local and cloud devices. Stable video diffusion is already producing promising short videos, and as the models rapidly evolve, we can expect even higher quality outputs in the near future. Development and updates can also be much faster than on closed source models due to the size and activity level of a large community of users. This large-scale collaboration is one of the biggest reasons why open source models are able to deliver cutting edge-performance updates at a rate that can even outpace large, well-funded models. Stable Diffusion is generally free to use, aside from the costs to run the models, which can be significant.

Any way you look at it, generating AI video is expensive. This is because you need huge amounts of compute for the simplest text-to-text tasks, let alone creating the complex results needed in AI video.

• Runway Gen-3 is $1 per 10 second video generated.

• Sora can be up to $6. A recent music video made with Sora required up to 700 generations, totaling over $4000.

• Luma AI charges $400 for 30 Dream Machine generations

The costs are high because the computing demands are heavy. On top of that, the servers capable of providing this power are in short supply. This is because almost all of the AI-ready compute is controlled by a small handful of companies like NVIDIA, Microsoft and Amazon. With all the power and the ability to control prices for an ever-increasing demand for AI compute, there isn’t much incentive to change. Added together with the prominence of private, closed-source AI video models, it’s clear that the future of video is at risk of becoming yet another tech black box.

This is where open source models, combined with decentralized networks have a real opportunity to smash the chokepoint on AI. Decentralized networks are a web of interconnected GPUs capable of processing AI tasks. Using an open job marketplace model on a decentralized network like Livepeer means that developers and small businesses can simply process individual inference jobs at minimal cost instead of renting whole servers. This essentially sidesteps the biggest cost factor for AI video compute.

A decentralized, open job AI marketplace like Livepeer is the most cost-effective and accessible option for developing the full potential of AI video. With affordable compute, the ability to experiment with the models and build innovative applications is open to all creative developers and companies, regardless of how well they are funded.

AI video is still a very young technology. Access to new, exciting technology is often gated and expensive, but drops down as competition comes along. In order for that to happen to AI video, the underlying foundations of the technology need to be changed, or competition will simply be priced out of the market.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.

Twitter | Discord | Website

Decentralization through open job marketplaces is a powerful alternative to the current AI bottleneck that can drastically lower prices, boost profitability for server providers and create a dynamic AI market through competition.

Let’s look at why AI compute is so expensive and what can be done about it.

Why AI compute is so expensive

The dawn of the AI era created a global gold rush for compute. While this is great for the handful of companies that enjoy a monopoly on computer clusters, it’s a disaster for innovation and accessibility of AI tools for small startups, independent developers, academic researchers, and other non-corporate actors . It’s also a recipe for sky high prices.

The world’s most famous chatbot, OpenAI’s ChatGPT, reportedly costs USD$700,000 per day to operate. And that’s just generating text. The most common generative AI media jobs, such as text-to-image, image-to-image and image-to-video require orders of magnitude more compute to generate. In fact, generating just 60 seconds of video could take up to 30 minutes of GPU time.

NVIDIA, a $3 trillion company, currently provides a staggering 92% of the GPU compute used in AI data centers, while Microsoft and OpenAI scoop up 69% of the foundational models and platforms market. With the necessary power for the world’s most transformative technology centralized and locked away behind closed doors, there’s no room for competition and little prospect of prices lowering.

Driving down prices with decentralization

Decentralized networks are a radical alternative to the current monopoly on AI-ready compute. These networks use blockchain-based coordination systems to connect geographically disparate GPUs and bring their compute power together in a vast, decentralized network.

Decentralized networks do not rely on servers from any one entity, which allows them to break the monopoly on pricing and access. They are most effectively able to outcompete centralized offerings through access to idle computing power. GPU providers then have extra opportunities to increase the profitability of their hardware, while lowering the cost.

Here’s how an AI video inference workflow could look on a decentralized network:

1. Select your decentralized GPU network provider of choice.

2. Scan their network for GPU resources based on location or capacity.

3. Select a server and pay for the required rental time.

4. Deploy your project and receive results.

Decentralization of GPU access is an enormous step forward, but a central component of this innovative model is also a potential blocker for lowering costs even further: having to rent a server.

Regardless of what you need done, you’ve still got to rent GPU resources, manage them, deploy software and pay for their running costs even when idle. A better solution must go even further to reduce responsibilities and costs.

Job-based marketplaces: reimagining competitive AI inference

Imagine a scenario in which you didn’t have to think about GPUs or rental responsibility. Instead, you simply pay for what you use, when you use it. No heavy hardware overheads, just the job you submit, on demand.

The good news is that this scenario actually exists. It’s called a decentralized open job marketplace and it’s one of the most attractive options for smaller developers, startups and most actors outside well funded companies. Decentralized open-job marketplaces are also networks of decentralized GPUs, but developers don’t have to reserve time on servers in order to process their AI tasks. Instead, they simply submit their individual AI inference tasks to the network and pay for them individually.

With a pay-per-task model, network users can immediately avoid the prohibitive costs of server rental. But open-job marketplaces also have other cost-saving initiatives baked in. When an AI task is submitted to the network, it doesn’t just go to the first server provider to see it. Hardware providers must then compete on price to complete the tasks as cheaply as possible. Because of this, network orchestrators are incentivised to complete tasks with the smallest possible charge on top of their own electricity costs.

Livepeer’s own AI inference offering is based on this very principle. Livepeer is a decentralized video processing network, dedicated to building the world’s open video infrastructure for the $100 billion streaming market. With ready access to idle compute power across thousands of GPUs in the network, Livepeer is uniquely placed to solve the AI video compute bottleneck and bring down costs for AI video.

Livepeer’s network is open access and the infrastructure providers compete to offer the services you need for the lowest price. All you have to do is submit a job to Livepeer’s network of GPUs and it will be done as cost effectively as possible, scaling automatically to meet your needs.

Here’s how an AI video inference job could look on Livepeer:

1. A user of an AI enabled application built on Livepeer takes an action that requires an image-to-image, image-to-video, video upscaling or subtitle generation job. The app submits the job to the network.

2. The job is then routed through the Livepeer AI gateway and picked up by a network orchestrator running a GPU for the lowest possible price.

3. The job is executed, resubmitted and the result is delivered back to the user..

With a pay-per-task model, the prohibitive overheads of GPU reservations for AI video generation are completely removed. Server providers are able to maximise the productivity and profitability of GPUs already running on the network, while users can make the most of idle servers to generate AI video at a fraction of the usual cost. It’s a win-win situation.

That’s not to say there aren’t a few tradeoffs that come along with a pay-per-job model rather than a rent-a-server model. Decentralized open job marketplaces favor popular AI models, which means that the network needs a lot of providers for the most common job types for it to be cost-effective.

AI models need to be open so everyone can access, run and compete on them. Proprietary models could be run once contractual agreements are set up with operators, but that’s not so different from using cloud-based services. Niche, custom or proprietary workflows may still need to use a server-rental marketplace.

Competition must prevail for an affordable, accessible AI

Without fundamental change, the development of AI will take place behind closed doors by the world’s most powerful and unaccountable companies. AI’s potential for transformation is hard to understate, yet its successful implementation requires open access and affordability. This will not happen without a systematic dissembling of the structures that allow them to lock out competition and keep prices high.

Decentralized, open marketplaces can break the AI computing bottleneck and incentivise GPU providers to compete on price. Livepeer’s network is already creating the world’s open video infrastructure that will support the next generation of video, cheaper and more efficiently than anywhere else. If you want to see a better, more affordable AI for everyone, come and join us.

Click here to boost your GPU profitability with Livepeer.

If you’re a video developer, check out Livepeer Studio for a full suite of streaming and AI services.

If you’re a developer tinkering with generative AI media, start experimenting Livepeer AI now.

Livepeer is a video infrastructure network for live and on-demand streaming. It has integrated AI Video Compute capabilities (Livepeer AI) by harnessing its massive GPU network and plans to become a key infrastructure player in video-focused AI computing.Twitter | Discord | Website

Livepeer AI Launch

One of the most significant milestones was the successful launch of Livepeer AI in May.

This critical infrastructure upgrade allows for advanced AI video capabilities, including text-to-image, image-to-image, and image-to-video transformations. Livepeer AI provides a robust and scalable environment for orchestrators to earn fees by performing AI inference jobs on their GPUs, marking a significant step toward decentralizing AI video compute.

Additionally, developers can experiment with Livepeer AI, benefiting from enhanced tools and documentation that allow them to easily integrate AI capabilities into their applications, leverage cost-effective AI compute, and build innovative solutions within the Livepeer ecosystem.

AI Startup Program Launch

Livepeer introduced the AI Video Startup Program in line with Livepeer AI launch. This three-month program is designed to accelerate the development of innovative startups at the intersection of AI and media. Selected teams will receive up to $40K in grants, mentorship, investor networking, technical support, and exclusive early access to Livepeer’s AI video infrastructure. This initiative aims to foster a new generation of AI-powered video applications, driving forward the vision of decentralized media.

GEN VIDEO Event in Berlin

On May 23, Livepeer hosted the GEN VIDEO event in Berlin, which brought together a diverse group of participants to discuss the opportunities for decentralized video creation and broadcasting in the era of generative AI.

The event was a success, featuring thought-provoking panels, workshops, and networking opportunities. Highlights included discussions on content authenticity, media verifiability, and the future of AI-enhanced video workflows.

To top it all off, the event concluded with a lively party hosted by Livepeer Video Disruptor Refraction, where participants enjoyed the presentation of Regenerate - the first ever onchain visual streaming software, music, drinks, and the chance to unwind and connect in a more relaxed setting.

Watch the panels and the video disruptor spotlights on LivepeerTV.

Early Node Operators and App Builders Join Livepeer AI

Livepeer AI has seen a growing number of early node operators and app builders joining and actively participating in the alpha phase. This community engagement has been instrumental in testing and refining the network’s capabilities. The promising results from these early adopters have demonstrated the reliability and efficiency of Livepeer's infrastructure, setting the stage for further expansion and innovation.

So far 26 AI node operators have been actively performing jobs on-chain, executing more than 38000 text-to-image and more than 18000 image-to-video tasks.

Find out more about Livepeer AI and how to get involved.

Key Partnerships and Events

Livepeer's commitment to building a robust and inclusive ecosystem is reflected in its strategic partnerships and community engagements:

C2PA Membership

Livepeer became the first open infrastructure network to join the Coalition for Content Provenance and Authenticity (C2PA).

This partnership aims to combat AI-generated deepfakes and misinformation by integrating verifiable metadata into content creation and distribution processes, enhancing trust and authenticity in digital media.

The C2PA is an open technical standard providing publishers, creators, and consumers the ability to trace the origin of different types of media. Other members include major industry players like TikTok, Adobe, Google, Sony, Intel, BBC, Microsoft, and OpenAI. Livepeer's involvement underscores its commitment to supporting content authenticity and rebuilding trust in media through decentralized infrastructure.

To find out more watch this video where Eric Tang, Livepeer Cofounder and CTO, and Andy Parsons, Senior Director of the Content Authenticity Initiative at Adobe, talk about content authenticity and provenance in AI video.

Web3 AI Day

Livepeer will be at the Web3 AI Day organized by Encode Club, with Founder Doug Petkanics speaking on the Creative AI Panel. This event highlights the intersection of Web3 and AI, emphasizing how DePIN tech can enhance and support AI applications.

Plus1 Partnership

Livepeer partnered with Plus1 for live streaming events, such as La Noche in Barcelona, demonstrating the capabilities of Livepeer's decentralized video infrastructure in real-world scenarios.

Open Source AI Summit

Livepeer ishappy to join the Open Source AI Summit, co-hosted with NEAR Protocol and KryptoPlanet. This summit features thought-provoking talks on the intersection between Web3 and AI, showcasing rising projects and builders in the space. It is an excellent platform for Livepeer to engage with the community and highlight its role in advancing AI and decentralized technologies.

ETHCC Happy Hour w/ Livepeer

Livepeer is hosting a happy hour during ETHCC in Brussels, providing an informal setting for developers, content creators, and video enthusiasts to connect and discuss the future of decentralized video.

Treasury Proposals Passed

Two significant treasury proposals were passed, marking a new phase of development and innovation for Livepeer:

Aquareum: The Video Layer for Everything

Aquareum aims to integrate rich video functionality with decentralized social networks like Farcaster, Lens, Nostr, and Bluesky's AT Protocol.

This initiative will provide an iOS/Android/Web app with a Twitch/YouTube interface, allowing millions of users to watch and create video content seamlessly. The project will include features like livestreaming, VoD, clipping, multistreaming, and AI video generation, ensuring scalable and cost-effective video distribution.

Read the full proposal here.

Realtime Interactive Video SPE

This initiative focuses on creating a new market segment for real-time interactive video, leveraging Livepeer's decentralized video transcoding technology. T

he project aims to increase demand for Livepeer's services by targeting high-streaming activity markets and providing an open-source template app to facilitate community innovation. The expected outcome is a significant boost in network scalability and robustness, benefiting the entire Livepeer ecosystem.

Find out more by reading the proposal.

Conclusion

The past two months have been a period of rapid progress and exciting developments for Livepeer.

As we enter the expansion phase of Livepeer AI, Livepeer is focused on enhancing the quality of service for AI Orchestrator and AI Gateway node operators. The goal is to expand the range of compatible GPUs, reduce container load times, and refine the onboarding experience for developers. The Livepeer team will also be present at ETH CC with core team members speaking and participating in various events, furthering our commitment to the community and the development of decentralized video technology.

Stay tuned for more updates, and join us on this exciting journey. For more information on our initiatives and how you can get involved, visit Livepeer AI and join our Discord.

This shift has reshaped how we engage with music virtually, with live performances now immortalized and shared widely on social media platforms like Instagram, enhancing their virality. The significance of live streaming became even more apparent during global lockdowns, helping maintain the communal spirit of music worldwide despite physical separation.

As the world's open and decentralized video infrastructure network, Livepeer is committed to powering the future of innovation and creativity in global live streaming. Today, Livepeer powers pioneering DJ streaming platforms like The Lot Radio,  and the future for this community in the Livepeer ecosystem looks bright.

Refraction Joins Livepeer’s Video Disruptor Program

We are excited to announce a collaboration with Refraction in line with this mission. This partnership, initiated through Livepeer's Video Disruptor Grants program, aims to explore new frontiers in video technology and creative experiences.

Refraction is a global, artist-owned community leading the next wave of digital art, music, and culture — online, on-chain, and IRL  .. Refraction has a deep understanding of how technology is a tool of culture and content and the need for creator-focused platforms that are not just technologically sound, but also culturally driven.

As a Livepeer Video Disruptor,  Refraction is redefining the possibilities of live streaming through the prism of Web3 technologies.

Understanding Video Creator Needs

Beginning with the question - what do livestream artists want from video technology? - Refraction has spent the last few months researching and discovering the needs of creators. Through a series of focus groups, Refraction engaged directly with creators to understand their perspectives on streaming, introducing them to Livepeer's ecosystem of video tools and applications, and fostering a dialogue with our developer community on how technologies can be tailored to meet creators' needs. The insights gathered revealed a shared understanding among musicians of the necessity to adapt to the visual-centric nature of the internet. However, the technical and logistical hurdles of live streaming remain significant challenges.

The findings also highlighted a flawed model of content ownership inherent to Web2 technologies, where creators do not own the content and engagement they generate on siloed platforms. This environment spurred the interest in exploring how a more interactive and equitable live streaming platform could benefit both artists and fans. Imagine a marketplace where moments from live streams can be owned and traded as tokens or NFTs, directly supporting artists and fostering a deeper connection between creators and their audiences.

Redefining Creative Livestreaming

Refraction's research highlighted the desire of artists to easily pair their audio content with visually compelling themes designed by the Refraction community. This approach could democratize the audio-visual experience, making high-quality live streaming accessible without requiring extensive technical expertise.

Thanks to this collaboration and Livepeer’s Video Disruptor Grants program, Refraction will be able to tease a new prototype bringing the talents of visual artists, DJs and musicians together to create generative, audio-visual video content. This new tool, designed with direct input from artists during a comprehensive Discovery phase, seeks to redefine engagement and creativity in live streaming.

The prototype reflects deep insights gained from initial research with creators, focusing on their needs and the challenges they face in the digital space. By refining our technology and potentially creating new, experimental products, we aim to give artists a platform that they lead—a stark shift from traditional, centralized platforms.

The Road Ahead

As we move forward, the next steps involve rigorous testing and development with the creative community. Artists will actively use the prototype to create content, providing essential feedback that will guide further customizations. This iterative process ensures that the final product not only meets but exceeds the creator's expectations.

The culmination of this project will see the integration of this creative output into Refraction's evolving platform, showcasing a content player that displays the collective works generated through this initiative. This is more than just a technological achievement; it’s a movement towards empowering creators and engaging audiences in more meaningful ways through decentralized infrastructure.

Stay tuned to our social media channels for updates on this pioneering initiative as we continue to reimagine the future of live music and performance, merging artistry, technology, and community in a celebration of decentralized innovation.