>3.7K subscribers
Share Dialog
Share Dialog
Today, we are excited to announce the launch of a BigQuery public dataset for Lens 🌿. With BigQuery, developers can quickly query large amounts of Lens data using SQL, unlocking new possibilities for what can be built on Lens Protocol.
Lens provides developers with an easy-to-use GraphQL API for querying Lens data quickly. This enables querying at traditional web2 speeds without a complex infrastructure setup and our dev community has built some amazing products in less than 300 days on Polygon mainnet. However, the API has its limits: it is difficult to access the entire dataset, which is necessary for data-intensive tasks such as training machine learning models or performing comprehensive data analyses.
Devs love Lens data. Over the past months, we have received many requests to make it easier to access Lens data in bulk for use cases such as training ML algorithms or performing data analytics. We listened. To empower builders, we have published the whole Lens database to a BigQuery public dataset 🔍
BigQuery public dataset brings many benefits. Here are a few of the ones we think are the most important:
Lens data is now available in bulk. Instead of spending days pulling data from the API, developers can get all the data they want in seconds with a SQL query. Previously, grabbing all the required data was a time-consuming task. Due to rate limits on the API set up to prevent DDOS attacks, it could take 1-2 days to retrieve every bit of data. With BigQuery, you can retrieve millions of rows in seconds. This greatly increases the speed at which developers can build features and ship ML and other code.
Indexing data from the chain can be difficult and most developers do not want to take on this task. Instead, they want to build incredible Lens apps. With the Lens BigQuery public dataset, developers can focus on the features they are passionate about developing. For example, ML engineers can concentrate on writing incredible algorithms, while data analysts can simply write SQL queries for generating fascinating insights. This allows people to focus on the points that matter most to them.
The variety of apps that can be built today are limited by the capabilities of the API. Despite these constraints, our community has already developed over 100 apps across various categories. With the release of this dataset, developers can now explore a more diverse universe of possibilities in the apps they build. We’re excited to see what developers can build now that they have access to the full Lens dataset.
We can’t wait to see what our developer community can build with access to the Lens BigQuery public dataset.
Creator analytics. Creators want to know which of their posts are performing best. They want to know who their audience is. Build a tool that lets creators better understand where they stand on Lens.
Better algorithms. There’s a lot of content on Lens, and not all of it is easy to find right now. Create a way of connecting Lens users with the content they want to see. Imagine Moderation, Engagement, and Discovery algorithms available in an open marketplace.
Bot detection. Being permissionless is a double-edged sword. If anyone can post freely, bots are soon to come. Implement an algorithm that detects bots based on their behavior, their position in the social graph, and whatever other features you can think up.
Trending topics. What are people talking about on Lens right now? What’s the latest news? What’s the hot issue? Build a tool that surfaces what topics seem to be especially prevalent right now.
Search. No one can possibly read everything that gets posted on Lens. Help users find the content they’re especially interested in by building an amazing search capability that goes beyond just matching strings. Bonus: how about an image search capability?
Lens Protocol is an open-source decentralized social media protocol, which is a public good. We encourage the Lens community to build transparent and open algorithms that anyone can contribute to. Together, we are building the social layer for web3. With the Lens BigQuery public dataset, we hope developers will have increased access to innovation with Lens. We believe that open datasets have the power to make the internet more accessible, and with updates like today’s, we are boldly going where no social media platform has gone before. And we’re just getting started.
Today, we are excited to announce the launch of a BigQuery public dataset for Lens 🌿. With BigQuery, developers can quickly query large amounts of Lens data using SQL, unlocking new possibilities for what can be built on Lens Protocol.
Lens provides developers with an easy-to-use GraphQL API for querying Lens data quickly. This enables querying at traditional web2 speeds without a complex infrastructure setup and our dev community has built some amazing products in less than 300 days on Polygon mainnet. However, the API has its limits: it is difficult to access the entire dataset, which is necessary for data-intensive tasks such as training machine learning models or performing comprehensive data analyses.
Devs love Lens data. Over the past months, we have received many requests to make it easier to access Lens data in bulk for use cases such as training ML algorithms or performing data analytics. We listened. To empower builders, we have published the whole Lens database to a BigQuery public dataset 🔍
BigQuery public dataset brings many benefits. Here are a few of the ones we think are the most important:
Lens data is now available in bulk. Instead of spending days pulling data from the API, developers can get all the data they want in seconds with a SQL query. Previously, grabbing all the required data was a time-consuming task. Due to rate limits on the API set up to prevent DDOS attacks, it could take 1-2 days to retrieve every bit of data. With BigQuery, you can retrieve millions of rows in seconds. This greatly increases the speed at which developers can build features and ship ML and other code.
Indexing data from the chain can be difficult and most developers do not want to take on this task. Instead, they want to build incredible Lens apps. With the Lens BigQuery public dataset, developers can focus on the features they are passionate about developing. For example, ML engineers can concentrate on writing incredible algorithms, while data analysts can simply write SQL queries for generating fascinating insights. This allows people to focus on the points that matter most to them.
The variety of apps that can be built today are limited by the capabilities of the API. Despite these constraints, our community has already developed over 100 apps across various categories. With the release of this dataset, developers can now explore a more diverse universe of possibilities in the apps they build. We’re excited to see what developers can build now that they have access to the full Lens dataset.
We can’t wait to see what our developer community can build with access to the Lens BigQuery public dataset.
Creator analytics. Creators want to know which of their posts are performing best. They want to know who their audience is. Build a tool that lets creators better understand where they stand on Lens.
Better algorithms. There’s a lot of content on Lens, and not all of it is easy to find right now. Create a way of connecting Lens users with the content they want to see. Imagine Moderation, Engagement, and Discovery algorithms available in an open marketplace.
Bot detection. Being permissionless is a double-edged sword. If anyone can post freely, bots are soon to come. Implement an algorithm that detects bots based on their behavior, their position in the social graph, and whatever other features you can think up.
Trending topics. What are people talking about on Lens right now? What’s the latest news? What’s the hot issue? Build a tool that surfaces what topics seem to be especially prevalent right now.
Search. No one can possibly read everything that gets posted on Lens. Help users find the content they’re especially interested in by building an amazing search capability that goes beyond just matching strings. Bonus: how about an image search capability?
Lens Protocol is an open-source decentralized social media protocol, which is a public good. We encourage the Lens community to build transparent and open algorithms that anyone can contribute to. Together, we are building the social layer for web3. With the Lens BigQuery public dataset, we hope developers will have increased access to innovation with Lens. We believe that open datasets have the power to make the internet more accessible, and with updates like today’s, we are boldly going where no social media platform has gone before. And we’re just getting started.
No comments yet