4227ccdafaafb86b3c1c6b1ab81d0e102352717425befdda1e88bd1bfbb1d05c
55c15c857c2ca0ccd466d6ae5b77d4e289ff5f682c6b1b71d23e572734031f2c
637c5a77a0292accbed02c82ba14355f4ae40b1a84e5a5a229c27d8f90316556
122a13e55b3ba54e5816fe866201becba30259e0
outline of the general idea: members can contribute media into a shared repository/library. new additions to this shared library are periodically (could be once an hour/day/week depending on rate of contribution) converted into embeddings, with all of the embeddings periodically reindexed. people (i.e., members and/or non-members) could use these embeddings for RAG-based (retrieval-augmented generation) chatbots. these chatbots could be accessed within the Discord server in bot-form, through an embedded chatbox on the Trust website, etc.
notes on near-term practicality: see trellis (https://beta.readtrellis.com/) for a current example of a consumer-friendly “upload pdf” ⇒ “create embeddings” ⇒ “chat with document library” pipeline. algovera (https://app.algovera.ai/) is currently marketing beta access for organization-level LLM solutions. we’ll soon have easy-to-use solutions for organizations for creating LLMs based on shared libraries. even without the inevitable productization of such solutions we can somewhat easily conceive of some scrappy DIY solutions that make use of shared Google Drive folders and serverless CRON jobs with AWS Sagemaker.
nitty-gritty
file formats: for now, just text-based media in .pdf, .txt, and .md format; while .mp3/.mp4-based embedding pipelines exist, it might be better to start out with simpler file formats.
permissioning: how can it be ensured that the method by which verified (i.e., formally “subscribed”) members add entries to be embedded (e.g., a .pdf file of a book) is exclusive to members? currently it’s possible to set up a pipeline whereby a list of e-mail addresses could be given permission to a Notion workspace or Google Drive workspace whereby members could add files to be periodically processed by an embedding pipeline. however such a solution needs to deal with issues of anonymity (members might not want their e-mails being public, which they are in both Notion and Drive workspaces) as well as granular permissioning wrt the ability to edit/delete entries
metering usage: how should quotas be set on how many entries individuals can contribute? is it a problem if a single individual contributes 100 pdfs? 1,000 pdfs? how should calling/inferencing the resultant chatbot be metered/limited?
conclusion: i’d argue that setting up a shared media library with appropriate permissioning would be valuable even without the embedding pipeline aspect. provisionally, this whole process could even be done manually (e.g., people post pdfs into a Discord channel and a designated person does the uploading/embedding step at the end of the week) until a solutions provider like Algovera releases a user-friendly platform.
some ways how this concept can be extended:
the embeddings can be used for network cluster visualizations
we can employ the “community LLM” as a modular input into something like a generative Unity-based model/character/scene that evolves as the community’s collective library evolves
members can contribute their own (potentially anonymized) self-written entries as an input into the community model
a community image repo; images can be clustered by visual similarity as well as through a “caption each image with a multimodal model” ⇒ “algorithmically cluster images based on semantic descriptions of images” pipeline