Google extracted ChatGPT’s Training Data using a silly trick
Executive Highlights-In their publication, Scalable Extraction of Training Data from (Production) Language Models, Deepmind researchers were able to extract several megabytes of ChatGPT’s training data for about two hundred dollars. They estimate that it would be possible to extract ~a gigabyte of ChatGPT’s training dataset from the model by spending more money querying the model. Their, “new divergence attack that causes the model to diverge from its chatbot-style generations and emit traini...
Google extracted ChatGPT’s Training Data using a silly trick
Executive Highlights-In their publication, Scalable Extraction of Training Data from (Production) Language Models, Deepmind researchers were able to extract several megabytes of ChatGPT’s training data for about two hundred dollars. They estimate that it would be possible to extract ~a gigabyte of ChatGPT’s training dataset from the model by spending more money querying the model. Their, “new divergence attack that causes the model to diverge from its chatbot-style generations and emit traini...
Temporal Graph Learning in 2024
Many complex networks evolve over time including transaction networks, traffic networks, social networks and more. Temporal Graph Learning (TGL) is a fast growing field which aims to learn, predict and understand evolving networks. See our previous blog post for an introduction to temporal graph learning and a summary of advancements last year. In 2023, we saw significantly increased interest from both academia and the industry in the development of TGL. Compared to last year, the number of s...
Temporal Graph Learning in 2024
Many complex networks evolve over time including transaction networks, traffic networks, social networks and more. Temporal Graph Learning (TGL) is a fast growing field which aims to learn, predict and understand evolving networks. See our previous blog post for an introduction to temporal graph learning and a summary of advancements last year. In 2023, we saw significantly increased interest from both academia and the industry in the development of TGL. Compared to last year, the number of s...
ADMET-AI: A machine learning ADMET prediction platform. Article review
With the rapidly increasing amounts of chemical data, it is still a huge problem to characterize it and be able to predict the properties of compounds accurately. Currently, this is still a challenge due to the lack of open-source data and tools, so it is super exciting to see great projects that attempt to solve this problem. I recently discovered a platform called ADMET-AI developed by Greenstone Biosciences and Stanford University. It not only has a web interface but can be used locally as...
ADMET-AI: A machine learning ADMET prediction platform. Article review
With the rapidly increasing amounts of chemical data, it is still a huge problem to characterize it and be able to predict the properties of compounds accurately. Currently, this is still a challenge due to the lack of open-source data and tools, so it is super exciting to see great projects that attempt to solve this problem. I recently discovered a platform called ADMET-AI developed by Greenstone Biosciences and Stanford University. It not only has a web interface but can be used locally as...
How to Write Memory-Efficient Classes in Python
A few years ago, I wrote a blog post on how to write memory-efficient loops in Python which became quite popular. The positive response encouraged me to write a second part, where I delve into additional memory optimization methods. When writing python code, loops are not the only place where we need to be mindful of memory usage. In data-related projects and object-oriented code development, it is important to make sure that our classes are also memory efficient. Often, we invest a significa...
How to Write Memory-Efficient Classes in Python
A few years ago, I wrote a blog post on how to write memory-efficient loops in Python which became quite popular. The positive response encouraged me to write a second part, where I delve into additional memory optimization methods. When writing python code, loops are not the only place where we need to be mindful of memory usage. In data-related projects and object-oriented code development, it is important to make sure that our classes are also memory efficient. Often, we invest a significa...
Towards Hybrid Reasoning: Assimilating Structure into Subsymbolic Systems
The recent advances in large language models (LLMs) have demonstrated their remarkable fluency and adaptability when generating text. After exposure to just a few examples, these models can produce surprisingly coherent continuations on a wide array of topics, exhibiting signs of flexible understanding and reasoning. However, further analysis reveals fundamental gaps that temper unbridled optimism. At their core, LLMs accumulate only statistical patterns reflecting correlations of terms in th...
Towards Hybrid Reasoning: Assimilating Structure into Subsymbolic Systems
The recent advances in large language models (LLMs) have demonstrated their remarkable fluency and adaptability when generating text. After exposure to just a few examples, these models can produce surprisingly coherent continuations on a wide array of topics, exhibiting signs of flexible understanding and reasoning. However, further analysis reveals fundamental gaps that temper unbridled optimism. At their core, LLMs accumulate only statistical patterns reflecting correlations of terms in th...
