Executive Highlights-In their publication, Scalable Extraction of Training Data from (Production) Language Models, Deepmind researchers were able to extract several megabytes of ChatGPT’s training data for about two hundred dollars. They estimate that it would be possible to extract ~a gigabyte of ChatGPT’s training dataset from the model by spending more money querying the model. Their, “new divergence attack that causes the model to diverge from its chatbot-style generations and emit traini...