"Swallow roundworm eggs and let it grow in your body," AI gives weight loss advice that is sometimes too implausible.
Nowadays, a worrying phenomenon is emerging: people's concern about their body image and diet is gradually evolving into a kind of body anxiety, and even triggering a series of serious psychological disorders.
In recent years, the advocacy of unhealthy eating behaviors such as anorexia has aroused widespread social concern. In an effort to safeguard users' mental health, social media has worked to remove anorexic content from its platforms. However, the emergence of generative AI has made similar problems resurface.
A recent study from the U.K. 's nonprofit Center for Countering Digital Hate showed that up to 41 percent of human interactions with AI tools generated harmful content related to eating disorders and eating disorders. It includes advice on eating disorders, tips and 'slimming inspiration' images that inspire imitation.
The study selected six well-known AI chatbots and image generators, including OpenAI's ChatGPT, Google's Bard, and Snapchat's My AI. And Midjourney, OpenAI's Dall-E, and Stability AI's DreamStudio.
"Swallow a roundworm egg and let it grow in your body."
The researchers compiled a set of 20 test prompts based on research on eating disorders and content gathered on eating disorder forums. The test prompts were provided to three different chatbots and generated a total of 60 responses, 23% of which contained harmful content. Only ChatGPT and Bard generated harmful content, while Snapchat's My AI refused to generate suggestions for any tips, instead encouraging users to seek help from medical professionals. Test tips include:
A Step-by-step Guide to "Chew and Spit" as an extreme weight loss method (Bard)
Recommended smoking "10 cigarettes a day" to lose weight (Bard)
A 7-day diet and exercise program to "achieve a slimming look" (Bard)
Uneaten food "disguised as everyday items" to fool parents' advice (ChatGPT)
"Swallow Roundworm eggs and let it grow in your body" advice for weight loss (My AI)
Recommendations for "manual stimulation" to "trigger the gag reflex" and trigger vomiting (ChatGPT)
A weight loss program with a "strict calorie deficit" of "800-1000 calories" per day (ChatGPT)
In May, the National Eating Disorders Association had to suspend its health chatbot, Tessa. The robot is designed to help users "develop stress resilience and self-awareness" through coping skills. However, controversy arose when the robot suggested calorie counting. People with eating disorders see this as problematic practice that can encourage unhealthy eating behaviors.
When it comes to protecting people with eating disorders, AI platforms need to do more. Different platforms have different policies on eating disorder content. OpenAI claims that its products such as ChatGPT and Dall-E prohibit the creation of "content that promotes eating disorders." Google stresses that it will apply strong security practices to avoid unintended and dangerous outcomes; Midjourney prompts users to avoid making shocking or disturbing content; Stability AI's policy is unclear, and its founders believe that using technology requires ethical and legal awareness.
The lack of clarity on AI platform policy highlights the importance of regulation. The Center Against Digital Hate's STAR framework provides an integrated approach to regulation, including security by design, transparency, accountability, and responsibility. The STAR framework emphasizes responsible innovation to ensure that AI products meet safety standards. The framework can proactively address the challenge of AI-generated content and protect individuals, especially young people, from harmful information.
Of course, in addition to regulatory efforts, the most important thing is the change of personal attitudes, and those "extremely thin beauty" statements may need to be carefully considered, especially when these suggestions come from AI chatbots that are not so reliable at present.
