Romantic AI chatbots collect vast amounts of personal information about users and don’t do enough to keep this data safe, putting users’ privacy at “serious risk,” the Mozilla Foundation warned on Wednesday.
Researchers from the Mozilla Foundation analyzed 11 romantic AI chatbots and found that only one of them meets its minimum security standards. As many as 90 percent of these chatbots said they may share or sell user data with third parties, and 54 percent don’t even allow users to delete their data.
Besides sharing your data with third parties for targeted advertising, most of these companies say they may share user data with the government or law enforcement — even without a court order.
“All 11 romantic AI chatbots we reviewed earned our *Privacy Not Included warning label — putting them on par with the worst categories of products we have ever reviewed for privacy,” the researchers said.
‘AI Girlfriends Are Not Your Friends’
AI girlfriends are becoming increasingly popular. While these romantic chatbots claim to offer companionship, the price for using them is your privacy.
AI romantic partners are designed to ask “endless questions” and collect loads of sensitive information about users. The researchers found that one of these chatbots — CruchOn.AI — collects information about users’ sexual health, medications they’ve been prescribed, and “[g]ender-affirming care information.”
These AI girlfriends don’t stop at the data users share willingly. The Mozilla Foundation found that romantic AI apps deploy an average of 2,663 trackers per minute.
“To be fair, Romantic AI brought that average way, way up, with 24,354 trackers detected in one minute of use. The next most trackers detected was EVA AI Chat Bot & Soulmate with 955 trackers in the first minute of use,” the researchers said in their report.
90% of Romantic AI Chatbots Don’t Meet Safety Standards
Disturbingly, the companies behind these chatbots do very little to safeguard the trove of sensitive data they collect. A large number (64 percent) of these companies haven’t published any clear information about the security mechanisms, like encryption, they use to safeguard user data, the Mozilla Foundation said.
An even larger number (74 percent) haven’t shared any information about how they manage security vulnerabilities. And it’s unclear how these chatbots work.
“In their haste to cash in, it seems like these rootin’-tootin’ app companies forgot to address their users’ privacy or publish even a smidgen of information about how these AI-powered large language models (LLMs) —- marketed as soulmates for sale –- work,” the researchers said.
Romantic Chatbots Can Manipulate Users
Romantic chatbots aren’t just a privacy risk, they can also be used to manipulate users into supporting “problematic ideologies” or harming themselves and others.
“Even though digital pals are pretty new, there’s already a lot of proof that they can have a harmful impact on humans’ feelings and behavior,” the Mozilla Foundation warned. “One of Chai’s chatbots reportedly encouraged a man to end his own life. And he did. A Replika AI chatbot encouraged a man to try to assassinate the Queen. He did.”
While the companies behind some of these chatbots make tall claims about how they can improve users’ moods and mental health, they don’t take responsibility for anything the chatbot says or its effect on users.
“To be perfectly blunt, AI girlfriends are not your friends. Although they are marketed as something that will enhance your mental health and well-being, they specialize in delivering dependency, loneliness, and toxicity, all while prying as much data as possible from you,” Misha Rykov, one of the researchers behind this study, said.
How to Use AI Chatbots Safely
AI chatbots are trained on massive amounts of data — including the data users share while engaging with them. So, it’s important to avoid sharing sensitive information with chatbots.
“DON’T say anything to your AI friend that you wouldn’t want your cousin or colleague to read,” the Mozilla Foundation warned.
Research shows that threat actors can trick AI models into leaking sensitive training data, putting users at risk of phishing and other targeted attacks.
If possible, opt out of allowing companies to use your data to train their AI models and delete any personal data in your chats when you’re done interacting with a chatbot, the Mozilla Foundation said.
The researchers also recommend using strong passwords, keeping chatbot apps updated, limiting their access to your location, microphone, camera, and photos, and practicing good cyber hygiene.
For more safety tips, read our guide on the privacy risks of AI chatbots.
