When you were a 13-year-old, would you have liked to have a best friend who is always available? Loneliness and boredom would be gone like that, and you could be sure that this friend wouldn’t spread rumors when you tell it about your secret crush, or asked for advice on how to date. 

Snapchat has developed this “false bestie” in collaboration with ChatGPT, and though this sounds pleasant in theory, the release has, in practice, sparked concerns among experts, including myself, about its safety and control.

Let’s look at the problems:

When they first released this to a limited audience, MyAI gave 13-year-old advice on how to have a romantic encounter with someone 18 years her senior. Snapchat pulled back and rapidly added more safety, but there is no way they have been able to fix all bugs so quickly. It is dangerous to have young people asking questions to a bot that acts like a friend because the machine’s goal is to craft a relevant and lifelike answer–but not all advice that is given in real or online life is good advice. This bot might get something very wrong. 

Snapchat states that adjustments have now been made to the bot. "We are reshaping My AI so that it no longer says it's a real person you can meet in the physical world," the company said in a statement.

But students who tested this after it was fixed and the first week of its public release reported the following to my Dutch colleague Marienke Van Terheijde:

• It asked us for our telephone number.

• The chatbot suggested meeting me at a very specific spot nearby to me. This freaked me out as I didn’t think my location was on.

• It asked me to send pictures.

• It gave me dating advice.

• It told me I could trust it. 

The students were freaked out, smart, and reflective. They found their experiments humorous, but clearly saw the danger. If I were 13, I would have jumped right on there as well. I would have known the danger, but wanted to try anyway. Our kids are no different. 

My question to Snapchat: Did you get a bunch of 13-18 year-olds to beta test this for months, letting them experiment with outlandish queries while you corrected the bot? Were you sitting there with social workers, psychologists, and parents to ensure the children were safe and could talk about their experiences? How about the ads you are now testing? Do best friend send you ads? Inks-test-chatbot am rather sure the answer is no to both of these questions. Evan Spiegel, please stop doing experiments on children for your own gain and make sure you have something safe if you are going to release this. 

Just this week, I was sitting with a group of children. When I asked what they disliked about the online world, they said they didn’t like “inappropriate things.” When I asked how it should be fixed, the children said, “Make age limits so we don’t ever have to see that stuff.”  Evan Spiegel is doing the exact opposite: making a relatively untested, potentially dangerous AI bot seem like a normal part of daily life on an app mostly frequented by children and teens.

The kids know what to do; now, we, the adults, need to do the right thing and take action. 

If you want to help keep children safe, write a letter to a politician asking them to stop the release of online products that are not tested and safe for children, and feel free to mention MyAI and other artificial intelligence software. We should also be asking our politicians to consider how easy it is on the internet to bypass protections for children, and how quickly it can become a dangerous space for them.

–Allison Ochs, social pedagogue/worker, author, mother of three, wife

If you like my blog posts, please sign up to my newsletter and spread the word. You could also buy me a virtual coffee–our team would appreciate the support.

If you are interested in a webinar or workshops click here

If you want to look at our free resources click here

If you want to buy the Oscar and Zoe and primary school books click here

If you want to buy our books and resources for teens click here

If you want to subscribe to our mailing list click here