Meta Alerts Parents When Their Teen Discusses Suicide or Self-Harm via AI Chatbot

On Thursday, Meta announced that it will inform parents if their teenager engages in discussions about suicide or self-harm with the Meta AI chatbot. The company is also working on a feature that will alert emergency services if conversations suggest a risk of self-harm.

These updates come amid growing scrutiny from regulators and parents regarding how AI chatbots manage users in crisis, especially minors—a concern that is increasingly shaping the design and marketing strategies of AI companies.

Meta asserts that it has developed a specialized AI system capable of detecting conversations where a teenager explicitly mentions self-harm.

“We understand that receiving these alerts can be unsettling for parents,” Meta stated in a blog post. “Therefore, as we enhance our detection capabilities, all conversations flagged by our AI will be manually reviewed before any alert is sent. If the teen’s intent is ambiguous, we will prioritize caution and notify the parent. Though this may result in alerts in scenarios that don’t merit serious concern, we believe this is the essential first step, and we will continuously evaluate our methods.”

Parents utilizing Instagram Parental Supervision in the U.S., U.K., Australia, and Canada can now receive these alerts, with a global rollout expected by year-end, according to Meta.

This new feature builds upon existing alerts that Meta sends when a teenager frequently searches for suicide or self-harm terms on Instagram. It also enhances a feature that allows parents to review topics their teen has engaged with on Meta AI over the past week.

Moreover, Meta revealed that its “Limited Content” setting—enabling parents to create a more restrictive environment on Instagram—now extends to Meta AI as well. The chatbot is already programmed to avoid discussions related to sexual or romantic topics and alcohol with teens. The Limited Content setting expands these protections by instructing the chatbot to decline a broader range of inquiries. Meta has not disclosed details regarding these additional prompts, but TechCrunch has requested more information from the company.

Additionally, Meta will notify emergency services if any conversation with Meta AI—regardless of whether the user is a teenager or an adult—indicates a potential suicide risk. It’s worth noting that Meta currently follows this protocol for posts on Facebook or Instagram, thereby extending this policy to interactions with its chatbot.