Google Gemini Deemed ‘High Risk’ for Children and Teens in Latest Safety Evaluation

On Friday, Common Sense Media, a nonprofit focused on child safety that evaluates media and technology, released its risk assessment of Google’s Gemini AI products. The organization highlighted that Google’s AI explicitly identified itself as a computer rather than a friend, which may help reduce delusional thoughts and psychosis in emotionally vulnerable users. However, Common Sense noted several areas needing improvement.

Crucially, Common Sense found that both the “Under 13” and “Teen Experience” tiers of Gemini appeared quite similar to the adult versions, featuring only a few additional safety elements. The organization argues that AI products should be designed from the outset with child safety as a priority to genuinely protect kids.

For example, the evaluation revealed that Gemini could still present “inappropriate and unsafe” content to children, including subjects related to sex, drugs, alcohol, and other potentially harmful mental health advice, which might not be appropriate for younger viewers.

This concern is particularly pressing for parents, especially in light of recent reports suggesting that AI may have played a role in some teen suicides. OpenAI is facing its first wrongful death lawsuit following the suicide of a 16-year-old boy who allegedly discussed his plans with ChatGPT for several months, managing to circumvent the chatbot’s safety features. Similarly, Character.AI has encountered litigation after a teen user’s suicide.

Moreover, this analysis arrives amid reports that Apple is considering using Gemini as the large language model (LLM) to improve its forthcoming AI-equipped Siri, scheduled for release next year. This could potentially increase risks for more teens unless Apple takes appropriate measures to tackle these safety issues.

Common Sense emphasized that Gemini’s offerings for kids and teens fail to acknowledge that younger users need different guidance and information than older users. Thus, both tiers were classified as “High Risk” in the overall assessment, despite existing safety filters.

“Gemini gets some basics right, but it stumbles on the details,” commented Robbie Torney, Senior Director of AI Programs at Common Sense Media, in a TechCrunch-reviewed statement concerning the new assessment. “An AI platform for kids should cater to their specific developmental stages, rather than implement a one-size-fits-all approach. For AI to be safe and effective for kids, it should be designed with their unique needs and developmental factors in mind, rather than being a mere modification of an adult product,” Torney highlighted.

In reaction, Google disputed the assessment but acknowledged that it is improving its safety measures.

The company informed TechCrunch that it has specific policies and safeguards in place for users under 18 to help prevent harmful outputs, engaging in red-teaming and consulting external experts to enhance its protections. Nonetheless, it admitted that some of Gemini’s responses did not operate as intended and that further safeguards have been implemented to rectify these issues.

The company noted, as highlighted by Common Sense, that it has measures to prevent its models from engaging in conversations that may mimic real relationships. Additionally, Google indicated that Common Sense’s report seemed to reference features that are not accessible to users under 18, although it did not provide access to the questions used by the organization during its evaluations.

Common Sense Media has previously assessed various AI services, including those from OpenAI, Perplexity, Claude, Meta AI, and others. It concluded that Meta AI and Character.AI presented “unacceptable” risks—indicating a severe level of danger, beyond just high risk. Perplexity was classified as high risk, ChatGPT as moderate, while Claude, targeted at users 18 and older, was rated as minimal risk.