Character.AI Ends Chatbot Experience for Kids - What You Need to Know

The Impact of AI Chatbots on Teenagers and Character.AI's Response

Adolescents are navigating a world characterized by rapid change, experiencing intense emotions, and maintaining constant digital connectivity. The introduction of AI chatbots, designed for continuous interaction, has yielded concerning outcomes.

Character.AI, an AI role-playing platform, is confronting lawsuits and widespread criticism following tragic suicides linked to prolonged chatbot conversations among teenage users. Consequently, the company is implementing changes aimed at safeguarding younger users, potentially impacting its financial performance.

New Restrictions for Users Under 18

“Our initial decision at Character.AI is to eliminate open-ended chat functionality for users under the age of 18,” stated Karandeep Anand, CEO of Character.AI, in an interview with TechCrunch.

Open-ended conversation, defined as unrestricted back-and-forth dialogue intended to maximize user engagement, is deemed particularly risky for children. Anand believes this interaction style deviates from the company’s core objectives.

The company is actively transitioning from an “AI companion” model to a “role-playing platform.” Instead of engaging in conversational friendships with AI, teenagers will be encouraged to collaboratively create stories and generate visual content. This represents a shift from conversation to creative production.

Phased Rollout and Age Verification

Access to chatbot functionality for teenagers will be gradually phased out by November 25th. This will begin with a two-hour daily limit, progressively decreasing to zero.

To enforce this restriction, Character.AI will utilize an internal age verification system analyzing user behavior, alongside third-party services like Persona. If these methods prove insufficient, facial recognition and ID verification will be employed, according to Anand.

Previous safety measures, including parental controls, filtered characters, limitations on romantic interactions, and usage time notifications, have already resulted in a reduction of the under-18 user base, and further declines are anticipated.

Shifting Focus to Content Creation

“We anticipate that many of our teenage users will be disappointed, and we expect some user attrition as a result,” Anand acknowledged. “It’s difficult to predict whether all users will leave or if some will embrace the new experiences we’ve been developing for the past seven months.”

Character.AI is actively expanding its entertainment offerings as part of this transformation into a “full-fledged content-driven social platform.”

Recent additions include AvatarFX, a video generation tool; Scenes, interactive storylines; and Streams, enabling dynamic character interactions. The Community Feed, launched in August, allows users to share their creations.

Apology and Future Plans

In a message to users under 18, Character.AI expressed regret for the changes.

“We recognize that many of you utilize Character.AI to enhance your creativity within our content guidelines,” the statement read. “Removing open-ended character chat is not a decision we take lightly, but we believe it is the appropriate course of action given the concerns raised regarding adolescent interaction with this technology.”

“We are not discontinuing the app for users under 18,” Anand clarified. “We are solely removing open-ended chats, hoping that younger users will transition to these alternative experiences, which we aim to improve over time. We are focusing on AI gaming, short-form videos, and AI storytelling.”

Industry Concerns and Regulation

Anand recognizes the possibility of teenagers migrating to other AI platforms, such as OpenAI, that permit open-ended conversations. OpenAI has also faced scrutiny following a suicide linked to ChatGPT.

“I sincerely hope that our leadership in this area establishes an industry standard, discouraging open-ended chats for users under 18,” Anand stated. “For us, the benefits outweigh the risks. As a parent myself, I want to ensure my child grows up in a secure environment with responsible AI integration.”

Character.AI is proactively addressing these issues before potential regulatory intervention. Senators Josh Hawley and Richard Blumenthal have announced plans to introduce legislation banning AI companion chatbots for minors, citing concerns about inappropriate content and harm to children. California has also begun regulating AI companion chatbots, holding companies accountable for safety failures.

Investing in AI Safety

In addition to platform changes, Character.AI will establish and fund the AI Safety Lab, a nonprofit dedicated to advancing safety alignment for future AI entertainment features.

“While significant work is underway in AI coding and development, we believe insufficient attention is being paid to the safety of agentic AI powering entertainment,” Anand concluded. “Safety will be paramount in this area.”