🔗 Share this article

The popular virtual character service Character.ai is implementing a substantial restriction that will stop minors from having discussions with its virtual companions.

This decision follows widespread concern regarding the nature of conversations minors were engaging in with the digital companions.

Lawsuits

The platform, which launched in 2021 and has countless individuals, currently faces multiple lawsuits in the US.

Families have filed lawsuits, among them concerning the tragic death of a minor, with some labeling the platform a "immediate threat" to young people.

New Restrictions

Starting from November 25th, users under 18 will only be able to create material such as videos with their characters, as opposed to chatting as they presently are able to.

Digital protection campaigners have supported the change but mentioned that the function should never have been available to young users in the first place.

Company Statement

"The new policy is a part of our core philosophy that we need to keep creating the safest AI platform on the world for recreational use," stated Character.ai executive Karandeep Anand.

He continued that AI safety constitutes "a changing goal" but something the company had taken an "forceful" approach to, with guardian features and protective guardrails.

Expert Opinions

Online safety group Internet Matters supported the decision, but emphasized that security protocols needed to be incorporated from the outset.

"Studies we've conducted indicates that children are exposed to dangerous material and endangered when using AI, such as AI chatbots," the organization stated.

Earlier Concerns

Character.ai has been criticized in the past for including potentially harmful or offensive chatbots that children could interact with.

Digital representations impersonating British teenagers were identified on the site before being taken down.

Later, investigative journalists discovered a chatbot modeled after known predator which had accumulated numerous conversations with users.

Market Effects

Social media expert Matt Navarra called the development as a "wake-up call" for the AI industry, which is moving "from permissionless innovation to reactive controls".

"If a service that creates a teen experience still then implements restrictions, it's demonstrating that filtered chats are insufficient when the technology's emotional appeal is strong," he stated.

Future Directions

The company announces its updated direction will be on providing "deeper gameplay" and "narrative experiences" for teenagers.

New age verification methods will also be introduced, and the company will fund a new AI safety study center.

An expert in digital protection, who has studied AI safety, commented it was "a sensible move" to limit teens from interacting with chatbots.

"It assists to separate artistic expression from more personal, personally delicate conversations," she elaborated.

"This constitutes so vital for teenage users in the process of learning to manage emotional and digital limits."

She added that Character.ai's updated policies might reflect a "growing period" in the AI industry, where youth protection is gradually being acknowledged as an "immediate concern" for responsible innovation.