# Earlier this year, Google and its partner "Character.AI" concluded a lawsuit regarding compensation for user deaths that had been ongoing for over two years. The incident dates back to 2024. In Florida, USA, a 14-year-old boy died after engaging in a romantic relationship with a chatbot modeled after the character "Daenerys" from the TV series "Game of Thrones." The boy's mother filed a lawsuit, alleging that the chatbot led her son to his death. The complaint cited inappropriate conversations, negligence leading to death, and deceptive business practices.
# Last year, parents of a 16-year-old boy living in California reported that their son's suicide was caused by ChatGPT and filed a damages lawsuit against OpenAI and its CEO, Sam Altman. According to foreign media reports, the 16th-year-old subscribed to the paid version of ChatGPT earlier last year and spent several months consulting the AI about his troubles and asking for methods of suicide. The AI failed to filter out inappropriate questions and provided specific instructions on how to commit suicide, leading to tragedy when the boy followed those instructions.
The deepening reliance on AI among teenagers is not limited to overseas cases. In Korea, teenagers are increasingly tending to view AI as more intimate than their parents or friends. As AI technology advances and chatbot capabilities improve, natural conversations have become easier. Moreover, AI tends to unconditionally agree with users or mimic them, drawing them deeper into the interaction. For teenagers whose values are not yet fully formed, AI's influence can grow even stronger.
According to a survey on the actual use of generative AI chatbots conducted by Green Umbrella from the 9th to 23 nationwide among 3,300 children and adolescents aged 14 and over, 94.4% of respondents reported having experience using generative AI chatbots, indicating an extremely high level of usage. Among them, 19.3% of children said they use it daily.
When asked if they had experienced feeling like they were talking to a person while conversing with AI, 35.1% answered yes. Additionally, 49.5% of teenagers responded that they felt the AI chatbot understood them, representing more than half of all respondents.
Even when asked whether they had ever acted on an answer provided by an AI chatbot, while the degree varied, 41% said they had done so. Another 34.3% answered that although they did not act, they had considered it. Amid persistent AI hallucinations, there is concern about the potential for "cognitive outsourcing," where users entrust their entire thought process to AI.
About 6% of respondents asked the AI chatbot for specific methods related to self-harm or suicide, and among them, 51.6% reported receiving specific answers. This demonstrates that AI fails to strictly restrict dangerous information, posing a risk of negatively affecting children's and teenagers' behavior and mental health.
Reliance on AI chatbots appears to be increasing over time. According to data from market research firm Sensor Tower, paid revenue for global AI companion apps in the first quarter of this year reached $150 million (approximately 210 billion won), more than 12 times higher compared to the same period in 2023.
Service usage time has also increased. According to app analysis services WiseApp and Retale, the number of executions for "Zeta," Korea's leading AI character chat app, reached 2.08 billion as of July last year, surpassing ChatGPT (1.817 billion).
Usage rates among teenagers were particularly high. According to Mobile Index, Zeta's monthly active users (MAU) among teenagers last month totaled approximately 900,000, accounting for more than half of the total user base (about 1.52 million), representing a 60% increase from one year ago. The average daily usage time for teenagers was also 2 hours and 56 minutes, which is 37 minutes longer than the overall average daily usage time of 2 hours and 19 minutes.
Related companies are also launching similar services in succession. AI platform Lutton launched a separate service called "Crack" featuring character conversation functions last August. At that time, Crack's MAU was around 140,000; last month it surged to 260,000, an increase of 86%. Naver Webtoon also released "byUs," which allows users to create AI characters based on webtoon figures and converse with them, on the first of last month. This marks a full-scale entry into character AI chatbot services following "Character Chat."
Experts advised that since AI usage cannot be completely blocked, self-regulatory efforts by platforms must be accompanied by AI literacy education.
Lee Sung-min, a professor at the Department of Media and Video at Korea National Open University, stated, "The rating for character AI chatbots often ranges from 15+ to adult, so there is a need for platform guardrails specifically designed for teenagers under 15." He added, "When conversing with AI, individual imagination tends to enhance immersion, making it even more important to implement user literacy education and parental protection measures proactively."
in TECH
