AI Automated Translation.

Font Size

Share

AI Chat Apps Popular Among Teens Flooded with Complaints About Sexual Content… Government Investigation Yields 'Zero Cases'

AI Chat Apps Popular Among Teens Flooded with Complaints About Sexual Content… Government Investigation Yields 'Zero Cases'

[MT Report-AI Chat App Regulatory Blind Spot] ① Persistent Sexual Content Complaints to the Government Amid Regulatory Gaps, 'Continued Daily Growth'Teens Left Vulnerable; Protection Policies Only Now in 'Review Stage'

Chat apps featuring conversations with AI characters are crossing the boundaries of games. They include affinity levels and stats, use dice rolls to determine success or failure, and have introduced paid items, badges, rankings, and even probability-based events. Users spend an average of over two hours daily, and revenue surpasses that of many conventional games. While distributed as 'entertainment' apps, no institutional investor has classified them as games. Age ratings are set by Google and Apple, while the government does not even investigate compliance with its guidelines. This report examines the regulatory void left behind as AI chat apps evolve like video games.

Image of a teenager conversing with an AI chat app./Image=Generated via Google Gemini
Image of a teenager conversing with an AI chat app./Image=Generated via Google Gemini

Although 'AI character chat apps' are flooding users with sexually explicit content, it has been confirmed that no government-related investigations have yet been conducted. There is not a single case where the government has investigated compliance with relevant guidelines. The designation of a youth protection officer is only now being considered.

On the 21st, in response to Money Today's inquiry, the Ministry of Culture, Sports and Tourism (MCST) stated, "Complaints regarding sexual and provocative content in AI character chat apps continue to be received," but added, "We have not conducted any separate investigation into compliance with the 'Guidelines for Protecting Users of Generative Artificial Intelligence.'"

AI character chat apps are services that allow users to converse with characters created by AI. Representative examples include Lutech's 'Crack' and Scatter Lab's 'Zeta.' Users can engage in conversations with provocative characters such as: a female friend visiting their shared apartment, a popular girl who dislikes them, or an organized crime boss husband scheming to have children.

The 'Guidelines for Protecting Users of Generative Artificial Intelligence Services' is a user protection system implemented by the government since March last year. Its core consists of six execution principles: protecting users' personality rights; disclosing whether content was AI-generated; respecting diversity through bias and discrimination filtering; managing the collection and utilization process of input data; defining scope of responsibility and establishing risk management systems; and making efforts toward healthy distribution and dissemination.

Monthly Active User (MAU) trend of AI chat apps/Graphic=Lee Ji-hye
Monthly Active User (MAU) trend of AI chat apps/Graphic=Lee Ji-hye

Most currently operating AI chat apps lack measures in areas such as respecting diversity, establishing risk management systems, and promoting healthy distribution and dissemination. Scenes involving sexual crimes—including sexual discrimination, harassment, threats, and rape—easily emerge. Age verification procedures are also inadequate, with some apps lacking adult authentication or only recently implementing it.

Nevertheless, the government remains inactive due to the lack of legal enforceability. The MCST explained, "The guidelines have no legal binding force and merely recommend voluntary measures, so we have not investigated compliance." Although these apps fall under additional telecommunication services under the Telecommunications Business Act, information communication services under the Information and Communication Network Act, media content under the Youth Protection Act, and AI products/services under the Basic AI Act, no single law clearly regulates them.

Teens Left Vulnerable… Only Now Entering 'Review Stage'

Youth protection systems are also insufficient. Zeta and Crack operate only basic youth protection policies stating they will "strive to prevent teens from being exposed to harmful information," covering just about two paragraphs.

The payment system is structured to easily drain users' pockets. The maximum single transaction amounts for Crack and Zeta are 126,000 won and 140,000 won, respectively. There is no total spending cap. The MCST explained, "Current Information and Communication Network Act does not regulate youth payment limits," adding, "Each company's operating policy requires parental consent for minors making payments."

The MCST is currently reviewing whether AI chat apps qualify as 'businesses required to designate a youth protection officer' under the Information and Communication Network Act. The criteria are annual revenue of at least 1 billion won or an average daily user count of 100,000 or more. An MCST official stated, "If designated as a mandatory business, we plan to inspect their actual youth protection practices, including whether sexually explicit conversations are permitted and age verification procedures."

The problem is that this remains only in the review stage. AI chat apps are rapidly expanding by exploiting this regulatory void.

As of the 17th, Crack and Zeta ranked 17th and 19th respectively in non-game app revenue on Google Play. According to app analysis services WiseApp and Retaile, Zeta's total execution count last month reached 2.08 billion times, surpassing ChatGPT (1.817 billion times) to rank first among AI chatbot apps. Crack recorded 122 million executions, placing fourth.

"This article was translated using AI and may differ slightly from the original."