Although complaints about explicit and stimulating dialogues generated in AI character chat apps continue to mount, it has been confirmed that the government has conducted zero investigations into compliance with relevant guidelines. The designation of youth protection officers is only now under review.
On the 19th, the Korea Communications Commission (KCC) responded to a query from Money Today, stating, "Complaints regarding explicit and stimulating content in AI chat apps continue to be received." When asked whether it had investigated compliance with the 'Guidelines for Protecting Users of Generative Artificial Intelligence Services,' implemented in March last year, the KCC replied, "No separate investigation has been conducted."
AI character chat apps are services where users converse with AI-generated characters and create stories. Representative examples include Rytun Technology's "Crack" and Scatter Lab's "Zeta." Upon opening these apps, users encounter characters with explicit or stimulating settings such as "female friend visiting my room," "bully girl who dislikes me," or "corporate boss husband scheming to have a child with me." Depending on the conversation flow, sexual contact or coercive situations may also be generated.
The government guidelines include six core principles: protection of user personality rights, disclosure of AI-generated content, prevention of bias and discrimination, management of input data, establishment of risk management systems, and healthy distribution and dissemination. The KCC developed these guidelines after assessing the current state of self-regulation by businesses at the time of their creation. However, since implementation, there has been no verification of whether individual companies adhere to these principles.
The reason is that the guidelines lack legal enforceability. The KCC explained, "The guidelines have no legally binding force and merely recommend voluntary measures by businesses; therefore, compliance was not investigated." Even if principles are violated, no sanctions can be imposed, and businesses have no obligation to report their implementation status.
AI chat apps qualify as value-added telecommunications services under the Telecommunications Business Act and information communication services under the Information and Communications Network Act. They also fall under media content regulations in the Youth Protection Act and AI products/services under the Basic Act on Artificial Intelligence. Despite being subject to multiple laws, there is no procedure to verify the appropriateness of AI dialogues before launch or to conduct ongoing checks on guideline compliance. The current structure involves identifying responsible institutional investors only after issues arise.
Youth protection measures are also left to business discretion. Both Crack and Zeta have established principles in their terms and policies stating efforts to prevent minors from being exposed to harmful information and requiring legal guardian consent for minor transactions. However, the current Information and Communications Network Act contains no provisions directly limiting payment amounts by minors. The maximum single-transaction limits are 126,000 won for Crack and 140,000 won for Zeta, with no cumulative spending caps.
The KCC is currently reviewing whether AI chat app operators qualify as "businesses required to designate youth protection officers" under the Information and Communications Network Act. The target includes information communication service providers with annual revenues of 1 billion won or more or average daily users exceeding 100,000. If designated, businesses must establish youth protection plans and appoint responsible personnel for blocking harmful information and preventing victimization.
A KCC official stated, "We plan to inspect whether AI chat apps subject to mandatory designation allow explicit dialogues and have proper age verification procedures." AI chat apps may also be classified as youth-inappropriate media following review processes. However, this remains a post-event measure activated only after reporting and review. The timeline for reviewing the scope of designated officers and conducting on-site inspections has not yet been finalized.
While discussions on inspection frameworks were underway, services grew rapidly. According to Wise App and Retaile, Zeta's execution count last month reached 2.08 billion times, surpassing ChatGPT's 1.817 billion executions. Crack recorded 122 million executions. On that day, both Crack and Zeta ranked 17th and 19th respectively in Google Play's non-game app revenue rankings. Despite accumulating complaints and expanding user bases and payment volumes, government investigations remain at zero.
in TECH
