Krafton has unveiled an artificial intelligence (AI) foundation model capable of understanding emotions and intonation embedded in voices and generating natural-sounding speech.
On the 29th, Krafton announced that it has open-sourced its voice language model, 'A.X K2 Raon-Speech,' on the global AI platform Hugging Face.
This model is one of the second-phase outcomes of the 'Independent AI Foundation Model' project promoted by the Ministry of Science and ICT. Krafton is participating in SK Telecom's elite team, focusing on multimodal model research.
A.X K2 Raon-Speech consists of 21 billion (21B) parameters. According to Krafton, it ranks first among publicly available voice language models with 30 billion parameters or fewer in overall Korean performance and third in overall English performance.
Evaluation was conducted across six areas: speech recognition and synthesis, speech understanding, voice-text question and answer, tool invocation, and more. A total of 46 benchmarks were utilized, including 24 for Korean and 22 for English.
Krafton combined its own trained voice encoder and codec with SK Telecom's A.X K2-based small language model. The voice encoder converts human voices into information that AI can understand, while the voice codec restores AI-generated information back into natural-sounding speech.
Existing voice conversation AI converts users' speech into text before synthesizing a response back into voice. In this process, emotions and intonation embedded in the voice may be lost. A.X K2 Raon-Speech is designed to understand these voice information elements together, enabling more natural responses.
Last April, Krafton launched its AI model brand 'Raon' and unveiled four models: a voice language model, real-time voice conversation, voice synthesis, and a vision encoder model. The current model applies the training methods of the voice language model developed at that time along with proprietary voice encoder and codec technologies.
Lee Kang-wook, Krafton's Chief AI Officer (CAIO), stated, "We will continuously advance AI foundation technologies required for games to create new gaming experiences."
Krafton plans to utilize this technology in developing an AI character called 'CPC (Co-Playable Character)' that plays games with users. The concept is for the CPC to understand users' voices and intentions and respond with voice appropriate to the game situation.
in TECH
