Following transparency concerns surrounding the second-stage evaluation of the government's "Independent AI (Artificial Intelligence) Foundation Model" project, additional details were released, including the top-ranked companies and their scores for each category. Motive Technology led in the global benchmark, SK Telecom topped the domestic benchmark and AI professional user evaluation, while LG AI Research achieved the highest score in evaluations by experts and general citizens.
On the 20th, the Ministry of Science and ICT issued a press briefing document outlining the criteria and results of the second-stage Independent AI Foundation Model evaluation, disclosing the top-ranked companies and their scores for each category at the same level as the first-stage evaluation.
Initially, the Ministry of Science and ICT did not disclose the top-ranked companies or their scores in the second-stage evaluation. This decision was made considering that during the first-stage evaluation, announcing the top-ranked companies led to increased recognition beyond their actual scores for those firms, while potentially stigmatizing other participants. However, amid growing criticism demanding transparency in evaluating a project funded by government budgets, the ministry reversed its stance. Nevertheless, total scores and overall rankings by company were not released.
The second-stage evaluation was conducted out of 100 points total: 40 points for benchmark testing, 35 points for expert evaluation, and 25 points for user evaluation. It specifically incorporated demands raised after the first-stage evaluation to strengthen checks on independence and usability assessments. The weight for this category was increased from 10 to 15 points, with evaluation criteria established through consensus among selected teams. The aim was to assess not only model performance but also practical usability, potential for real-world deployment, and contributions to Korea's AI ecosystem.
In the AAII (Artificial Analysis Intelligence Index) benchmark measuring global AI performance, Motive Technology ranked first with 11.9 points out of a possible 25. The average score across all four teams was 9.48 points. Evaluations covered agents, coding capabilities, general performance, and scientific reasoning.
In the Korea Institute for Intelligence Development (NIA) benchmark assessing seven areas—mathematics, knowledge, long-text understanding, safety, reliability, Korean language proficiency, and instruction following—SK Telecom achieved the highest score of 13.4 out of 15 points. The average score across all four teams was 13.05 points.
In an expert evaluation involving ten external specialists, LG AI Research secured first place with 29.5 points out of 35. Evaluations covered development strategy, technology and outcomes, future plans, and the ripple effects on the AI ecosystem. Additionally, minimum standards for independence across technical, policy, and ethical dimensions were reviewed.
In user evaluations using actual AI models, SK Telecom and LG AI Research led in their respective categories. Among 49 professional users, including representatives from AI startups, SK Telecom ranked first with 11.6 points out of 15. In an evaluation involving 185 general citizens, LG AI Research received the highest score of 7.6 out of 10.
The Ministry of Science and ICT stated, "The Independent AI Foundation Model project is not intended as a simple ranking competition or to eliminate teams." It further explained, "We balanced considerations for fairness and transparency in the evaluation process and results while minimizing direct and indirect harm to companies resulting from public disclosure." The ministry added, "We will expand policy support to enable each company to continuously develop its AI models and technologies and facilitate their practical application and widespread adoption in real-world settings."
in TECH
