Following the announcement by the Ministry of Science and ICT on the 18th of the results from the second phase evaluation of the 'Independent AI Foundation Model' (hereinafter referred to as Dokpamo), Vice Minister Ryu Je-myeong stated, "There is a consensus to restructure the competition format in a way comparable to frontier models," signaling a major shift in direction for the existing Dokpamo project.
However, since the government budget for next year has not yet been finalized, he refrained from providing specific details. Ryu (Vice Minister) said, "A new competitive system will be established, and once the government budget is confirmed, I will provide further information as soon as possible."
In today's second phase evaluation results of Dokpamo, among the four companies challenging for 'National Representative AI'—Upstage, SK Telecom, LG AI Research, and Motif Technologies—Motif Technologies was eliminated, while the remaining three companies advanced to the next stage.
Below is the Q&A from Ryu Je-myeong (Vice Minister)'s press briefing.
-Since detailed scores for each team were not disclosed, only the gaps between evaluations can be confirmed. What indicator showed the most significant change compared to the first phase?
▶The gaps among all four companies in benchmark evaluation, expert evaluation, and user evaluation were extremely narrow. It is difficult to say that any single item was decisive; the margin between the first-place company and those in second and third place was minimal. Please understand this result as a comprehensive outcome of these small differences.
-What was the decisive factor in Motif's elimination?
▶Motif is a technology company that has built its R&D-focused technical capabilities with a very small research team. Demonstrating global competitiveness through highly original algorithms and model design using AI technology was a significant achievement.
However, in this second phase evaluation, the assessment of usability and applicability was expanded. While Motif's technical capabilities were excellent, it received relatively lower evaluations compared to other companies in the usability and applicability areas, which carried considerable weight in this evaluation. I cannot provide further explanation at this time.
-Will there be additional government support for Motif?
▶Apart from Motif applying for and participating in programs the government will promote in the future, I believe it is not appropriate to prepare special measures specifically for Motif.
-Did the controversy over 'benchmaxxing' affect the evaluation?
▶In conclusion, the benchmaxxing controversy was not reflected in this evaluation. As a result of an official analysis commissioned from AAII (Artificial Analysis), AAII stated that it could not find evidence proving that there were issues of overfitting to the Dokpamo evaluation or memorization methods. Through expert evaluations verifying various aspects, we concluded that there was no unfair technical support in the competition.
-Starting from the second phase evaluation, scores from a citizen evaluation panel were newly added. Did this affect the outcome?
▶Our goal was to recruit 200 citizens for the citizen evaluation panel, and we observed intense interest with 1,400 applicants. Ultimately, 185 citizens participated in the evaluation. However, the weight of the general citizen evaluation is only 10 points. After review, it was determined that the general citizen evaluation did not influence the outcome.
-Critics argue that conducting Dokpamo under government leadership may not be appropriate in the era of global AI.
▶This is a concern for both the government and the four elite teams. With model performance from global frontier companies developing exponentially, we are deeply discussing whether the current scale of support and competition format for Dokpamo projects can keep pace with this trend.
Since the government budget has not yet been finalized, it is difficult to provide specific details, but there is a consensus to restructure the Dokpamo approach in whatever way possible to enable competition comparable to frontier company models.
The Dokpamo project will proceed through three phases in the second half of this year, and a new competitive system will be established starting in 2027. Once the government budget is confirmed, I will provide further information promptly.
-Does this mean the Dokpamo goal of 'selecting two final models' may also change?
▶This remains uncertain. We are seriously considering a complete overhaul and restructuring of the current approach.
We are currently discussing with the companies involved whether to disperse or concentrate the resources invested so far, and whether it is advisable for companies to form consortia. Within the government, discussions are ongoing regarding various aspects such as the possibility and effectiveness of financial support.
The original goal of Dokpamo was to select two final models through the third phase evaluation and then develop these models to global standards over one year starting in 2027. However, this is not merely about succeeding with one or two companies. The larger objective is to enhance the competitiveness of Korea's AI ecosystem. Please understand this as a process where new startups rise through government projects and funding, elevating our country's technological capabilities, rather than focusing on who remains behind.
-Was there any team that consistently ranked first in both the first and second phase evaluations?
▶No team maintained a consistent first-place ranking.
in TECH
