The government is reviewing whether to disclose additional scores related to the second evaluation of "DOKPAMO" (Domestic AI Foundation Model), which selects national representative AIs. This move follows an appeal filed by Motif Technology, which was eliminated in the second round of evaluations earlier today, demanding a specific disclosure of scores.
Kim Kyung-man, Director of the Ministry of Science and ICT's AI Division, stated on the morning of the 27th during a meeting with reporters regarding Motif's appeal: "We will place maximum value on the fairness and objectivity of the DOKPAMO system and decide and announce the scope for releasing additional scores by this afternoon."
The Director said, "We do not wish to release specific company scores that could fuel competition or highlight negative images of eliminated companies," adding that they will continue to prioritize the fairness and objectivity of evaluations.
Earlier today, Motif Technology filed an appeal requesting specific evaluation scores for usability and applicability assessments, which were cited as reasons for their elimination in the second round of DOKPAMO evaluations. This led to follow-up questions regarding the specific criteria for user and expert evaluations, as well as how evaluation standards change at each stage.
Regarding this, the Director stated, "The principle of DOKPAMO is that it must be fair and leave no room for doubt," explaining, "We did not release scores by category during the second round because we had no intention of releasing scores to fuel competition among companies." However, he noted that individual evaluation scores were conveyed to each company, allowing them to verify results by comparing their own scores with the overall average.
He also expressed concerns that due to the competitive nature of DOKPAMO, releasing scores could lead to models from lower-ranked companies being perceived negatively beyond their actual performance. The Director said, "We are concerned that the models of eliminated companies might be perceived as poor models," and added, "We hope Motif's achievements are recognized alongside those of other companies."
He explained that the changing criteria by round were agreed upon between the industry and evaluators. He emphasized, "The criteria for each DOKPAMO evaluation item are established through agreements between evaluators and participating companies, not unilaterally decided by the government," noting that this evaluation item was also announced through prior agreement.
As AI trends change constantly, the first round focused on securing originality in LLMs (Large Language Models), while the second round shifted focus to applicability in an environment where AI agents are becoming prominent. He also mentioned that AA (Artificial Analysis) evaluation is recognized as the most credible benchmark in the industry, which is why it was included as an evaluation item.
in TECH
