Krafton has applied its AI-native recruitment and evaluation solution, 'Cofa-Probe,' which evaluates the entire process of solving problems alongside AI, to a real-world hackathon.
According to Krafton on the 28th, Cofa-Probe, developed in-house by Krafton, is a solution that evaluates not only the final output but also how applicants work with AI. It analyzes the entire process from problem definition to role allocation and instructions for AI, evidence exploration, decision-making, error correction, and result verification.
Applicants collaborate with multiple work agents in an environment similar to actual job tasks. During this process, Cofa-Probe automatically collects conversations, prompts, code, tool configurations, and verification records generated. After the task concludes, applicants participate in a post-task Q&A session where they explain both their final output and decision-making process. The system then synthesizes this information to produce an evaluation result and an interview reference report.
Evaluation is conducted across three core dimensions: Technique, Intent, and Cognition. Assessments examine not only code executability and structure but also security, whether the AI was accurately informed of goals and constraints, whether applicants directly verified and corrected AI-generated results, and whether they can explain how the results work and potential failure scenarios.
The core technology behind Cofa-Probe is 'meta-harness,' which evaluates the evaluation AI itself. Evaluation based on large language models (LLMs) may yield different results for the same submission or fail to detect actual capability differences between submissions. To address this, Krafton repeatedly tested and improved the execution structure of its evaluation agents using OpenAI's Codex.
The system verified whether scores and ratings remained stable across multiple evaluations of the same submission and whether it could properly distinguish between submissions with differing levels of quality and approach. Consistency-critical elements such as evaluation criteria and score conversion were handled by code, while qualitative judgments requiring contextual understanding were managed by LLM-based evaluation agents.
Cofa-Probe was applied to the recruitment-linked AI Native hackathon 'Cofathon: AI Native Battlegrounds,' held on July 30 at PUBG Seongsu in Seongsu-dong, Seoul. The event was co-hosted by Krafton and CJ Olive Young, with OpenAI participating as a technical partner. The program included tracks for Krafton's Forward Deployed Engineers (FDE) and CJ Olive Young AI engineers.
Participants tackled tasks aimed at solving real-world inefficiencies using AI. Krafton collected data on each participant's AI collaboration process from task execution through submission and conducted real-time evaluations. After submissions, open discussions were held based on blinded evaluation reports to review scores alongside the rationale behind judgments.
Krafton demonstrated that Cofa-Probe could present concrete evidence of problem-solving approaches and AI utilization capabilities that are difficult to discern from final outputs alone. It also confirmed that the same evaluation principles can be applied regardless of job roles or task types.
Park Jae-min, Head of Krafton's AI Frontier Division, stated, "Competitiveness in the AI era comes from defining problems with AI, verifying evidence, and rigorously validating results to completion." He added, "Cofathon provided an opportunity to observe participants' actual working methods beyond their final outputs. Krafton will further advance Cofa-Probe to establish new standards for evaluating AI-native talent."
Kim Tae-young, Account Director at OpenAI Korea, remarked, "Cofathon was significant not only because Codex supported real-world tasks but also because it helped validate and enhance Cofa-Probe's evaluation framework." He continued, "Moving forward, we will continue collaborating with Krafton to explore diverse possibilities for deeper integration of OpenAI technologies into actual business operations and organizational problem-solving."
Krafton plans to continuously refine Cofa-Probe's evaluation criteria and agent environments based on operational experience and field feedback gathered from Cofathon.
in TECH
