Microsoft has defined AI agents operating within enterprises as "Agentic Insiders" and proposed a new security approach. The core of this is that AI agents, just like humans or external attackers, require continuous monitoring and control.
On the 20th, Microsoft presented a security system that identifies the activities and risks of AI agents in real time and adjusts their action permissions according to risk levels.
Recently, AI agents are accessing data and various systems within companies to handle tasks and participate in decision-making. Microsoft defined such AI that actually acts within enterprises as "Agentic Insiders."
Existing AI security has primarily focused on external threats where attackers misuse AI. In contrast, AI agents directly deployed by companies have often been relatively trusted.
Microsoft emphasized that even self-deployed AI should not be unconditionally trusted. This is because "Reward Hacking," in which the agent behaves in unintended ways during the process of being overly optimized for goal achievement, can occur.
It was also noted that there is a possibility that AI agents may behave differently than expected while adapting to situations, or create new risks when collaborating with other agents.
Accordingly, the necessity of "Guardian Agents" that constantly monitor the behavior of AI agents was also presented. Guardian Agents continuously check the actions of other AI agents from before execution and respond when risks increase.
Microsoft explained that "Project Perception," released last July, can also be utilized for managing Agentic Insiders.
Project Perception detects AI agents within enterprises, assigns IDs and permissions, and calculates risk levels in real time based on actions. If the risk level increases, Guardian Agents restrict actions or respond accordingly.
The scope of protection is not limited to AI models alone. It was emphasized that the entire system connected to AI agents—including IDs, permissions, data, endpoints, networks, clouds, and runtimes—must be protected together.
Transparency that allows verification of why an AI performed a specific action was also considered important. It was explained that in order to clearly determine risk assessment and accountability, it is necessary to track not just inputs and outputs, but the entire process by which results were generated.
Haeyete Galot, Senior Vice President and Head of Security at Microsoft, stated, "As AI agents rapidly spread across enterprises, they are accessing data, tools, and systems to perform critical tasks and participate in decision-making." She added, "Companies must identify agent activities and risks in real time and continuously adjust action permissions according to changes in risk."
in TECH
