China Releases AI Safety Governance Framework 3.0 to Strengthen Agent Risk Controls

Large AI models are gradually moving from information-processing tools into enterprise production, research and development, office work, and business decision-making. Several international large-model companies have disclosed information about their models in cybersecurity testing and actual malicious use. OpenAI disclosed related security incidents, and Anthropic published a threat intelligence report covering AI abuse. As AI agents develop, related security issues involve the model output layer, applications, data, identity, and execution permissions, including which tools a model can invoke, which data it can access, which systems it can connect to, how much execution authority it receives, and whether deviations from preset objectives during actual execution can be detected and blocked in time.

After enterprises connect large models to internal knowledge bases, databases, business systems, and various APIs, the links between AI and traditional IT infrastructure deepen. Once connected to an enterprise knowledge base, a large model can invoke internal information; once connected to business systems, it can execute related tasks; and as it develops into an AI agent, it can autonomously plan steps, invoke tools, and execute continuously according to task objectives. Security issues at the model level involve prompt injection, jailbreaking, model theft, training data poisoning, and malicious input. At the application level, connecting large models with plug-ins, APIs, knowledge bases, and business systems introduces new interface and permission risks. At the execution level, as AI agents gain stronger operational capabilities, the security chain involves identity authentication, permission control, behavior auditing, and anomaly interception.

The outline of the 15th Five-Year Plan explicitly calls for comprehensively implementing the AI Plus initiative, improving the safety regulatory framework for new technologies and new business forms, refining relevant laws, regulations, policies, application standards, and ethical guidelines in the AI field, improving systems for algorithm filing, transparency management, and security assessment, and promoting the establishment of a full-lifecycle risk management system for AI, forming a risk prevention and control system covering security monitoring, risk early warning, and emergency response.

On September 14, 2026, the National Technical Committee 260 on Cybersecurity released the AI Safety Governance Framework 3.0. Focusing on new trends brought by AI development and new challenges in safety governance, it updates risk classifications, technical responses, and comprehensive governance measures, further improving the AI safety governance system.