GenAI Security Risks
Home /
Glossary Index /
Alphabet G
GenAI Security Risks in 2025: 7 Threats Your Security Team Cannot Afford to Ignore
Definition: GenAI (Generative AI) security risks refer to the specific cybersecurity threats, vulnerabilities, and risks that arise from the development, deployment, and use of generative artificial intelligence systems, including large language models (LLMs), AI-powered applications, and agentic AI systems. These risks span data exposure, adversarial manipulation, supply chain vulnerabilities, and the use of GenAI tools to enhance cyberattacks.
GenAI security risks differ fundamentally from traditional software vulnerabilities. Because generative AI models produce probabilistic, context-dependent outputs based on training data and user inputs, the attack surface includes not just code but the model itself, the data it was trained on, and the way users interact with it.
Why GenAI Creates a New Security Paradigm
Traditional application security focuses on finding and patching code vulnerabilities. You identify a bug, you fix it, you deploy the patch. The behavior of the application is deterministic: given the same input, it produces the same output.
GenAI systems break this model. Two similar prompts to the same model might yield very different outputs. The same model might refuse a request under normal conditions but comply if the request is framed differently. A model trained on data that included sensitive information might expose that information in generated outputs under certain conditions. These are not bugs in the traditional sense; they are emergent behaviors rooted in how generative AI works.
A 2025 McKinsey survey found that 88% of organizations now use AI in at least one business function. Yet only 24% of GenAI projects formally consider security, according to an IBM Institute for Business Value survey. Gartner predicts that by 2028, 25% of all enterprise GenAI applications will experience at least five minor security incidents per year, up from 9% in 2025.
The gap between adoption speed and security posture is the defining challenge of GenAI security right now.
The 7 Most Significant GenAI Security Risks
1. Prompt Injection Prompt injection occurs when an attacker manipulates the input to an AI model to override its intended behavior or extract sensitive information. In indirect prompt injection, malicious instructions are embedded in content that the AI is asked to process (a document, a web page, an email) rather than entered directly by the user. The AI treats the embedded instructions as legitimate commands.
The EchoLeak vulnerability (CVE-2025-32711) in Microsoft 365 Copilot demonstrated this risk at scale. Attackers used indirect prompt injection in a Copilot/RAG workflow to automatically exfiltrate data without any user interaction, a zero-click data exfiltration attack.
2. Data Leakage Through LLM Inputs When employees use public GenAI tools for work tasks, they often paste sensitive content into the prompt: customer data, internal strategy documents, source code, financial projections. Most public LLM providers use interaction data for model training or store it in ways that could expose it. Companies have seen internal data appear in LLM outputs long after employees submitted it as training context.
This risk exists for public LLMs. Organizations using private or enterprise AI deployments still need to manage what data flows into their AI systems and through what channels.
3. Training Data Poisoning Attackers can introduce malicious or misleading data into a model’s training dataset, causing the model to produce biased, incorrect, or attacker-controlled outputs. In enterprise contexts, fine-tuning models on internal data creates risk if that data can be manipulated before or during the fine-tuning process. A model trained on poisoned data may behave normally in most contexts while producing predictably incorrect outputs for specific inputs that only the attacker controls.
4. Model Inversion and Extraction Attacks Through carefully crafted queries, attackers can attempt to extract information from a model’s training data or reconstruct portions of that data. A model fine-tuned on sensitive internal documents might reveal fragments of that data when queried in specific ways. Model extraction attacks attempt to replicate a proprietary model’s behavior by querying it extensively and training a competing model on the responses.
5. Agentic AI and Autonomous Action Risks Agentic AI systems, which can take autonomous actions like browsing the web, writing and executing code, sending emails, and managing files, represent an expanding attack surface. OWASP’s Top 10 for Agentic Applications (December 2025) highlights goal hijacking, identity abuse, and rogue autonomous behaviors as primary risks. When an AI agent has access to your systems and can take actions without human approval for each step, a single compromised prompt or malicious instruction can trigger a cascade of unauthorized actions.
6. Shadow AI and Ungoverned Deployments Employees adopt GenAI tools without IT or security approval, creating visibility gaps. Security teams cannot protect what they cannot see. Shadow AI deployments may process sensitive data with no data residency guarantees, no audit trail, and no enterprise data protection controls.
7. AI-Augmented Cyberattacks Attackers use GenAI tools to enhance their own attack capabilities. Microsoft’s 2025 Digital Threats Report found that nation-state actors from Russia, China, Iran, and North Korea have more than doubled their use of AI for cyberattacks. GenAI is used to write more convincing phishing emails in multiple languages, generate deepfake audio and video of executives to enable social engineering and fraud, automate the creation of malware variants that adapt to evade detection, and accelerate reconnaissance by rapidly processing publicly available information about targets.
How to Manage GenAI Security Risks
- Establish AI governance policies. Define which AI tools are approved for use, what data can be processed through them, and what approval process applies to new AI deployments. Include AI usage in your acceptable use policy.
- Implement AI-aware DLP. Configure your DLP tools to detect and control the flow of sensitive data into AI applications and APIs. Some DLP platforms now include specific policies for GenAI interactions.
- Conduct AI security assessments. Before deploying any new AI application, assess it for data handling practices, prompt injection vulnerabilities, access controls, and data retention policies.
- Monitor AI usage. Visibility into what AI tools your employees use, and what data flows into them, is a prerequisite for risk management. API gateway monitoring and browser-based controls can provide this visibility.
- Apply least privilege to AI systems. Agentic AI systems should have only the permissions they need for their defined purpose. An AI assistant that can read your emails should not also have the ability to send emails on your behalf unless that is explicitly required and approved.
- Red-team your AI deployments. Adversarial testing of your AI systems, including prompt injection attempts and model behavior analysis, identifies vulnerabilities before attackers do.
Frequently Asked Questions About GenAI Security Risks
Using public LLM tools for work purposes creates data exposure risk if employees paste sensitive content into prompts. Most public LLM providers' terms of service allow the use of user inputs for model improvement or store inputs in ways that could expose them. Organizations should have clear policies about what data can be shared with public AI tools and should evaluate enterprise-tier offerings with stronger data privacy guarantees for sensitive use cases.
Prompt injection is an attack that manipulates an AI model's behavior by inserting malicious instructions into its input. It is dangerous because it can cause AI systems to ignore their safety guidelines, reveal sensitive information, take unauthorized actions, or produce attacker-controlled outputs. In agentic AI contexts, where the AI can take real-world actions, a successful prompt injection could lead to data exfiltration, unauthorized transactions, or system compromise.
GenAI removes one of the most common indicators of phishing: poor language quality. Phishing emails previously often contained grammatical errors, awkward phrasing, or unusual formatting that trained users could recognize. GenAI allows attackers to produce grammatically perfect, stylistically convincing phishing content in any language, at scale, significantly increasing the effectiveness of phishing campaigns.
