Most organizations get AI security wrong.
They focus on keeping sensitive data out of large language models. They guard against prompt injection. Meanwhile, the real exposure happens somewhere else entirely: at execution.
Here is what execution looks like. An AI agent with legitimate credentials opens a database, pulls a report, sends it through a processing pipeline, and forwards the output to a third-party integration. Every single step looks normal. The credentials are valid. The access patterns match expected behavior.
But somewhere in that chain, the agent’s instructions were altered. The data is now moving to a destination it should not reach. Traditional security tools see nothing wrong. They are watching who accessed what. Nobody is watching what the system actually did.
That gap is the difference between traditional security and AI security.
The surface-level concerns (data leakage into training sets, model theft, prompt injection) are real. But the harder problem is governing autonomous systems that operate inside your perimeter, using legitimate credentials, performing actions that look productive by design.
Quick orientation: the nine practices ahead
| Practice | Core question it answers | Difficulty |
|---|---|---|
| 1. AI bill of materials | What do we even have? | Low |
| 2. API and endpoint security | How do things connect? | Low |
| 3. Zero trust and least privilege | Who can do what? | Medium |
| 4. Poisoning and adversarial defense | Is our model being manipulated? | High |
| 5. Behavioral monitoring | What are our systems actually doing? | High |
| 6. Third-party vetting | Can we trust our dependencies? | Medium |
| 7. Data encryption and privacy | Where does sensitive data go? | Medium |
| 8. Governance and accountability | Who is responsible when something fails? | Medium |
| 9. Employee training | Do our people know what not to do? | Low |
Why traditional security misses the point with AI
Traditional cybersecurity answers three questions: Who accessed the system? From where? What data did they touch?
That model works when humans are the actors. A login at 3 a.m. from an unrecognized location triggers an alert for a reason.
AI agents do not fit that model.
They log in from expected IP ranges. During business hours. Using provisioned service accounts. They access exactly the systems they were configured to access. The anomaly is not in the access pattern. It is in the sequence of actions, the combination of operations, the emergent behavior that no individual log entry captures.
Human threat vs AI threat: What monitoring catches
| Signal type | Human attacker | AI agent compromise |
|---|---|---|
| Unusual login time | Common, triggers alert | Rare, looks normal |
| Unfamiliar IP | Common, triggers alert | Rare, uses expected ranges |
| Unauthorized data access | Detected by access controls | Uses legitimate credentials |
| Suspicious sequence of actions | Often noisy or obvious | Blends into normal operations |
| Volume anomaly | Could indicate exfiltration | Could indicate abuse, but looks like legit usage |
The NIST AI Risk Management Framework, released in January 2023 and updated with additional guidance in 2024, was one of the first major standards to explicitly call out this gap. It recommends mapping AI system behavior across the full lifecycle rather than auditing discrete access events. MITRE ATLAS, the adversarial threat landscape framework for AI systems, catalogs tactics and techniques that exploit exactly this blind spot: attackers coercing AI systems into actions that blend into normal operations.
If your AI security strategy starts and ends with access control and data classification, you are defending against yesterday’s threat model.
1. Build an AI bill of materials before anything else
You cannot secure what you cannot see.
That principle applies to AI with more urgency than most teams realize.
An AI bill of materials (AI-BOM) is an inventory of every component that touches your AI systems:
- Models (production, staging, development)
- Training datasets and fine-tuning data
- Inference pipelines
- Embedding services and vector databases
- API integrations
- Third-party libraries and their dependencies
Most organizations, when they first build one, discover components they did not know were connected to AI workflows.
A model card is the documentation layer that sits on top of the AI-BOM. For each model, it records:
- What the model does
- What data it was trained on
- Known limitations
- Security requirements
- Who is responsible for it
Without model cards, your inventory is just a list of names. With them, it becomes something your security team can act on.
Building your first AI-BOM
Start by cataloging every model across production, development, and staging environments. Expect this to take three to five days if someone knows where everything lives.
Then map each model’s dependencies — frameworks, libraries, data sources, and external APIs. That is another five to eight days.
Next, identify which identities (human and service accounts) have access to each component, which should take three to five days.
Finally, create a model card for each model documenting its purpose, data provenance, and security posture. Budget one to two days per model for this.
Aim for completeness over perfection on the first pass.
A partial inventory with a few gaps is actionable. No inventory at all means you are flying blind. Teams that skip this end up finding out about shadow deployments during incidents, not audits.
2. Secure APIs, endpoints, and model access points
AI models do not sit in isolation. They are exposed through APIs, embedded in applications, and connected to data pipelines that stretch across cloud boundaries.
Each connection point is an attack surface.
The four API security layers most teams skip
Layer 1: Authentication. API keys are not enough. Use OAuth 2.0 tokens with short lifetimes. Rotate credentials on a schedule that assumes compromise. A leaked API key valid for six months is a gift to an attacker.
Layer 2: Input validation. Do not pass raw user input directly to a model. Sanitize it. Validate schema conformance. Reject anything that does not match expected formats. Model inference endpoints should never be the first line of input validation.
Layer 3: Rate limiting. Apply it at multiple levels. A single misconfigured client should not be able to exhaust your inference capacity. Rate limiting protects availability and makes abuse more expensive.
Layer 4: Network isolation. If a model does not need to be reachable from the public internet, it should not be. Configure private network endpoints for model serving infrastructure. Route traffic through your existing network security controls.
API keys exposed in client-side code can be scraped and abused within minutes. This is not unique to any single provider. It happens across every model service.
The fix is not a better API key. The fix is never putting keys where they can be scraped.
3. Lock down authentication with zero trust and least privilege
AI systems interact with data stores, internal services, and external APIs. Each interaction happens through an identity.
Each identity should have exactly the permissions it needs and nothing more.
Role-based access control (RBAC) is the starting point, not the destination.
Identities in a typical AI pipeline and what they need
| Identity | Should access | Should never access |
|---|---|---|
| Training job service account | Curated training data bucket | Model registry (write) |
| Inference endpoint identity | Model weights (read) | Training data |
| Data preprocessing worker | Raw data staging area | Production inference endpoints |
| Monitoring agent | Logs and metrics | Model weights or training data |
When a training job that should only read from a curated dataset bucket suddenly attempts to write to a model registry, that should fail by default. Not because someone remembered to check, but because the permission was never granted.
Multi-factor authentication (MFA) should apply to every human accessing AI infrastructure. For service accounts, use managed identities where your cloud provider supports them. This eliminates static credentials entirely.
The zero-trust principle matters more for AI than for most workloads. An AI agent operating under a compromised identity can chain together legitimate API calls in ways that produce illegitimate outcomes. If each step in that chain requires explicit, scoped authorization, the blast radius shrinks to a single operation instead of an entire workflow.
A practical rule: if you cannot explain, in one sentence, exactly what permissions a given service account has and why, it has too many.
4. Defend training pipelines against data poisoning and adversarial inputs
Data poisoning does not announce itself.
An attacker introduces a small number of crafted examples into a training dataset (sometimes as few as a few hundred samples in a corpus of millions). The model learns a behavior the attacker controls. The behavior might activate only under specific input conditions, making it invisible during normal testing.
Adversarial inputs work differently. They target inference time rather than training time. A stop sign with carefully placed stickers makes a vision model see a speed limit sign. A block of text with strategically inserted characters makes a language model produce a specific, unintended output.
Both attacks exploit the same underlying vulnerability: models learn correlations from data, and those correlations can be manipulated.
Data poisoning vs adversarial attacks at a glance
| Data poisoning | Adversarial attacks | |
|---|---|---|
| When | Training time | Inference time |
| How | Contaminated training data | Crafted inputs to deployed model |
| Visibility | Invisible during normal testing | Visible in model output |
| Detection | Statistical checks on training data | Input sanitization + output monitoring |
| Prevention | Data provenance validation | Adversarial training + preprocessing |
Defense strategies across both attack types:
- Validate dataset provenance. Know where every training dataset came from and who had access to it before it reached your pipeline.
- Run statistical checks on training data distributions before ingestion. A sudden change in label distribution or feature values can signal tampering.
- Implement adversarial training. Generate attack examples during development and include them in your training set so the model learns to recognize manipulation attempts.
- Preprocess inference inputs through sanitization layers. Strip or neutralize known attack patterns before they reach the model.
None of these defenses is perfect on its own. Deployed together, they raise the cost of a successful attack from trivial to expensive. That is often the difference between being targeted and being ignored.
5. Monitor what your AI actually does, not just what it accesses
This is where most security programs fall short.
Traditional monitoring answers: did someone access the customer database?
AI monitoring needs to answer: did the agent that accessed the customer database then send that data somewhere unexpected?
The access itself was legitimate. The sequence of subsequent actions is where the risk lives.
What to monitor across the AI stack
| Layer | What to watch for | Tool category |
|---|---|---|
| Model inputs and outputs | Distribution shifts that suggest manipulation | Drift detection |
| AI agent API call sequences | Patterns that deviate from established baselines | Behavioral analytics |
| Resource consumption | Anomalies suggesting resource jacking (crypto mining, unauthorized training) | Infrastructure monitoring |
| Data movement across trust boundaries | Legitimate-looking transfers to unexpected destinations | Data flow analysis |
Anomaly detection is useful here, but behavioral baselining matters more.
You need to know what normal looks like before you can recognize abnormal. For an AI agent that typically processes 50 transactions per hour during business hours, 2,000 transactions at 3 a.m. is worth investigating, even if every transaction succeeds.
Static analysis looks at code. Runtime observes what the code actually does when it executes. Most security programs are strongest at the layers that matter least for AI.
6. Vet every third-party component
AI development depends on open-source models, community frameworks, pre-trained weights, and third-party datasets.
Each dependency is a potential entry point.
The risk is not hypothetical. Pre-trained models downloaded from public repositories can carry unexpected behavior. Malicious weights can be embedded in model files. Third-party datasets can contain poisoned samples that influence model outputs in ways that only surface under specific conditions.
Vetting third-party components means:
- Checking provenance of pre-trained models. Where did the weights come from? Who trained them? On what data?
- Running dependency scans on AI frameworks and libraries, the same way you scan application dependencies for CVEs
- Testing model files for embedded code before loading them into any environment with network access
- Maintaining an allowlist of approved model sources and blocking imports from unvetted repositories
Third-party component vetting checklist
| Component type | Risk | Verification step | Tooling |
|---|---|---|---|
| Pre-trained models | Embedded malicious code | Test model files before loading | Model scanning tools |
| AI frameworks and libraries | Known CVEs | Run dependency scans (same as app deps) | SCA scanners |
| Training datasets | Poisoned or biased data | Validate provenance and distribution | Statistical profiling |
| Model weights from public repos | Backdoor weights | Verify source and integrity hash | Cryptographic verification |
Open-source AI accelerates development. Blocking it entirely is not practical for most teams. The alternative is treating external AI components with the same scrutiny you apply to any other software supply chain: automated scanning, provenance verification, and runtime isolation.
7. Encrypt, anonymize, and control sensitive data
AI systems consume data. Some of that data is sensitive.
Customer records. Proprietary code. Internal communications. Financial projections.
Every place data pauses or passes through is a place it can leak.
Encryption is table stakes. Data at rest should be encrypted. Data in transit should be encrypted. But encryption alone is not enough.
AI models can memorize and later reproduce training data. In 2021, researchers demonstrated that large language models can regurgitate verbatim training examples under certain prompting conditions, including personally identifiable information (Carlini et al., USENIX Security 2021.
Privacy preservation techniques compared
| Technique | What it does | Strength | Limitation |
|---|---|---|---|
| Encryption (at rest/in transit) | Prevents unauthorized access to stored/moving data | Strong | Does not prevent model memorization |
| Differential privacy | Adds calibrated noise, making it mathematically infeasible to identify individual records | Formal guarantee | May reduce model accuracy slightly |
| Role-based access controls | Limits which identities can access which datasets | Operational | Does not protect against authorized users misusing data |
| Data loss prevention (DLP) | Catches sensitive data in prompts and responses before it leaves | Fast, practical | Can miss novel sensitive data patterns |
| Regular audits | Confirms what data each model can access and whether access is justified | Catches drift over time | Requires manual effort |
If an employee pastes a customer list into a model prompt, DLP should block it. If a model is memorizing training data, differential privacy makes extraction infeasible. If permissions have drifted over six months, an audit catches it.
No single technique solves the problem. The combination does.
8. Establish governance before regulators force your hand
AI regulation is not coming. It is here.
The EU AI Act entered into force in August 2024, with obligations for high-risk AI systems phasing in over 2025 through 2027. NIST continues to update the AI Risk Management Framework with sector-specific profiles. ISO/IEC 5338 and ISO/IEC 27090 (under development for AI security) are the two most relevant international standards.
Organizations that build governance frameworks now will spend 2026 adjusting controls. Organizations that wait for enforcement actions will spend 2026 scrambling.
The minimum viable governance program
| Component | What it means | Who owns it |
|---|---|---|
| Current AI inventory | Updates to the AI-BOM whenever a model is added, changed, or retired | ML engineering lead |
| Audit trails for AI decisions | Log inputs, outputs, and model version for every consequential decision | Security engineering |
| Named accountable person per system | Someone who can answer “why did this system do that?” and has the authority to pull it offline | System owner (not the model) |
| Regular review cadence | Reviews tied to model updates, regulatory changes, and incident post-mortems | GRC team |
Explainable AI techniques help, but the minimum bar is logging inputs, outputs, and model version for every consequential decision.
When a model denies a loan, flags a transaction, or routes a patient to a specific treatment path, someone needs to be able to trace why.
Governance is not a one-time project. The frameworks (NIST RMF, ISO standards, the EU AI Act) provide the scaffolding. Your job is to put walls on it that correspond to what your AI systems actually do.
9. Train every employee, not just the security team
Shadow AI is the fastest-growing category of AI risk in most organizations.
The term describes employees using AI tools without the security team’s knowledge or oversight. This is the core of what we call shadow AI. Managed AI services are now widespread across organizations of all sizes, but many security teams still cannot enumerate which AI tools their employees are actively using.
Real examples of what employees do:
- Paste proprietary code into public chatbots for debugging help
- Upload customer contracts to summarize with AI
- Connect personal AI accounts to work data without changing default privacy settings
Each of these is well-intentioned. Each creates exposure.
AI security training: what every employee needs to know
| Topic | The message |
|---|---|
| Approved tools | These are the AI tools you can use, and here is why the approval process exists |
| Forbidden data | Never enter these categories of data into a public AI service |
| Privacy settings | Here is how to check whether an AI tool’s default settings expose company data |
| Reporting | How to report suspected AI misuse (without fear of punishment) |
Policy should be clear, short, and enforced.
A 15-page AI usage policy that nobody reads is worse than a one-page policy that everyone knows. Make the rules obvious. Make the enforcement visible. The shadow AI problem shrinks from a crisis to a manageable risk.
What these practices cost in time and resources
Nobody talks about cost in AI security articles. That is why most teams start three practices and finish none.
Effort and resource estimates
| Practice | Effort | Team needed | Cost tier | Start this quarter? |
|---|---|---|---|---|
| AI-BOM | 2-4 weeks initial | Security + ML engineering (2-3 people PT) | Low | Yes |
| API/endpoint security | 1-2 weeks per integration | Platform engineering | Low | Yes |
| Zero trust + least privilege | 4-8 weeks, phased rollout | Identity + platform teams | Medium | Yes, phased |
| Data poisoning defenses | 6-12 weeks for pipeline integration | ML engineering + security | High | Roadmap |
| Behavioral monitoring | 8-16 weeks for baseline + tuning | Security operations + ML engineering | High | Roadmap |
| Third-party vetting | 2-4 weeks for process | Security + procurement | Medium | Yes |
| Data privacy (diff. privacy + DLP) | 4-8 weeks depending on data volume | Data engineering + security | Medium | Yes |
| Governance framework | 6-12 weeks initial | GRC + legal + security | Medium | Yes |
| Employee training | 2-4 weeks for materials | Security awareness + L&D | Low | Yes |
Which practices to start with, by team size and maturity
| If you have… | Start here | Add next quarter |
|---|---|---|
| No AI security program | AI-BOM + API security | Training + third-party vetting |
| Basic visibility, no monitoring | Behavioral monitoring | Data privacy + governance |
| Monitoring in place, weak governance | Governance framework | Data poisoning defenses |
Practices 1, 2, and 9 can start this quarter with existing staff and minimal budget. Practices 4 and 5 require dedicated engineering time. Plan them into the roadmap rather than attempting them as side projects.
How priorities shift by organization size
The same nine practices apply regardless of size. But the order of operations changes.
Startup (under 50 people, fewer than 5 AI models in production)
Start with practice 1 (AI-BOM) and practice 2 (API security). You need to know what you have, and you need to secure how it connects to everything else. Practices 8 and 9 (governance and training) should follow within the first year. Advanced monitoring and differential privacy can wait until you have something worth stealing.
Midmarket (50-500 people, 5-25 AI models)
Practices 1 through 4 are table stakes. You should not be deploying without them. Practices 5 (behavioral monitoring) and 6 (third-party vetting) become critical as your attack surface grows with each new integration. Practice 8 (governance) starts becoming a customer requirement, especially if you sell to enterprises.
Enterprise (500+ people, 25+ AI systems)
All nine practices apply with equal urgency. The difference is scale. An AI-BOM at enterprise scale is a program, not a document. Behavioral monitoring requires dedicated headcount. Governance needs to satisfy regulators, auditors, and customers with conflicting requirements. The practices do not change, but the cost of skipping any one of them compounds across the organization.
The regulations shaping AI security
AI security regulation is moving faster than most teams’ compliance programs.
| Regulation / Framework | Status | Who it affects | Key deadline |
|---|---|---|---|
| EU AI Act | In force (Aug 2024) | Any org with AI systems touching EU users | High-risk obligations phase in 2025-2027 |
| NIST AI RMF | Voluntary, but becoming de facto standard | U.S. orgs, especially those selling to government/enterprise | Sector-specific profiles coming (critical infrastructure first, 2026) |
| ISO/IEC 27090 + 5338 | 5338 published 2023; 27090 under development | International compliance programs | Timeline TBD; early alignment via OWASP AI Exchange |
| CISA AI Data Security Guide | Released May 2025 | Defense industrial base, critical infrastructure | Available now; recommendations apply broadly |
EU AI Act. In force since August 2024. High-risk AI system obligations (risk management, data governance, transparency, human oversight) phase in over 2025-2027. Fines reach the higher of EUR 35 million or 7% of global annual turnover. If your AI systems touch EU users, this applies to you regardless of where your company is based.
NIST AI RMF. Voluntary today. But it is becoming the standard that U.S. regulators, auditors, and enterprise customers reference. NIST released a concept note in April 2026 for a profile on trustworthy AI in critical infrastructure. More sector-specific profiles are coming. Organizations that map their controls to NIST RMF now will have an easier time adapting to whatever follows.
ISO/IEC standards. ISO/IEC 5338 (AI system life cycle processes) was published in 2023. ISO/IEC 27090 (AI security) is under development, with OWASP contributing content directly through its AI Exchange. Once 27090 is published, these will become the baseline that international compliance programs reference.
Frequently asked questions
How is AI security different from traditional cybersecurity?
Traditional cybersecurity focuses on perimeter defense, access control, and data protection.
AI security adds behavioral monitoring at execution: watching what AI systems do with the access they have, not just controlling who gets through the door.
An AI agent with legitimate credentials can chain together operations that produce harmful outcomes. No access log will flag the sequence as anomalous unless you are specifically monitoring for it.
What is an AI bill of materials and why does it matter?
An AI-BOM is an inventory of every component in your AI pipeline: models, datasets, frameworks, APIs, and dependencies. You need one because you cannot secure what you do not know exists. Most organizations discover shadow AI deployments and undocumented data connections when they build their first AI-BOM.
What are adversarial attacks and how do they differ from data poisoning?
| Data poisoning | Adversarial attacks | |
|---|---|---|
| Timing | During training | During inference |
| Method | Contaminate dataset | Craft deceptive input |
| Goal | Model learns attacker-controlled behavior | Model produces wrong output |
| Analogy | Tainting the water supply | Handing someone a map to the wrong destination |
Why are APIs the most common attack vector for AI systems?
AI models are almost never accessed directly. They sit behind APIs.
APIs expose authentication tokens, rate limits, input validation, and authorization logic. Every one of those layers can be misconfigured. Attackers scan for exposed API keys because those keys give them exactly the same access your application has. Once inside, they can query models at scale, extract training data through repeated prompting, or pivot to other services the model connects to.
What role does NIST play in AI security standards?
NIST’s AI Risk Management Framework is the most widely referenced AI security standard in the United States.
It is voluntary. But it shapes how regulators, auditors, and enterprise procurement teams evaluate AI systems. NIST also publishes adversarial machine learning guidance and has signaled that sector-specific AI RMF profiles are coming, starting with critical infrastructure.
Which best practice should an organization implement first?
Build an AI-BOM.
You need to know what AI systems you have, where they run, what data they access, and who is responsible for them before you can effectively secure any of it. The AI-BOM is not the most exciting practice on this list. It is the one that makes every other practice possible.
How often should AI security practices be reviewed?
| Practice area | Review frequency | Triggered by |
|---|---|---|
| AI-BOM and access controls | Quarterly | New model deployment |
| Governance frameworks | When regulations change | New model types deployed |
| Adversarial testing | Before every major model release | Material change to training data or architecture |
| Monitoring baselines | Continuously | System behavior evolution |
Where to go from here
Pick one practice and ship it this month.
The AI-BOM is the obvious starting point. It creates the visibility that every subsequent practice depends on. If you already have one, move to API and endpoint security. If you have both, start on behavioral monitoring. That is where most teams discover problems they did not know they had.
The gap between having an AI security strategy and having one that actually works is not about the number of practices you adopt. It is about whether you are watching what your AI systems do or just what they access. The organizations that get this right stop treating AI like another application workload and start treating it like an autonomous decision-maker operating inside their perimeter.
For autonomous agents specifically, the practices tighten into 12 agentic AI security controls you can verify by artifact.