Key takeaways
- LLM supply chain security covers every third-party model, adapter, dataset, and repository you depend on. OWASP calls it LLM03:2025.
- You have a LLM supply chain even if you never trained anything: proxies, gateways, and MCP servers all count.
- Poisoned weights pass standard benchmarks, so you cannot diff or hash your way to trust.
- Start with an inventory, then verify provenance and red team continuously.
LLM supply chain security is the practice of protecting every third-party component that goes into building, training, and deploying a large language model, from pre-trained weights and fine-tuning adapters to the datasets, repositories, and infrastructure that touch them before your application ever sees a prompt. OWASP formalized it as LLM03:2025 Supply Chain, one of the top risks in its Top 10 for LLM Applications, precisely because a compromise anywhere in that chain can undermine a system that otherwise looks secure on the surface.
If you’re pulling a fine-tuned model off Hugging Face, calling a hosted API through a proxy gateway, or feeding a vector database from scraped documents, you already have a supply chain. Whether you trained anything yourself is irrelevant.
How LLM supply chains differ from traditional software supply chains
Traditional software supply chain security has a working playbook. When a package gets flagged:
- Read the code
- Diff the version
- Verify a hash
- Swap in a patched release
None of that translates cleanly to LLMs.
A model file is a few hundred gigabytes of floating-point weights. You can’t read it like source code. You can’t diff two checkpoints and understand what changed in any meaningful sense. And a model can pass every static scan you throw at it while still behaving maliciously the moment a specific trigger phrase shows up in a prompt. That’s the core problem: LLM supply chain risk lives partly in behavior, not just in code, and you can’t hash a behavior.
This distinction matters because it changes what “verification” even means. A checksum tells you the file wasn’t altered in transit. It tells you nothing about whether the training data was poisoned six months before that file was ever uploaded.
The core components of the LLM supply chain
Most teams think of their supply chain as “the model we’re using.” In practice it’s five separate surfaces, each with its own failure modes.
| Component | What it includes | Typical risk |
|---|---|---|
| Pre-trained models | Foundation models pulled from public or commercial repositories | Backdoors, tampered weights, unclear training provenance |
| Fine-tuning adapters | LoRA and PEFT adapters layered onto a base model | Malicious adapters merged in without re-validating the combined model |
| Training & fine-tuning datasets | First-party and scraped/public data used to train or adapt a model | Data poisoning, licensing violations, embedded bias |
| Model repositories & registries | Hugging Face, package indexes like PyPI, internal model stores | Fake or typosquatted uploads, compromised accounts, scanning gaps |
| Deployment & serving infra | Inference servers, orchestration frameworks, RAG pipelines, vector stores | Shared-infrastructure exploits, misconfigured access, injected retrieval content |
Two of those, fine-tuning adapters and repositories, get far less attention than the model itself. That’s exactly where the interesting failures happen.
Key LLM supply chain security risks
Data poisoning and backdoored modelsMust-have
Data poisoning is an attacker inserting manipulated samples into a training or fine-tuning set so the model learns a hidden behavior alongside its legitimate one. The model performs normally on every benchmark you’d think to run. Then a specific input, sometimes a single unusual phrase, flips it into a different mode entirely: leaking data, producing biased output, or executing an instruction it was never supposed to follow.
Mithril Security demonstrated this publicly with PoisonGPT: a modified version of the open-source GPT-J-6B model, edited with the ROME algorithm to spread misinformation on a narrow set of facts while performing identically to the original on every standard evaluation. Nobody running a benchmark suite would have caught it. That’s the point.
Vulnerable and tampered pre-trained modelsMust-have
Separate from poisoned training data, this is direct tampering with a model’s weights after training, sometimes through techniques like ROME (Rank-One Model Editing) that let someone alter specific facts or behaviors in a model’s parameters without retraining it from scratch. A model can be edited in minutes, re-uploaded under a familiar name, and downloaded by anyone who trusts the source.
Microsoft’s brief, accidental release of WizardLM 2 in April 2024 is a good illustration of how little control anyone has over a model once it’s public. Microsoft pulled it within hours after realizing it had skipped required toxicity testing, but by then users had already re-uploaded the untested weights to GitHub and Hugging Face under unofficial accounts. Nobody downloading those copies could verify they matched what Microsoft actually intended to ship.
Malicious LoRA and PEFT fine-tuning adaptersMust-have
LoRA and PEFT exist because full fine-tuning is expensive. Instead of retraining billions of parameters, you train a small adapter and merge it onto a base model at inference time. That efficiency is also the attack surface: a malicious LoRA adapter can be distributed independently of the base model, merged in by someone who never inspected it, and the combined model inherits whatever the adapter was designed to do. Model-merge services that combine adapters from multiple sources compound the problem, because nobody re-validates the merged artifact as a whole.
Weak model provenance and model cards
Model cards were supposed to solve documentation and transparency. In practice, a model card is a text file someone wrote. It carries no cryptographic guarantee that the weights you downloaded match what the card describes, or that the card is even accurate. Compare that to a signed software package, where a broken signature fails the build outright. Most model distribution today has no equivalent gate.
Compromised model repositories and registries
Hugging Face and PyPI both function as high-trust distribution points, and both have carried real malicious payloads. Researchers found a fake Hugging Face model masquerading as an OpenAI release rack up 244,000 downloads before it was caught, and separate research from JFrog and ReversingLabs has identified pickle-format models on Hugging Face carrying reverse-shell payloads designed to evade the platform’s own scanning safeguards. None of this requires you to have trained anything yourself. It just requires you to have typed pip install or clicked download.
Hugging Face’s own Safetensors conversion bot exists because format-conversion services, which turn an unsafe pickle-format model into the safer Safetensors format, have themselves become a vector for injecting malicious code during the conversion step. HiddenLayer researchers showed they could hijack the bot’s own token and submit malicious pull requests to any public repository on the platform, including models from Google and Microsoft that had already accumulated more than 16 million downloads in a single month. The service meant to fix a security problem became one itself.
Emerging 2026 supply chain risks
Two attack surfaces show up constantly in AI security research right now but rarely get a full section in existing writeups. Both are underserved enough to be worth calling out on their own.
MCP and agentic tool-chain risk
The Model Context Protocol, introduced by Anthropic in November 2024, lets an LLM call external tools and services directly. That’s the entire point of it, and it’s also a new supply chain: every MCP server your model can reach is a third-party dependency your model implicitly trusts.
In 2026, researchers at OX Security disclosed an architectural flaw in MCP’s official SDKs across Python, TypeScript, Java, and Rust that allows arbitrary command execution through the protocol’s STDIO transport, affecting an estimated 200,000 server instances. Anthropic’s response was that the behavior was expected, not a bug to patch. Static model scanning does nothing here, because the model itself was never touched.
Another threat to LLM supply chains in this area are shadow MCPs. These are MCPs used by employees in the workplace without the knowledge or approval of their employer. Unapproved MCPs can pose threats through data leakage or the injection of malicious code.
RAG pipeline and vector database poisoning
Retrieval-augmented generation pulls context from a vector database at query time and feeds it into the prompt.
If an attacker can get documents into that store, whether through a scraped web source, a shared knowledge base, or a public wiki your pipeline indexes, they can poison what the model retrieves without ever touching the model or its training data.
The model behaves exactly as designed. It’s just been handed a lie and asked to summarize it. Pinecone, Weaviate, and Chroma-backed pipelines all inherit this risk equally; the vulnerability lives in what goes into the index, not which vector database you picked.
Examples of real-world LLM supply chain security incidents
The LiteLLM PyPI compromise (March 2026). LiteLLM, a proxy gateway that sits between applications and LLM providers and gets downloaded roughly 3.4 million times a day, had its PyPI package compromised after attackers exfiltrated a publish token through a poisoned, unpinned Trivy security-scanner action in LiteLLM’s own CI/CD pipeline.
The attackers pushed two malicious versions, one that ran a payload on import and one that triggered on every Python interpreter startup with no import required, then attempted to exfiltrate credentials and establish persistence. PyPI pulled the packages within about three hours, but at that download volume, three hours is a lot of exposure.
The lesson isn’t about the proxy itself. It’s that the security scanner protecting your build pipeline is itself a supply chain dependency, and an unpinned one at that.
PoisonGPT. Already covered above, but worth restating as an incident rather than a technique: this wasn’t a theoretical attack. It was a working, publicly documented demonstration that a poisoned model can clear every standard benchmark while lying about specific facts on demand.
Infrastructure-level exploits. Oligo Security’s ShadowRay disclosure identified five separate vulnerabilities in the Ray framework used to orchestrate distributed ML workloads, one of which remains disputed and unpatched. Separately, Trail of Bits’ LeftoverLocals research (CVE-2023-4969) showed that GPU local memory could leak data across processes on shared infrastructure, an entirely different failure mode from anything happening at the model or dataset level.
How to secure your LLM supply chain
There’s no single control that closes this gap. It’s a defense-in-depth problem, and the mitigations map roughly one-to-one onto the risk categories above.
| Risk category | Mitigation | Why it matters |
|---|---|---|
| Untrusted pre-trained models | Vendor and source vetting; only pull from providers with a documented security process | Reduces exposure to backdoored or tampered uploads before they reach your pipeline |
| Data poisoning | Dataset validation and provenance checks on training/fine-tuning data | Poisoned data is nearly impossible to detect after the fact; catching it at ingestion is far cheaper |
| Model/weight tampering | Integrity verification via cryptographic hashes or signed model artifacts | Turns “trust the filename” into an actual verifiable check |
| Component visibility | Software Bill of Materials (SBOM), extended to cover models and datasets (an AI-BOM) | You can’t respond to a disclosed vulnerability in a component you don’t know you’re using |
| Unknown behavioral risk | AI red teaming and adversarial testing against your actual deployed configuration | Static scans miss behavior that only appears under specific triggers |
| Malicious LoRA/adapter merges | Re-validate merged models as a single artifact, not two independently-trusted pieces | A clean base model plus a malicious adapter is still a malicious model |
| Drift and silent changes | Ongoing anomaly and behavioral monitoring post-deployment | Providers push silent updates; a one-time review doesn’t stay valid |
| Licensing and data-use ambiguity | Licensing governance and review before adopting a model or dataset | Unclear terms turn into legal exposure long after production |
AI red teaming deserves particular emphasis because it’s the one control on this list that targets behavior directly instead of provenance. Everything else in the table verifies where a component came from. Red teaming actually tries to break the thing you shipped.
None of this is a checklist you run once. Providers update models silently, fine-tuning degrades safety behavior over time, and a component that was clean in January can be compromised by June. The LiteLLM incident above went from a poisoned CI/CD action to malicious packages on PyPI in a single build cycle. Treat supply chain review as continuous, the same way you’d treat dependency scanning in a normal software pipeline.
LLM supply chain security FAQ
What is OWASP LLM03:2025 Supply Chain?
It’s the third risk category in OWASP’s Top 10 for LLM Applications, covering vulnerabilities introduced by third-party components used to build, train, deploy, or maintain an LLM system: pre-trained models, adapters, datasets, model repositories, and the infrastructure around them.
Does this apply if we only use third-party LLM APIs and don’t train our own models?
Yes. If you’re calling a hosted model through middleware, a proxy gateway, or an orchestration framework, all of that tooling is part of your supply chain. The LiteLLM incident above is a direct example: teams that never trained a model of their own were still exposed through a compromised CI/CD action sitting several layers upstream of their actual API calls.
How is an LLM supply chain attack different from a traditional software supply chain attack?
Traditional supply chain security assumes you can inspect the artifact: read the code, diff a version, verify a hash. LLM weights aren’t readable in any meaningful sense, and a tampered or poisoned model can pass every standard check while still behaving maliciously under specific conditions.
How can you tell if your AI pipeline was affected by a supply chain attack?
Start by mapping every third-party component in your pipeline (models, adapters, datasets, repositories, serving infrastructure, and the CI/CD tooling that builds and publishes them), then check each against known incidents and CVEs. Behavioral anomalies that don’t show up in standard benchmarks are the harder case, which is why ongoing monitoring and red teaming matter more here than in traditional software.
What’s the fastest way to reduce LLM supply chain risk?
Start with visibility: know exactly which models, adapters, datasets, and repositories you depend on. You can’t secure a component you don’t know exists. After that, vendor vetting and integrity verification on new components give the best return for the least effort.
Are open-source LLM tools more vulnerable to supply chain attacks?
Not inherently, but open distribution does mean lower barriers to publishing a tampered model under a convincing name, and less consistent vetting than a commercial vendor with a formal security process. The tradeoff is transparency: open weights can at least be inspected and re-hosted by third parties, which closed APIs don’t allow.
Map your own deployment against the five components above before your next model or adapter upgrade. It’s a couple of hours of work, and it’s the cheapest place to find a gap before something else finds it for you. OWASP’s full Top 10 for LLM Applications is worth reading end to end if supply chain is only one of several risk categories you’re responsible for.
Agent connectors carry the same supply chain risk, which the agentic AI security checklist treats as a dependency review problem.