Part 6 · LLM Application Security
LLM04: Supply Chain
LLM applications inherit risk from every model, dataset, adapter, library, and service they depend on.
What it is
LLM supply chains can be compromised at any stage: training data, pre-trained models, fine-tuning adapters, conversion pipelines, and deployment platforms. Beyond conventional software flaws, model artifacts can be tampered with or replaced, and their behavior is difficult to inspect.
How the attack unfolds
A model artifact is code-adjacent. Unsafe formats can execute instructions the moment they are loaded.
Where the risk appears
- Model files: unsafe formats, substituted weights, and hidden backdoors.
- Adapters: LoRA and other fine-tuning components from untrusted authors.
- Packages: vulnerable dependencies and slopsquatting, where attackers register package names that coding assistants tend to invent.
- Pipelines: compromised conversion, build, and deployment systems.
- Terms and licensing: unclear data use or model usage rights.
Real-world case
In 2024, JFrog reported about 100 suspicious models on Hugging Face. Some used Python's pickle format to run code when loaded.
Defenses
- Vet suppliers. Review sources, terms, and security posture.
- Keep an inventory. Maintain an AI bill of materials for models, data, and components.
- Verify integrity. Check signatures and hashes, and pin versions.
- Prefer safe formats. Use formats that store data only, such as safetensors.
- Test before promotion. Evaluate third-party models for your own use case.
- Patch and monitor. Track vulnerabilities in every layer.
Common misconception
Common belief
"A popular model is a safe model."
In practice
Download counts show popularity. They do not verify integrity.
Key takeaways
- LLM applications inherit risks from models, data, and dependencies.
- Provenance, signatures, and safe formats reduce tampering risk.
- Popularity is not evidence that a component is safe.