Detecting Malware in AI Models with MetaDefender Sandbox

The idea that AI models could be abused to deliver malware is not new. As early as 2018, researchers published studies such as Model-Reuse Attacks on Deep Learning Systems showing that pretrained models from untrusted sources could be manipulated to behave maliciously.

At first, this seemed like a thought experiment—a “what if” scenario debated in academic circles. Many assumed it would remain too niche to matter. But history shows that every widely adopted technology becomes a target, and AI was no exception.

The shift from theory to practice happened when real examples of malicious AI models surfaced demonstrating that Pickle-based formats like PyTorch can embed not just model weights but executable code.

A striking case was star23/baller13, a model uploaded to Hugging Face in early January 2024. It contained a reverse shell hidden inside a PyTorch file and loading it could give attackers remote access while still allowing the model to function as a valid AI model. This highlights that security researchers were actively testing proof-of-concepts at the end of 2023 and into 2024.

Attackers also began exploiting the trust built into software ecosystems. In May 2025, fake PyPI packages such as aliyun-ai-labs-snippets-sdk and ai-labs-snippets-sdk mimicked Alibaba’s AI brand to trick developers. Although they were live for less than 24 hours, these packages were downloaded around 1,600 times, demonstrating how quickly poisoned AI components can infiltrate the supply chain.

For security leaders, this represents a double exposure:

• Operational disruption if compromised models poison AI-powered business tools.
• Regulatory and compliance risk if data exfiltration occurs via trusted-but-trojanized components.

Once attackers saw the potential, they began experimenting with ways to make malicious models even harder to detect. A security researcher known as coldwaterq demonstrated how “Stacked Pickle” nature could be abused to hide malicious code.

By injecting malicious instructions between multiple layers of Pickle objects, attackers could bury their payload, so it looked harmless to traditional scanners. When the model was loaded, the hidden code would slowly unpack step by step, revealing its true purpose.

MetaDefender Sandbox integrates Fickling with custom OPSWAT parsers to break down Pickle files into their components. This allows defenders to:

• Inspect unusual imports, unsafe function calls, and suspicious objects.
• Identify functions that should never appear in a normal AI model (e.g., network communications, encryption routines).
• Generate structured reports for security teams and SOC workflows.

Beyond parsing, the sandbox disassembles serialized objects and traces their instructions. For example, Pickle’s REDUCE opcode—which can execute arbitrary functions during unpickling—is carefully inspected. Attackers often abuse REDUCE to launch hidden payloads, and the sandbox flags any anomalous usage.

Stacked Pickle can be utilized as a trick to hide malicious behavior. By nesting multiple Pickle objects and injecting the payload across layers then combined with compression or encoding. Each layer looks benign on its own, therefore many scanners and quick inspections miss the malicious payload.

MetaDefender Sandbox peels those layers one at a time: it parses each Pickle object, decodes or decompresses encoded segments, and follows the execution chain to reconstruct the full payload. By replaying the unpacking sequence in a controlled analysis flow, the sandbox exposes the hidden logic without running the code in a production environment.

For CISOs, the outcome is clear: hidden threats are surfaced before poisoned models reach your AI pipelines.