LLM Hallucinated Packages:
How AI Suggestions Become Malware Delivery Vectors.

The Hallucination Problem in Package Suggestions

Large language models generate statistically plausible completions — they don't query registries. When asked to write code that requires a specific library, an LLM may suggest a package name that follows the naming conventions of the ecosystem but doesn't actually exist in the registry.

The LLM is confident. It generates the import statement, the usage code, and sometimes even fake documentation for the suggested package. Nothing in the output signals that the package doesn't exist. A developer who trusts the AI output runs pip install or npm install without first verifying the package exists in the registry.

Attack Mechanics

The attack chain is straightforward:

1. Discovery — attacker identifies commonly hallucinated package names by querying LLMs with a variety of coding tasks and recording suggestions that resolve to non-existent packages
2. Registration — attacker registers the hallucinated package names on target registries (npm, PyPI, etc.) with malicious payloads
3. Wait — developers using AI coding assistants receive suggestions for these packages; some follow the suggestion without verifying registry existence
4. Compromise — the package installs and executes its payload on developer machines and CI environments

The attack requires no active exploitation, no social engineering beyond the AI's own output, and scales passively across all users of the target LLM.

Research Findings

Academic research on LLM package hallucination has produced consistent findings:

• A 2023 study (Lanyado et al.) found that ChatGPT and other models hallucinated package names in approximately 5.2% of programming queries, with some task categories seeing rates above 20%
• The same hallucinated names appeared consistently across multiple independent queries — suggesting the hallucination is reproducible and therefore exploitable
• Researchers who registered a subset of the hallucinated package names recorded thousands of installs within days, with no promotion or advertising
• Newer models with larger training datasets and more up-to-date knowledge have lower hallucination rates but do not eliminate the problem

Vibe Coding as Attack Amplifier

Package hallucination attacks existed before vibe coding became common, but vibe coding dramatically amplifies the risk:

• Trust without verification — vibe coders accept AI suggestions as expert recommendations; they're less likely to independently verify package existence before adding to requirements.txt
• Speed pressure — the value proposition of vibe coding is speed; stopping to verify every package suggestion slows down the workflow
• Missing context — a developer who understands the domain knows which packages exist; a vibe coder relying entirely on AI doesn't have that context
• Agentic amplification — autonomous AI coding agents can add packages to dependency files without human review, converting hallucination directly into installation

Real-World Examples

Documented hallucinated package registrations that attracted organic installs:

# LLM suggested:
from huggingface_transformers_utils import ModelTokenizer
# This package doesn't exist in PyPI
# The real package is: transformers (from huggingface)
# An attacker can register huggingface_transformers_utils
# with a malicious postinstall hook

# LLM suggested (npm):
import { parseEnv } from 'dotenv-parser-extended'
# Real package: dotenv
# Hallucinated variant registered by attacker

These aren't cherry-picked examples — researchers can generate lists of hundreds of consistently hallucinated package names by querying LLMs with standard programming tasks.

Prevention and Verification

• Verify before installing — for every new package AI suggests, check it exists in the official registry, has a real source repository, and has a meaningful download history
• Private registry allowlisting — maintain a vetted list of allowed packages; anything not on the list requires explicit approval, regardless of who suggested it (human or AI)
• Lock file enforcement — lock files prevent new packages from being silently added; any new package requires a lock file change that goes through code review
• Package age gates — do not install packages registered in the last 30 days without explicit security review; attacker-registered hallucinated packages are always new
• AI tool configuration — some AI coding tools allow configuring an approved package list that the tool uses to ground its suggestions in reality
• Scan before merge — automated supply chain scanning should flag any new dependency that exhibits signals of being a hallucinated package (new, low downloads, no source repo)