back arrow Blog
The LiteLLM Supply Chain Attack: A Complete Technical Breakdown of What Happened, Who Is Affected, and What Comes Next

The LiteLLM Supply Chain Attack: A Complete Technical Breakdown of What Happened, Who Is Affected, and What Comes Next

RECOMMENDED ARTICLES

Executive Summary

On March 24, 2026, security researcher isfinne discovered that LiteLLM version 1.82.8—the most popular open-source LLM proxy in the Python ecosystem, with approximately 97 million monthly downloads—contained credential-stealing malware published to PyPI. Within hours, version 1.82.7 was confirmed to carry a similar payload through a different injection method. The attack was traced to TeamPCP, a threat actor conducting a coordinated multi-week supply chain campaign that had previously compromised Aqua Security's Trivy scanner and Checkmarx's GitHub Actions. The entire LiteLLM package has been quarantined on PyPI. This article provides a complete technical analysis of the attack, its downstream impact, and what it signals about the security of AI infrastructure in 2026.

Timeline: From Trivy to LiteLLM

The LiteLLM compromise did not occur in isolation. It was the third known strike in a coordinated supply chain campaign that methodically escalated from security tooling to AI infrastructure:

Date Target Method Impact
March 19, 2026 Trivy (Aqua Security) Spoofed commits to GitHub Actions using impersonated identities (rauchg, DmitriyLewen); injected credential stealers into trivy-action and setup-trivy CI/CD secrets exfiltrated from organizations running Trivy in pipelines; 75 tags hijacked; 44 Aqua Security repos defaced
March 21, 2026 Checkmarx AST GitHub Actions Identical credential stealer pattern injected into Checkmarx's AST GitHub Action Additional CI/CD pipeline secrets harvested; encrypted and sent to typosquat domain
March 24, 2026 LiteLLM Used CI/CD credentials harvested from Trivy attack to publish malicious versions to PyPI Full credential theft on every machine with v1.82.7 or v1.82.8 installed; Kubernetes lateral movement attempted

The progression is deliberate and strategic: compromise security scanners first (tools that run with elevated permissions in CI/CD pipelines), harvest credentials, then use those credentials to poison downstream infrastructure. By attacking Trivy—a tool that many organizations trust implicitly and run with broad access—TeamPCP obtained the PyPI publishing token and GitHub personal access token needed to publish malicious LiteLLM releases.

Technical Analysis: How the Malware Worked

Version 1.82.8: The .pth File Technique

The primary payload in version 1.82.8 used a technique that is novel in the context of PyPI attacks: a .pth file named litellm_init.pth (34,628 bytes) placed in the Python site-packages directory.

Python's site module processes .pth files during interpreter startup. Lines that begin with import are executed as Python code. This means the malicious payload ran on every Python process started on the machine—not just processes that imported LiteLLM. A data scientist running a Jupyter notebook, a DevOps engineer running an Ansible playbook, a developer running unit tests—all triggered the malware if LiteLLM 1.82.8 was installed in the same Python environment.

The payload was double base64-encoded to evade static analysis tools that scan for common malware patterns.

Version 1.82.7: The Import Hook

Version 1.82.7 used a more conventional approach: an obfuscated payload embedded directly in proxy/proxy_server.py. This executed when any code imported the LiteLLM proxy module, including automated test suites, CI/CD pipelines, and production deployments.

The Three-Stage Payload

Both versions executed the same core payload:

Stage 1: Collection

The malware systematically harvested credentials from over a dozen categories:

  • SSH keys and configurations (~/.ssh/)
  • AWS credentials (~/.aws/credentials, ~/.aws/config)
  • GCP service account keys (~/.config/gcloud/)
  • Azure credentials (~/.azure/)
  • Kubernetes configurations (~/.kube/config)
  • Docker credentials (~/.docker/config.json)
  • Git configurations (~/.gitconfig, credential helpers)
  • Shell history (~/.bash_history, ~/.zsh_history)—which often contains credentials passed as command arguments
  • Cryptocurrency wallets
  • SSL/TLS private keys
  • Environment variables—all of them, including OPENAI_API_KEY, ANTHROPIC_API_KEY, AWS_SECRET_ACCESS_KEY, database connection strings, and every other secret stored in the process environment
  • .env files from the current directory and parent directories
  • Cloud metadata endpoints—attempting to reach instance metadata services for additional cloud credentials

Stage 2: Encryption and Exfiltration

Collected data was compressed into tpcp.tar.gz, then encrypted using a layered scheme:

  1. A random AES-256 session key was generated
  2. Data was encrypted using AES-256-CBC with PBKDF2 key derivation via openssl
  3. The session key was encrypted with a hardcoded 4096-bit RSA public key
  4. The encrypted archive was POSTed to https://models.litellm.cloud/

The exfiltration domain was carefully chosen: models.litellm.cloud visually resembles legitimate LiteLLM infrastructure but is entirely controlled by the attacker. Casual inspection of network logs might not flag it as suspicious.

Stage 3: Lateral Movement

If the malware detected Kubernetes tokens on the system, it escalated its attack:

  • Read cluster secrets via the Kubernetes API
  • Attempted to create privileged pods
  • Installed persistent backdoors at ~/.config/sysmon/sysmon.py

This stage transforms a supply chain compromise into a persistent infrastructure breach. Even after removing the malicious LiteLLM version, organizations with Kubernetes exposure may have ongoing unauthorized access.

The Accidental Fork Bomb

Ironically, the malware contained a critical implementation flaw. The .pth file spawned child processes via subprocess, which in turn triggered the same .pth file when the child Python process initialized. This created an exponential fork bomb that crashed affected machines—which is actually how some users first noticed the compromise. Developer dot_treo reported a process consuming all system RAM, which led to the investigation that uncovered the full attack.

Blast Radius: Who Is Affected

Direct Impact

Any system that installed LiteLLM 1.82.7 or 1.82.8 between their publication and PyPI quarantine should be considered fully compromised. With LiteLLM averaging 3.6 million daily downloads, the exposure window—even measured in hours—is significant.

Transitive Dependencies

LiteLLM is a dependency of numerous AI frameworks and tools. Organizations using any of the following may have pulled the compromised version as a transitive dependency:

  • DSPy—uses LiteLLM for multi-provider LLM calls
  • CrewAI—imports LiteLLM for agent orchestration
  • MLflow—integration with LiteLLM for model serving (MLflow has since pinned to ≤1.82.6)
  • Nanobot, Browser-use, Harbor—and over 600 public GitHub projects with unpinned LiteLLM dependencies

CI/CD Pipelines

Pipelines that run pip install litellm or pip install -r requirements.txt without version pinning automatically pulled the latest available version. These environments typically have access to deployment credentials, cloud provider tokens, and production database connection strings. The malware's explicit targeting of environment variables and credential files means CI/CD runners are among the highest-risk targets.

Docker Images

Docker images built without version pinning during the exposure window contain the malicious package. These images may be cached in registries and continue to be deployed after the PyPI quarantine. Organizations should audit their container image build logs and scan existing images.

The Official Response

The LiteLLM maintainer's response was notable for its directness. Key actions taken:

  • Compromised versions deleted from PyPI
  • All maintainer accounts reset
  • All keys for GitHub, Docker, CircleCI, and PyPI revoked
  • The entire package quarantined on PyPI, blocking all downloads (not just compromised versions)

The maintainer acknowledged that the root cause was a compromised Trivy installation in their CI/CD pipeline, which exposed the PyPI publishing token. The maintainer disclosed they were pinned to Trivy v0.69.3, which was among the compromised versions.

However, the community response also highlighted concerns:

  • The Trivy compromise was publicly disclosed on March 19—five days before the LiteLLM attack. Credentials should have been rotated immediately upon that disclosure.
  • GitHub issue #24512 was initially flooded with approximately 170 spam bot comments, likely from other accounts compromised in the same campaign, hampering communication during the critical response window.
  • The original issue was briefly closed as "not planned" before being reopened, creating confusion about the team's awareness of the severity.

What This Reveals About AI Infrastructure Security

The Single Point of Failure Problem

LiteLLM's value proposition is also its greatest security liability: centralizing credentials for 100+ LLM providers into a single tool. This creates a high-value target that is disproportionately rewarding for attackers. Compromising one library yields credentials for an organization's entire AI stack.

The .pth File Blind Spot

The .pth file technique represents an underappreciated attack surface in the Python ecosystem. Most supply chain scanning tools focus on setup.py, __init__.py, and entry points. The .pth mechanism is a legitimate Python feature that has been largely overlooked as an injection vector. Expect this technique to appear in future attacks.

The Cascading Compromise Pattern

TeamPCP's campaign demonstrates a sophisticated attack pattern: compromise security tools first, harvest credentials, then use those credentials to attack higher-value targets. Each stage leverages trust established in the previous stage. Organizations that ran Trivy in CI/CD pipelines trusted it with broad access—and that trust was weaponized against them.

The Open-Source Trust Paradox

LiteLLM's popularity (97 million monthly downloads) was itself a risk factor. High download counts are often used as a heuristic for trustworthiness. But popularity means a compromise affects more systems, making it a more attractive target. The same download volume that signals community trust also signals attacker reward.

The Dependency Pinning Gap

Over 600 public GitHub projects had unpinned LiteLLM dependencies. This is a systemic problem across the Python AI ecosystem, where rapid iteration is prioritized over supply chain hygiene. The LiteLLM attack is a costly lesson in why pip install litellm and pip install litellm==1.82.6 have fundamentally different risk profiles.

Immediate Actions for Affected Organizations

Detection

  1. Check all Python environments: pip show litellm | grep Version
  2. Search for the malware artifact: find / -name "litellm_init.pth" 2>/dev/null
  3. Check pip caches: find ~/.cache/pip -name "litellm*1.82.8*" -o -name "litellm*1.82.7*"
  4. Audit network logs for connections to models.litellm.cloud
  5. Search for persistence mechanisms: find / -path "*/.config/sysmon/sysmon.py" 2>/dev/null

Containment

  1. Uninstall LiteLLM from all environments
  2. Purge pip caches to prevent reinstallation from cached wheels
  3. Audit Kubernetes clusters for unauthorized pods, service accounts, or RBAC changes
  4. Isolate affected machines from production networks

Credential Rotation

If versions 1.82.7 or 1.82.8 were installed on any system, treat all credentials accessible from that system as compromised. This includes:

  • All LLM provider API keys (OpenAI, Anthropic, Google, Azure, AWS, etc.)
  • Cloud provider credentials (AWS IAM, GCP service accounts, Azure AD)
  • SSH keys
  • Database passwords
  • CI/CD tokens (GitHub PATs, Docker Hub, container registries)
  • Kubernetes service account tokens
  • Any secret stored in environment variables or credential files

Long-Term Mitigations

  • Pin all dependency versions in requirements.txt and lockfiles
  • Use hash verification (pip install --require-hashes) for critical dependencies
  • Separate CI/CD concerns: Security scanning jobs should not have access to publishing credentials
  • Adopt OIDC-based trusted publishing for PyPI instead of long-lived tokens
  • Implement egress filtering to restrict outbound connections from development and CI/CD environments
  • Monitor for .pth files in Python site-packages directories as part of endpoint security
  • Evaluate architectural alternatives that do not require centralizing all AI provider credentials in a single Python package

What Comes Next

As of this writing, the LiteLLM package remains quarantined on PyPI. The full scope of credential theft is unknown. TeamPCP's campaign appears ongoing, with security researchers tracking their activities across multiple ecosystems.

For the AI ecosystem, this is an inflection point. LiteLLM's role as the default LLM routing layer for hundreds of AI projects means its compromise has cascading effects that will take weeks to fully enumerate. Organizations that depended on it for production AI workloads need to evaluate alternatives—whether that means direct provider SDKs, alternative gateways like Bifrost or Portkey, or architectural changes that eliminate the need for a centralized credential-holding proxy entirely.

The deeper lesson is structural. Any software component that centralizes access to high-value credentials will be a priority target. The Python ecosystem's reliance on environment variables for credential configuration, combined with the .pth execution mechanism and the transitive dependency resolution model, creates an attack surface that cannot be fully mitigated by any single tool or practice. Defense in depth—pinned versions, hash verification, egress filtering, sandboxed environments, and server-side credential management—is the only viable path forward.

The age of implicit trust in open-source AI infrastructure is over.

Frequently Asked Questions

Is LiteLLM still available on PyPI?

No. As of March 24, 2026, the entire LiteLLM package has been quarantined on PyPI. This means no version is available for download—not just the compromised versions. Organizations that need LiteLLM functionality should evaluate alternatives or vendor a known-safe version internally until the situation is resolved.

How do I know if my organization was affected?

Check installed versions across all Python environments (developer machines, CI/CD runners, Docker images, staging, production) using pip show litellm | grep Version. If version 1.82.7 or 1.82.8 was installed at any point, assume full credential compromise for that machine and all systems reachable from it. Also check pip caches and Docker image layer histories, as the malicious package may have been cached even if the installed version was later downgraded.

What is the .pth file attack technique?

Python's site module processes .pth (path configuration) files in site-packages during interpreter startup. Lines beginning with import are executed as Python code. This means a malicious .pth file runs its payload on every Python process, not just processes that import the compromised package. This is a legitimate Python feature being exploited as an attack vector, and it largely evades existing supply chain scanning tools that focus on setup.py and __init__.py.

Who is TeamPCP?

TeamPCP is the threat actor behind a coordinated supply chain campaign in March 2026 that targeted security and AI infrastructure tools. They first compromised Aqua Security's Trivy scanner (March 19), then Checkmarx's AST GitHub Actions (March 21), then LiteLLM (March 24). Their technique involves spoofing commits from legitimate maintainers, deploying encrypted credential stealers, and using harvested CI/CD credentials to attack downstream targets. They publicly defaced 44 Aqua Security repositories. Security researchers are tracking their activities at multiple organizations.

Should I rotate credentials even if I was on an older LiteLLM version?

If you can confirm you were never on 1.82.7 or 1.82.8, credential rotation for the LiteLLM-specific attack is not strictly necessary. However, if you were using Trivy in your CI/CD pipeline (particularly v0.69.3 or nearby versions), your CI/CD credentials may have been separately compromised in the earlier Trivy attack, which could expose you through a different path. When in doubt, rotate.

What alternatives to LiteLLM should I evaluate?

The Hacker News discussion around this incident highlighted several alternatives: Bifrost (open-source, Go-based, with benchmarked 54x faster P99 latency), Portkey (commercial AI gateway), TensorZero (Rust-based, sub-millisecond overhead), and Cloudflare AI Gateway (managed edge service). Some developers are moving to direct provider SDK calls to eliminate the proxy layer entirely. The right choice depends on whether you need multi-model routing, caching, cost tracking, or can simplify to fewer providers.