Listen to this Post
A Dangerous AI Supply Chain Attack Disguised as an OpenAI Privacy Tool
The rapid growth of artificial intelligence platforms has created a new attack surface for cybercriminals, and one recent incident on Hugging Face proves just how dangerous that ecosystem can become. Security researchers uncovered a malicious repository masquerading as an official OpenAI project called “Privacy Filter,” tricking users into downloading information-stealing malware hidden inside what appeared to be a legitimate AI tool.
The fake repository managed to rise to the top of Hugging Face’s trending list and reportedly accumulated more than 244,000 downloads before it was finally removed. The scale of the incident highlights a growing problem in the AI industry: attackers are now weaponizing trust in open-source AI communities.
Unlike traditional malware campaigns that rely on suspicious email attachments or cracked software, this operation specifically targeted developers, researchers, and AI enthusiasts who believed they were downloading a useful machine learning resource. The campaign demonstrates how cybercriminals are adapting their techniques to exploit the booming AI ecosystem.
Researchers Discover the Fake “Privacy Filter” Project
The malicious repository was discovered by security researchers at HiddenLayer on May 7. The fake project was named “Open-OSS/privacy-filter,” intentionally designed to imitate OpenAI’s legitimate Privacy Filter release.
According to researchers, the attackers copied the original project’s model card almost word-for-word, making the repository appear highly convincing to unsuspecting users. At first glance, the files looked like standard AI-related resources commonly shared on Hugging Face.
However, hidden within the repository was a malicious Python script named “loader.py.” While the script displayed harmless-looking AI code to avoid suspicion, its real purpose was far more dangerous.
The malware silently disabled SSL verification, decoded a hidden Base64 URL, and connected to an external server to retrieve additional malicious instructions. Those instructions eventually launched a PowerShell command completely invisible to the victim.
This attack chain allowed the malware to quietly install itself without drawing attention from the user.
Multi-Stage Infection Process Designed for Stealth
The infection process used several layers of execution designed to evade detection and analysis.
After the PowerShell command executed, the malware downloaded a batch file called “start.bat.” This secondary script escalated system privileges and downloaded the final payload known as “sefirah.”
Researchers identified “sefirah” as a Rust-based infostealer specifically designed to target Windows users.
The malware then added itself to Microsoft Defender exclusion lists, making it harder for antivirus software to detect or remove it. Once persistence was established, the infostealer began harvesting large amounts of sensitive data from the compromised system.
The stolen information included browser cookies, saved passwords, encryption keys, session tokens, browsing histories, and other highly sensitive user data from Chromium-based and Gecko-based browsers.
Discord credentials and tokens were also targeted, alongside cryptocurrency wallets, wallet browser extensions, SSH credentials, VPN configurations, FileZilla data, and locally stored wallet seed phrases.
The malware additionally captured system information and even took screenshots across multiple monitors connected to the infected computer.
After collecting the data, the malware compressed everything and exfiltrated it to a remote command-and-control server.
Advanced Anti-Analysis Features Increased the Threat
One of the most concerning aspects of the campaign was the malware’s sophisticated anti-analysis functionality.
HiddenLayer researchers reported that the malware included checks for virtual machines, sandboxes, debugging environments, and common malware analysis tools. These techniques are typically seen in advanced malware operations rather than low-level cybercrime campaigns.
By detecting research environments, the malware could avoid execution when security analysts attempted to inspect it. This significantly complicated investigation efforts and helped the operation remain active longer.
The use of Rust as the malware’s programming language is also notable. Rust-based malware has become increasingly popular among threat actors because it offers strong performance, cross-platform compatibility, and makes reverse engineering more difficult.
Download Numbers May Have Been Artificially Inflated
Although Hugging Face reported massive engagement numbers associated with the repository, researchers believe much of that activity may have been fake.
The malicious project reportedly received hundreds of likes, but investigators noticed that most of the accounts interacting with it appeared automated or artificially generated.
Similarly, the 244,000 download count may not accurately reflect the number of real victims. Attackers may have inflated those numbers intentionally to make the repository appear more trustworthy and popular.
This tactic exploits human psychology. Users often assume highly downloaded or trending projects are legitimate because large numbers create a false sense of credibility.
The attackers effectively manipulated social proof to boost the repository’s visibility and lure more victims.
Links to Other Malware Campaigns
While investigating the repository, HiddenLayer researchers discovered connections to additional malicious infrastructure.
Some repositories appeared to use the same loader mechanisms, suggesting this was not an isolated incident. Researchers also found overlaps with an npm typosquatting campaign that distributed the WinOS 4.0 implant.
This indicates that the attackers may be operating across multiple software ecosystems simultaneously, targeting developers through AI platforms, JavaScript package managers, and potentially other open-source distribution channels.
The convergence of these attacks shows how software supply chain threats are evolving rapidly.
Cybercriminals are no longer focusing only on enterprises. They are now directly targeting developers and researchers who serve as gateways into larger organizations and ecosystems.
Hugging Face Faces Ongoing Security Challenges
This is not the first time malicious content has appeared on Hugging Face.
Threat actors have repeatedly abused the platform to distribute harmful AI models and malicious code disguised as legitimate machine learning tools. Although Hugging Face has implemented security controls, the open nature of the platform makes moderation extremely difficult.
AI repositories frequently contain executable code, model weights, scripts, dependencies, and configuration files. This creates numerous opportunities for attackers to embed malicious payloads without immediate detection.
As AI adoption accelerates globally, platforms hosting open-source models are becoming increasingly attractive targets for cybercriminals.
The incident also raises broader concerns about trust in the AI supply chain. Many users assume that repositories trending on popular platforms are safe, especially when they appear connected to well-known organizations like OpenAI.
This attack demonstrates how dangerous that assumption can become.
What Undercode Say:
The Hugging Face malware incident represents one of the clearest examples yet of how AI ecosystems are becoming prime territory for cybercrime operations. What makes this campaign especially dangerous is not just the malware itself, but the psychological manipulation behind it.
Attackers understood exactly how developers behave inside AI communities. Most users browsing Hugging Face are searching for useful tools, pre-trained models, or productivity-enhancing projects. Trust is built quickly in those environments because collaboration and open sharing are core principles of the AI world.
By impersonating an OpenAI project, the attackers weaponized brand reputation. Users tend to lower their guard when they recognize familiar names connected to major AI companies. The fake repository exploited that instinct perfectly.
The operation also demonstrates how software supply chain attacks are evolving beyond traditional package managers like npm or PyPI. AI repositories now function almost like app stores for machine learning models, which means attackers see enormous opportunity there.
Another critical detail is the use of social engineering through popularity metrics. Trending rankings, fake likes, and inflated download counts create artificial legitimacy. This mirrors tactics commonly seen on social media platforms where bots manufacture credibility to manipulate human perception.
The malware’s technical sophistication is also significant. The use of anti-analysis checks, PowerShell execution chains, Windows Defender exclusions, and Rust-based payloads indicates that this was not a low-effort attack. The operators clearly invested time into making the malware resilient and stealthy.
The Rust programming language is becoming increasingly common in modern malware campaigns. Security researchers have observed a growing trend where attackers prefer Rust because it produces efficient binaries and complicates reverse engineering efforts. This trend will likely continue throughout the next several years.
The targeting of cryptocurrency wallets and browser sessions further reflects modern cybercrime priorities. Browser-stored credentials have become one of the most valuable assets for attackers because they often provide access to email accounts, financial services, cloud dashboards, and authentication sessions without requiring passwords.
The inclusion of Discord token theft is another important indicator. Discord has evolved into a major communication platform for developers, gamers, crypto communities, and even corporate teams. Stolen Discord tokens can lead to secondary attacks, social engineering campaigns, or malware propagation inside trusted communities.
The incident also exposes a growing weakness in AI governance. Open-source AI platforms currently struggle to balance accessibility with security. Manual moderation cannot realistically keep pace with the massive number of repositories uploaded daily.
Automated scanning solutions may help, but attackers are adapting quickly. Obfuscation methods, staged payloads, encoded URLs, and delayed execution techniques can bypass many detection systems.
This creates a future where AI security increasingly depends on behavioral analysis rather than simple static scanning.
Developers and researchers must now treat AI repositories with the same caution historically reserved for suspicious software downloads. Running unknown scripts locally without inspection is becoming extremely risky.
Organizations should also rethink how employees interact with open-source AI ecosystems. Sandboxed environments, isolated virtual machines, restricted execution policies, and zero-trust workflows may become essential protections in AI development pipelines.
Another overlooked issue is how quickly fake repositories can spread before moderation occurs. Even short-lived malicious projects can infect thousands of systems if they reach trending pages or social media visibility.
The broader cybersecurity industry is likely to pay much closer attention to AI model repositories after this incident. Similar attacks will almost certainly appear on other AI hosting services in the future.
There is also a reputational challenge for AI platforms themselves. If users begin associating open-source AI repositories with malware risks, trust across the ecosystem could weaken substantially.
The incident reinforces a fundamental cybersecurity truth: attackers always follow user behavior. As millions of developers shift toward AI tools and repositories, cybercriminals naturally move there as well.
This is not merely a malware story. It is the beginning of a much larger AI supply chain security problem that will likely define the next era of cyber threats.
Fact Checker Results
✅ HiddenLayer researchers did identify a malicious Hugging Face repository impersonating an OpenAI-related project and distributing infostealer malware.
✅ The malware chain included PowerShell execution, Windows Defender exclusion abuse, and theft of browser credentials, Discord tokens, and cryptocurrency wallet information.
❌ The exact number of real victims remains unknown because researchers believe the repository’s download and engagement statistics may have been artificially inflated.
Prediction
🔮 AI-focused malware campaigns targeting Hugging Face, GitHub, and model-sharing ecosystems will increase dramatically over the next 12 months.
🔮 Threat actors will increasingly disguise malware as AI utilities, productivity models, and privacy-enhancing tools to exploit developer trust.
🔮 Security verification systems for AI repositories will likely evolve into mandatory automated sandbox analysis platforms rather than relying primarily on community reporting.
🕵️📝Let’s dive deep and fact‑check.
References:
Reported By: www.bleepingcomputer.com
Extra Source Hub (Possible Sources for article):
https://www.medium.com
Wikipedia
OpenAi & Undercode AI
Image Source:
Unsplash
Undercode AI DI v2
Bing
🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]
📢 Follow UndercodeNews & Stay Tuned:
𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin | 🦋BlueSky | 🐘Mastodon
