Malicious AI models on Hugging Face backdoor users’ machines

JFrog's security team found around 100 AI/ML models on the Hugging Face platform that contained malicious functionality. Some of these malicious models could execute code on a victim's machine, potentially giving attackers a backdoor into their systems. Hugging Face is a company that provides a platform for communities to collaborate on and share AI/ML models, datasets, and applications. The malicious models were able to evade Hugging Face's security scanning measures like malware, pickle, and secrets scanning. One highlighted malicious PyTorch model gave capabilities to establish a reverse shell connection to a specific IP address. The model used Python's pickle module to execute arbitrary code upon loading, avoiding detection. JFrog found the same payload connecting to other IP addresses, suggesting the operators may be AI researchers rather than hackers. Their experimentation was still risky and inappropriate even if not malicious. AI/ML models can pose significant security risks that have not been properly appreciated or discussed. JFrog's findings call for increased vigilance and proactive security measures to safeguard the AI ecosystem from malicious actors.