As advanced generative AI systems, such as OpenAI’s ChatGPT and Google’s Gemini, continue to evolve, startups and tech companies are increasingly utilizing them for routine tasks like scheduling appointments and making purchases. Nevertheless, as these AI assistants are integrated into a growing number of applications with increased autonomy, there is a rising concern about potential exploitation by hackers.
Researchers from Cornell University, Technion-Israel Institute of Technology, and software developer Intuit have developed the first generative AI worm, which can steal your confidential data or deploy malware and spread from one system to another. Called ‘Morris II,’ after the first computer worm that caused a global nuisance on the early Internet, the new worm can poison the AI systems that power your email assistants and trick them into revealing sensitive information like credit card numbers and social security numbers. It can even use the compromised assistants to harvest personal information from other users and send that to other email addresses.
In a test environment, the researchers demonstrated that this “zero-click worm” can infect ChatGPT 4.0, Gemini Pro, and an open-source AI model called LLaVA. The worm injects itself into these assistants by sending malicious prompts via text and images, causing them to reveal private information or to send spam emails in a test scenario. The team got the worm to spread by creating a text-based self-replicating prompt and hiding it inside an image file.
The worm exploits vulnerabilities in generative AI models that allow them to process data from outside their own systems, including text, images, and other objects. The researchers created an adversarial prompt to infect a generative AI-powered email assistant and sent it to their test GPT-4 or Gemini Pro. When the assistant uses this text prompt to generate an answer, it “poisons” the database by using a feature known as retrieval-augmented generation to pull in extra data from outside its system. The team says the reply containing this data later infects new hosts when it is used to send an email to a new victim.
Nassi and his team expect to see generative AI worms become more common as firms build GenAI networks into their cars, phones, and home systems. He says they will likely appear in two to three years when generative AI will be capable of completing more tasks for people and will be able to interact with more devices simultaneously. But he stresses that avoiding these threats is possible by designing more secure algorithms and protecting against malicious input from cyber criminals. The authors of this study say that it’s essential for software developers to make their generative AI systems “resilient and robust against malicious input,” which includes detecting when they are being attacked. This can be done using methods like heuristics and machine learning or through a method called supervised learning that requires human supervision. Until these defenses are in place, this attack will remain a realistic threat.
