Generative AI is everywhere. Grok is busy offending Twitter users. Microsoft is pushing Copilot hard. And Google apps are now tightly integrated with Gemini.
Google’s AI can do all sorts of things for you, even if you’re a hacker. At the Black Hat security conference in Las Vegas, a team of researchers revealed how Gemini can be weaponized via Targeted Promptware Attacks—malware that subverts Gemini through its input prompts.
What Is a Promptware Attack?
A promptware attack manipulates a large language model (LLM) with input that makes it do the attacker’s bidding. The result is nothing short of magic.
“Traditional cyberattacks target memory corruption,” said infosec researcher Ben Nassi. “But now the most vulnerable component is the LLM. Promptware is engineered to trigger a malicious activity. It behaves as malware, exploiting the LLM.
“Despite the rise of promptware variants,” he continued, “most of you are not familiar with it, or don’t consider it a critical risk. Why don’t you? It’s due to a few misconceptions.”
Nassi noted that many security researchers assume that subverting LLMs with promptware requires an attacker with serious expertise, massive GPU power, or both. “These presumptions were true for classic adversarial attacks,” he said. “They do not hold water for LLM attacks.”