Security researchers conducted an experiment using ChatGPT as part of a method called “Shadow Leak” to extract sensitive information from Gmail accounts without user awareness. This exploit involved a vulnerability related to the design of AI agents, which are capable of performing tasks autonomously once granted access to personal data such as emails and calendars. Although OpenAI has addressed the issue, this incident illustrates potential risks associated with the use of AI technology.
The Shadow Leak study was released by the security firm Radware and highlighted a technique known as prompt injection, which manipulates an AI agent to act on behalf of the attacker. Such vulnerabilities present challenges for cybersecurity, as they can be utilized for various malicious purposes, including manipulating academic peer reviews and executing scams. Users often remain unaware of these breaches since the deceptive instructions can be embedded in ways that are not easily visible, such as using white text on a white background.
In the specific incident, the researchers utilized OpenAI’s Deep Research tool, embedded within ChatGPT. They inserted a prompt injection into an email within a Gmail inbox that the AI had access to. When the user interacted with the tool, the AI executed the hidden commands, which involved searching for sensitive information and potentially relaying it to the attackers.
The process of exploiting the AI’s capabilities involved numerous challenges, with the researchers describing it as a series of unsuccessful attempts followed by eventual success. Notably, the method directly leaked data from OpenAI’s cloud infrastructure, rendering it impervious to conventional cyber defense mechanisms.
Radware emphasized that their findings serve as a proof of concept, indicating that other applications connected to Deep Research, such as Outlook, Google Drive, and Dropbox, could be similarly susceptible to exploitation. OpenAI has since addressed the vulnerability identified by Radware in June.
Source: https://www.theverge.com/news/781746/chatgpt-gmail-shadow-leak