Researchers have developed a new agent named SIMA 2, claiming it can perform a variety of complex tasks within virtual environments, autonomously solve certain challenges, and engage in conversation with users. It is designed to improve by tackling increasingly difficult tasks and learning through trial and error.
Joe Marino, a research scientist at Google DeepMind, emphasized the role that games have played in advancing agent research. He noted that even straightforward actions in games, such as lighting a lantern, can involve many steps, making task resolution intricate.
The overarching goal is to create next-generation agents capable of following instructions and performing open-ended tasks in more complex environments than standard web browsers. Google DeepMind envisions these agents eventually facilitating real-world robots. Marino stated that skills learned by SIMA 2, including environment navigation, tool usage, and human collaboration, are foundational for future robotic companions.
In contrast to earlier game-playing agents like AlphaZero and AlphaStar, which excelled in specific competitive contexts, SIMA 2 aims to learn through open-ended gameplay without predefined objectives. Instead, it responds to instructions from users, who can communicate via text chat, voice, or by drawing on the game screen. The agent processes video game pixels frame by frame to determine necessary actions.
SIMA 2 was trained using footage of humans playing eight commercial video games, including “No Man’s Sky” and “Goat Simulator 3,” along with three virtual worlds created by the company, enabling it to associate keyboard and mouse inputs with corresponding actions.
The researchers claim that when integrated with their Gemini system, SIMA 2 demonstrates enhanced capabilities in following instructions and performing complex tasks autonomously. The agent was tested in unfamiliar environments, where it successfully navigated and executed instructions generated by the latest version of the company’s world model.
Source: https://www.technologyreview.com/2025/11/13/1127921/google-deepmind-is-using-gemini-to-train-agents-inside-goat-simulator-3/