Google DeepMind released its latest world model, Genie 4, on Tuesday in the form of a research preview. This model generates the most realistic and interactive world to date, allowing users to input prompts during the simulation process to change the scenarios. It can also produce real-time audio that matches the simulated world, including music, environmental noises, and interactive dialogues.
A few months before the release of Genie, AI, a video startup company known as Runway, just released its second world model, GWM Worlds 2 – a model that is playable, can dynamically respond to user input, and includes audio generation capabilities. At the same time, the world model industry is beginning to consolidate. Li Feifei's World Labs is being acquired by AMD, Amazon is in talks to acquire Decart, and Rivian has just acquired Third Dimension in order to integrate these modeling capabilities for use in autonomous driving tests internally.
Genie 4 is the latest move by Google to maintain a leading position in the crowded race.
Google released the predecessor product Genie 3 in a research preview form over a year ago. In addition to being much more realistic and featuring audio capabilities than its predecessor, Genie 4 also allows for free-form editing, such as real-time changes to the weather or camera angle, or adding new characters or objects. It can maintain a consistent scene for up to two minutes, which is twice the duration of its predecessor, and it offers better physical effects and characters that are more human-like.
The AI model of Google is also more closely integrated with Genie; it acts as the "director," responsible for planning the steps and checking the visuals, while Genie is in charge of generating the visual content. In a research experiment related to Gemini Robotics, the Argon model breaks down the task into steps, which are then simulated in real-time by Genie. This experiment demonstrates a possible approach: testing robot scenarios in simulation before they occur in the real world.
"In the future, we will increasingly see this kind of hybrid world model, where Gemini writes code and text, and Genie receives and simulates reality at different levels of abstraction in real time," said Tejas Kulkarni, the research director of Google, at a briefing.
Like other world models, its promoted use cases include gaming, interactive learning, and robotics. An speculative example of interactive learning is: entering a painting that depicts a historical scene and engaging in conversation with the characters depicted in the painting about the events portrayed.
However, "trusted testers" (the first group to obtain access to the Google advanced products) are pointing to a possible new use: film prototype production.
"Real-time performance is of great value in maintaining a state of flow in the creative field," said engineer Diego Rivas from Google DeepMind at a press briefing. "For example, during the conceptualization phase, the storyboarding phase—anytime you need to think quickly."
Google also provides Veo, which is its AI video generation model. It can create high-definition clips with built-in synchronized audio based on text, images, or video prompts.
Rivas added that users may ultimately rely on Gemini Omni to transform these prototypes into creative assets. However, he noted that there is still some way to go before these capabilities are fully realized.
Throughout the briefing, Rivas emphasized that Genie 4 is still in its early stages, and it is not yet clear what the end-to-end product experience will be like. The team also acknowledged that there are still gaps in Genie 4: the longer the simulation runs, the more likely there may be drifts in details and objects; they have not yet fully figured out how to ensure accurate audio and clear, readable text within the world.
Genie is currently available to trusted testers in a research preview format. In the coming weeks, this model will be made available to Google AI Ultra subscribers for Project Genie, and also to early access partners of Gemini API and Google AI Studio.
