Google DeepMind unveils Genie 4: real‑time audio‑enabled world simulations
Google DeepMind released Genie 4, a world‑model that adds real‑time audio, longer photorealistic scenes and mid‑simulation text steering.
Google DeepMind released Genie 4 as a research preview on Tuesday, touting the first world model that can generate synchronized audio, maintain photorealistic scenes for up to two minutes, and accept free‑form text prompts while a simulation is running. The upgrade signals a notable step toward interactive AI environments that could reshape gaming, film pre‑visualisation, and robotics testing.
What is Genie 4 and how does it differ from earlier models?
Genie 4 builds on the Genie 3 preview launched a little over a year ago. According to the TechCrunch report, the new model doubles the scene‑duration limit—from one minute to two—while delivering "the most photorealistic and interactive worlds yet". In addition to higher‑fidelity visuals, Genie 4 introduces real‑time audio generation (music, ambient sounds, and dialogue) that matches the visual context. The model also supports "free‑form edits" such as changing weather, camera angle, or inserting new objects and characters on the fly.
How does real‑time audio and steering work in the simulation?
During the briefing, DeepMind demonstrated the system with two YouTube videos: the first demo (https://www.youtube.com/watch?v=iYRxsYRkbic) shows a static scene that suddenly receives a textual command to add rain, prompting the model to adjust lighting, particle effects and a corresponding soundscape. The second demo (https://www.youtube.com/watch?v=D4rkQXAYO3g) walks through a longer interaction where a user types a request to "add a street musician" and the model instantly renders a character, its instrument, and a layered musical track that blends with ambient city noise.
The audio is generated in parallel with the visual frames, guided by Gemini, Google’s large language model that acts as a director. Gemini plans the sequence of steps, checks each frame for consistency, and feeds the instructions to Genie 4, which then renders both the visuals and the matching sound.
What are the announced use cases and early tester feedback?
Google lists gaming, interactive learning and robotics as the primary commercial targets. A speculative example presented in the article imagines uploading a historical painting and chatting with the depicted figures to learn about the event. Trusted testers—who receive early access to Google’s AI tools—are already exploring a different angle: rapid film prototyping. DeepMind engineer Diego Rivas said the ability to steer a simulation in real time keeps creators in a "state of flow" during storyboarding and ideation.
Rivas also mentioned that, once Gemini Omni (a multimodal LLM capable of generating video from images, audio, and text) matures, creators could turn these prototypes into finished assets. The briefing stressed that Genie 4 is still experimental; issues such as object drift over longer runs and occasional mismatches between audio and visual details remain unresolved.
How does Genie 4 integrate with Google’s Gemini models?
Genie 4 does not operate in isolation. The article explains that Gemini serves as the "director" that breaks down high‑level prompts into concrete steps, validates each frame, and synchronises audio. In a Gemini Robotics experiment, a Gemini 4 Argon model decomposed a robot‑task description into discrete actions, which Genie 4 then simulated in real time. This hybrid approach—text‑to‑code via Gemini, real‑world physics via Genie—illustrates a potential workflow for testing robot behaviours before physical deployment.
Tejas Kulkarni, research director at Google, framed the vision as "hybrid world models" where Gemini writes code or text and Genie ingests it instantly, simulating reality at multiple abstraction levels.
Where does Genie 4 fit in the broader world‑model landscape?
Genie 4 arrives shortly after Runway’s GWM Worlds 2, a playable model that also supports dynamic user input and audio generation. The article notes a wave of consolidation in the sector: AMD is acquiring Fei‑Fei Li’s World Labs, Amazon is negotiating to buy Decart, and Rivian has bought Third Dimension for in‑house self‑driving simulation. Google’s release is positioned as a move to stay ahead of this competitive push, especially by coupling high‑fidelity visuals with synchronized sound and a tighter Gemini integration.
How can Grafy users experiment with similar capabilities?
While Grafy does not yet host a world‑model that matches Genie 4’s real‑time audio‑visual loop, creators can approximate parts of the workflow using existing apps:
- Film Studio (/film) lets you assemble AI‑generated video clips, add narration, and layer soundtracks. You could prototype a storyboard by generating short scenes with the Creator (/creator) app, then stitch them together in Film Studio, mirroring the rapid ideation loop described by Rivas.
- Game Studio (/game) enables you to turn a brief into a playable 2D/3D game. By feeding text prompts that describe environment changes, you can experiment with dynamic world alterations, albeit without the continuous audio sync of Genie 4.
- For research‑oriented exploration, the Reader (/reader) app can ingest the TechCrunch article and related papers, letting you query details about Genie 4’s architecture and Gemini integration.
- Finally, the Chat (/chat) interface lets you converse with multiple LLMs side‑by‑side, which can help you draft prompts or scripts that you later feed into Creator or Film Studio.
By combining these tools, Grafy users can simulate a miniature version of Genie 4’s pipeline: generate visuals, add audio, and iterate with text prompts—all within a single workspace.
Where can you find the original announcement?
The full briefing and technical details are available in the TechCrunch article: https://techcrunch.com/2026/10/27/googles-genie-4-world-model-adds-audio-more-realistic-worlds-and-real-time-steering/.
Frequently asked questions
How long can Genie 4 keep a scene stable?
Genie 4 can maintain a coherent visual scene for up to two minutes, which is double the duration supported by its predecessor, Genie 3.
Does Genie 4 generate speech that matches character lip‑sync?
The model produces interactive dialogue audio that aligns with the simulated environment, but the article notes that perfect lip‑sync and text legibility are still challenges the team is working to solve.
When will Genie 4 be available beyond trusted testers?
Google plans to roll Genie 4 out to Google AI Ultra subscribers in Project Genie and to early‑access partners of the Gemini API and Google AI Studio in the weeks following the research preview launch.
More news
Liquid AI launches LFM3: Six open‑weight 2B‑class multimodal models
Liquid AI unveiled LFM3, six open‑weight 2 billion‑parameter multimodal models for text.
OpenAI and Anthropic Claim AI Models Refute the Exponential Time Hypothesis
OpenAI and Anthropic released algorithms that allegedly solve ETH‑class problems in sub‑exponential time.
AgentOrch Benchmark Shows Global Orchestration Closing Gap to Step‑by‑Step AI Agents
The new AgentOrch benchmark compares global orchestration with step‑by‑step LLM agents.