Source: Training Agents Inside of Scalable World Models

Do AIs Dream of Diamonds? DeepMind's New Model Conquers Minecraft Through Imagination Alone

Imagine learning a skill as complex as driving or surgery, but your only study material is watching videos of others performing the task. You can't practice, you can't make mistakes, and you won't get to sit in the driver's seat or hold a scalpel until you've completely mastered the skill.

It sounds like the stuff of fantasy, but it's precisely the holy grail the field of artificial intelligence (AI) has long pursued: enabling an agent to learn complex tasks without expensive, or even dangerous, interactions with the real world.

Recently, a research team from Google DeepMind unveiled a new model named Dreamer 4, which has achieved a historic breakthrough in a seemingly impossible challenge. By solely "watching" gameplay videos of human players, without a single instance of real-time game interaction, this AI agent successfully completed the entire process of starting from scratch to mining a diamond in the complex, open world of Minecraft.

How was this accomplished? How can an AI that has never "played" the game build a mental model of the world's rules and plot a complex path to a diamond?

Learning in a 'Virtual Brain': The Magic of World Models

The secret to Dreamer 4's success lies in its ability to build and utilize an incredibly powerful and efficient "World Model." You can think of it as a high-fidelity "virtual brain" or a "physics engine" tailored specifically for Minecraft.

This "brain" can internally simulate everything that happens in the game world. When the AI agent makes a move in its "imagination"—like "take a step forward" or "swing a pickaxe"—the world model can predict, in real-time and with vivid detail, how the game screen will change.

By conducting millions of low-cost "what-if" scenarios and reinforcement learning trials within this imagination, Dreamer 4 can rapidly iterate and optimize its strategies through virtual trial-and-error, bypassing the need to spend vast amounts of time in the actual game environment.

This process, as shown in the image below, allows the agent to master complex task sequences—from chopping wood and crafting tools to digging deep underground for ore—all within the confines of its world model's imagination.

Imagination Training Inside the World Model

Caption: Dreamer 4 solves a series of complex tasks—from logging and mining to crafting tools and digging for diamonds—all through reinforcement learning within its internal world model. Source: Training Agents Inside of Scalable World Models, arXiv:2509.24527

This powerful "virtual brain" is primarily composed of two parts, both based on the Transformer architecture, working in concert:

• Causal Tokenizer: Acting like a pair of sharp eyes, it's responsible for "understanding" the game screen, compressing and distilling high-resolution pixel information into a series of key features the model can comprehend.
• Interactive Dynamics: This plays the role of the "physics engine," predicting how the world will evolve in the next moment based on the agent's actions and the environmental features "seen" by the tokenizer.

Dreamer 4's World Model Architecture

Caption: Dreamer 4's world model consists of a 'Causal Tokenizer' and an 'Interactive Dynamics' model, both using efficient Transformer architectures to predict the game world's evolution in real time. Source: Training Agents Inside of Scalable World Models, arXiv:2509.24527

The Data Doesn't Lie: Unprecedented Offline Learning Performance

So, how does an agent trained in "imagination" perform in practice? Under a purely offline learning setting, Dreamer 4 delivered astonishing results.

In the renowned "Offline Diamond Challenge," all models were trained on the same dataset of human player recordings and were forbidden from any real-time interaction with the game environment. The results were clear:

• Dreamer 4 is the first agent ever to obtain a diamond purely offline, achieving a success rate of 0.7%. This number may seem small, but it represents a qualitative leap from 0 to 1, as all previous models had a success rate of 0.
• In acquiring an "iron pickaxe," a critical intermediate step, Dreamer 4's success rate was a remarkable 29%, while the previous best model managed only 0.6%.
• Even more impressively, Dreamer 4 achieved this feat using 100 times less data than OpenAI's earlier VPT model, demonstrating exceptional data efficiency.

Success Rate Comparison in the "Offline Diamond Challenge"

Caption: In the offline challenge to acquire key items from logs to diamonds, Dreamer 4 (dark blue bars) comprehensively and significantly outperforms previous methods like Behavior Cloning (BC), Visual Language Models (VLA), and VPT. Source: Training Agents Inside of Scalable World Models, arXiv:2509.24527

Beyond Gaming: A Hyper-Realistic, Interactive Simulator

The world model of Dreamer 4 isn't just a training tool for an AI; its simulation is so high-fidelity that it can even support interaction with human players.

Researchers conducted a "counterfactual" experiment: they had a human player attempt to complete the task of "building a 3x3 wall" within Dreamer 4's world model. The results showed that Dreamer 4 could accurately predict and generate the visual changes resulting from each of the player's actions (like placing a wooden plank), ultimately allowing the player to successfully complete the task.

In contrast, previous world models like Oasis and Lucid began to rapidly degrade after just a few interactions. Their generated visuals started to show misplaced objects and structural "hallucinations," making any meaningful interaction impossible.

Human Interaction Comparison in Different World Models

Caption: In a task where a human player attempts to build a wall, Dreamer 4 (top) accurately simulates the interaction until completion. In contrast, the models from Oasis (middle) and Lucid (bottom) quickly degrade, generating erroneous and hallucinatory visuals. Source: Training Agents Inside of Scalable World Models, arXiv:2509.24527

This high-fidelity, interactive simulation capability has implications far beyond gaming. When the same technology is applied to robotics, Dreamer 4 can learn to simulate the physics of real-world objects just by watching videos, enabling humans to "teleoperate" robots in a virtual environment. This means that, in the future, the bulk of robot training could be completed in a safe and efficient virtual world, drastically reducing development costs and potential risks.

Generalization Capability on Robotic Tasks

Caption: Dreamer 4's world model can also learn the physics of real-world object interactions, supporting human control of a virtual robot in a simulated environment to complete tasks like picking, pouring, and throwing. Source: Training Agents Inside of Scalable World Models, arXiv:2509.24527

Behind the Scenes: The Engineering Magic That Made It All Possible

Dreamer 4's exceptional performance is also due to a series of clever innovations in its model and training methods.

One of the most critical is "Shortcut Forcing." Traditional methods, like Diffusion Forcing, are akin to deriving physics formulas from Newton's first law—precise but slow. Shortcut Forcing, however, allows the model to "take a shortcut," using more macroscopic principles to predict outcomes directly. This enables Dreamer 4 to achieve comparable generation quality while being 16 times faster in sampling than traditional methods. This speed is the key to enabling real-time interaction, reaching 21 FPS on a single H100 GPU.

Efficiency Gains from Shortcut Forcing

Caption: Shortcut Forcing (dark blue line) achieves the same generation quality in just 4 sampling steps that Diffusion Forcing (light blue line) requires 64 steps for, making it 16 times faster. Source: Training Agents Inside of Scalable World Models, arXiv:2509.24527

The Dream Continues: The Future of AI is Here

The emergence of Dreamer 4 is more than just a stunning achievement in Minecraft. Its deeper significance lies in illuminating a highly promising path for the development of artificial general intelligence: by learning how the world works from massive amounts of unlabeled video data, we can build an internal model for "imagination," thereby decoupling the agent's learning process from expensive, slow, and even dangerous real-world interaction.

The researchers envision future models being pre-trained on the vast expanse of internet videos to acquire more general world knowledge. Combined with language understanding, this could ultimately lead to general-purpose robots that can understand human commands, plan in their imagination, and execute complex tasks in reality.

From this perspective, Dreamer 4 dreams not just of diamonds, but of the future of more powerful, more general artificial intelligence.