Google researchers recently unveiled a new artificial intelligence (AI) game engine called GameNGen, which is powered entirely by a neural model and capable of real-time generation over extended periods. The researchers assert that GameNGen can create complex environments at a high frame rate.
Impressively, the engine managed to interactively simulate the classic game Doom at more than 20 frames per second, all while running on a single Tensor Processing Unit (TPU).
In a paper published on the online pre-print journal arXiv and detailed in a GitHub listing, Google described the neural model-powered game engine. Developing a game engine is a challenging task, as it requires generating intricate 2D and 3D environments at high speeds, while also maintaining logical sequencing to ensure smooth level progression.
The paper emphasized GameNGen’s ability to simulate the 1993 video game Doom interactively at over 20 frames per second, meaning that these are not just static images or videos—players can actively engage with the generated content.
Diffusion Models Are Real-Time Game Engines
abs: https://t.co/Fg8EOZ9DlI
project page: https://t.co/HK9IhZwYDaGoogle presents GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectories… pic.twitter.com/8YBvNbBsy0
— Tanishq Mathew Abraham, Ph.D. (@iScienceLuvr) August 28, 2024
To train the AI-powered game engine, researchers employed two main processes. First, they used Stable Diffusion v1.4 for training and introduced a novel approach to reduce auto-regression drift—where the AI predicts the next sequence based on previous sequences—by adding Gaussian noise to encode the frames. Second, they utilized automated reinforcement learning (RL) agents to collect data. Since gathering data at scale from human players would have been impractical, these AI agents played the game, providing a substantial dataset for training.
Currently, the GameNGen AI game engine is not yet available for public download or testing, with only the research paper being accessible. It’s important to note that publishing on arXiv does not involve peer review, so a full assessment of the engine’s claims and methodology is still pending.
