Make a New Super Mario-Like Game With DeepMind’s Genie

In a nod to Super Mario fans, Google DeepMind has introduced Genie, a model that transforms brief descriptions, sketches, or photos into playable 2D platform video games. All of this comes on the heels of OpenAI’s reveal of Sora, a generative model for text-to-video, pushing forward the boundaries of creative AI applications.

Genie stands out by enabling the creation of classic platform-style games, reminiscent of Super Mario Bros from just a few simple inputs. But, it’s not the same. These generated games differ in pacing, operating at a deliberate one frame per second — a stark contrast to the fluid 30 to 60 fps seen in contemporary titles.

Training on an extensive dataset comprising 30,000 hours of 2D platform game footage sourced online, DeepMind’s Genie demonstrates a novel approach to game generation. Unlike previous methods that relied on correlating video footage with input actions, such as Nvidia’s GameGAN, Genie exclusively uses video footage for training.

This technique simplifies the learning process and expands the potential training material available online. The technique employed will likely be used by other models later in the future, or further altered.

One of the more interesting features of DeepMind’s Genie is its dynamic frame generation based on player actions, offering a unique gaming experience where each move directly influences the game’s visual outcome. As players interact with the game, Genie responds in real-time, updating the game’s visuals to reflect the characters’ actions, be it jumping or moving sideways.

Looking to the future, the development team, led by research scientist Tim Rocktäschel, hints at the potential for significant performance improvements, aiming to achieve a more conventional gameplay speed of 30 fps. This ambition is supported by advancements in technologies underpinning large language models, promising a more seamless gaming experience in future iterations of Genie.

Beyond game creation, the Genie project explores broader applications in AI research, particularly in open-ended learning and reinforcement learning. Similar to DeepMind’s XLand, which teaches AI bots cooperation through simple tasks.

Genie’s technology could also serve as a foundation for developing virtual environments for training AI on diverse challenges. This capability extends to robotics, where Genie’s learning model could enable robots to acquire new skills from video tutorials, illustrating the model’s versatility beyond gaming.

Although currently an internal research tool without plans for public release, Genie’s potential as a game-making platform remains intriguing. Both the AI and gaming communities eagerly anticipate the future developments stemming from Google DeepMind’s Genie, promising new avenues for creativity and innovation in game design and beyond.

Originally posted on OpenDataScience.com

Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.