In February 2024, Google's DeepMind released a generative model called Genie, which can take a static image and transform it into a side-scrolling 2D platformer that players can interact with, and we got a first look at the technology. In December, the company released Genie 2, a model that can transform an initial image into an entire virtual world.

Other companies are developing similar technologies. In October, AI startups Decart and Etched unveiled an unofficial cracked version of Minecraft, where every frame in the game is generated instantly as players play the game. World Labs is a startup co-founded by Feifei Li, a well-known AI scientist and known as the "godmother of artificial intelligence", which is building the so-called Large World Model (LWM). (Li Feifei is also the creator of ImageNet, a massive photo dataset that started the deep learning craze.) ）

One obvious area of application is video games. These early experiments are fun, and generative 3D simulations can be used to explore the design concepts of new games, turning sketches into playable environments on the fly. This could lead to a whole new type of game.

But they can also be used to train robots. WorldLab wants to develop so-called spatial intelligence – the ability of robots to interpret and interact with everyday life. But robotics researchers lack high-quality data from real-world scenarios to train this technique. Building countless virtual worlds, putting virtual robots into them, and learning through trial and error can make up for this shortcoming.