It isn’t Minecraft, but an incredible simulation of the popular game generated in real time by artificial intelligence. The game is called Oasis, and it lets players explore a 3D world made of square blocks, mine for resources, craft items and explore. Much like the original game it was trained on, it’s immersive, if not always responsive in the way you’d like. It also bears the distinction of being the first fully AI generated game to escape from the lab, so to speak, one that you can play yourself.
The game runs on a single computer, with an NVidia H100 GPU providing the incredible horsepower it requires. Even there, Oasis can only run at 720p resolution, and only at about 20 frames per second, and that’s sometimes an optimistic estimate. The H100 retails for about $25,000, so this technology isn’t available at the consumer level just yet. Thankfully, your own computer doesn’t have to do any of the heavy lifting. It’s all online, using their computer to create the graphics. And, to be clear, there is no actual game engine involved. It’s all a simulation of what you might see and experience if there was one.
When you start your session, you can choose from a number of different starting environments. You can even upload an image yourself, and it will generate a world using that as a starting point. As you play, your actions and movements trigger changes in the environment around you. Things happen more or less as you expect them to, because the AI was trained on millions of gameplay hours and it knows approximately what to show you at any given moment given the control inputs it’s received from you.
However, it also has very little in the way of object permanence, i.e., if you look at something, then look away, the thing you were looking at may or may not be there when you look back. The world you see is reasonable and behaves reasonably well, but the part you aren’t looking at essentially disappears and has to be regenerated from scratch when you turn in that direction. This yields a sort of dreamlike hallucination of a game. Even distant objects are subject to the random morphing as you approach them. Inventory items may magically appear or disappear, and walking up to a blank wall or cliff face may result in the environment changing entirely when you turn away from it again.
“You’re about to enter a first-of-its-kind video model, a game engine trained by millions of gameplay hours,” the game’s creators, Decart and Etched, explain as you enter. “Every step you take will shape the environment around you in real time.”
“The model learned to allow users to move around, jump, pick up items, break blocks, and more, all by watching gameplay directly,” the team explained in a statement. “We view Oasis as the first step in our research towards foundational models that simulate more complex interactive worlds, thereby replacing the classic game engine for a future driven by AI.”
A construction game that has no object permanence is obviously not something you want to play for very long, but that isn’t the point. What makes this exciting is that AI has advanced so far and so fast that we are even having this discussion.
“In essence, diffusion models learn to reverse the iterative process which adds Gaussian noise to the input and thereby enable generating new samples given noise,” the team explained. “This approach can be extended towards video generation by adding additional temporal layers to the model architecture that receive a context consisting of the previous frames that were generated (e.g., in an autoregressive fashion).”
The team has high hopes that AI generation could “revolutionize” games, enabling more interaction between gamer and the game they are playing.
“Simply imagine a world where this integration is so tight that foundation models may even augment modern entertainment platforms by generating content on the fly according to the user preferences,” the team says, “or perhaps a gaming experience that provides new possibilities for the user interaction such as textual and audio prompts guiding the gameplay (e.g., ‘imagine that there is a pink elephant chasing me down’).”
Oasis is more a curiousity than an actual, viable game, but diffusion models are being made into real time game engines. For example, Google is working on a project called GameNGen, which does for Doom what Oasis does for Minecraft, but at the moment, all the public gets is a white paper. However, you can play Oasis yourself here. It’s free to play, and runs in your browser, though you will probably have to wait in a queue for the opportunity.
Artificial intelligence is already being rapidly adopted by game companies in the creation of game assets, for not only helping to write the code, but for generating assets such as textures and even 3d models in some cases. Oasis is the next logical step. Would anybody pay to play a game like this? Probably not, but think of this as our Pong, compared to games like League of Legends and Skyrim. At the speed at which AI breakthroughs are taking place, three years from now may see us asking that same question and getting a different answer.
