Minecraft Is Digging Deep Into AI For More Immersive Gameplay

For those of you who don't know what Minecraft is (you probably are having a great time under that rock), it is a sandbox game where players explore a free world that they generate through elaborate steps using 3D blocks, build tools and structures, among other human things to do in a pixelated world. Maybe just take a quick look at this movie trailer based on the game for a sneak peek.

The wildly popular game has been the centre stage of experiments around the use of AI for a while now. For instance, the MineRL Project in which researchers used "reinforcement learning" - a machine learning technique whereby the AI system learns based on rewards or penalties in a specific environment- to train an AI system for solving complex problems in Minecraft. Even OpenAI in 2022 trained its own AI bot based on 70,000 hours of gameplay footage, to perform tasks autonomously in Minecraft world like creating the diamond tool.

But the latest advances in Minecraft gameplay have sparked special intrigue amongst gaming enthusiasts. The Oasis AI model, created by Decart AI- a California-based AI startup that uses open-source libraries- is a real-time interactive simulation model offering an AI-driven unique Minecraft experience. The model is essentially a third-party mod based on Minecraft's API.

Oasis spread like hotcakes when it was released in November, with about 1 million unique users joining the platform in just about 3 days and seven hours.

Oasis v1 has surpassed 1M unique users in 3 days and 7 hours since launch

As a reminder, this was just a teaser

Our goal was to show real time video inference and to test our infra before a much bigger launch later this year 👀

We ❤️ the AI community and the… pic.twitter.com/HWtpPDMHy5

— Decart (@DecartAI) November 4, 2024

So why is it such a big deal?

For one, this means that each move that players make dictates the nature of the game itself - each frame is generated using an LLM model that has been trained on millions of hours worth of videos from the games played in Minecraft. That is the same as ChatGPT generating text but the difference is it "speaks" the language of Minecraft, creating a unique pixel world for you to experience.

Second, you can play this real-time simulated game on your phone, or desktop without any heavy lifting required by hardware engines that run the game traditionally, such as Unreal Engine and Unity.

Oasis currently is just a "demo," according to Decart AI. There is more to come in their upcoming models, with the company banking on the advancement of transformer-specific AI chipsets, such as Sohu by Etched.

But wait, Is that all there is to AI in Minecraft?

Not at all. Consider this mind-boggling experiment conducted by an AI startup called Altera, as reported in the MIT Review. The experiment involved up to 1000 AI agents powered by LLMs introduced into the open world of Minecraft games. The AI agents were given basic text prompts and were then allowed to interact with each other on their own.

The experiment involved multiple combinations of AI agent players inside Minecraft. In one of the experiments involving around 500 agents, they observed that the agents created cultural memes and sent them to each other.

In another experiment, the team nudged a select few agents to spread a parody religion called Pastafarianism- "the word of the Church of the Flying Spaghetti Monster," and they observed that the agents were successful in spreading the religion across towns and rural areas inside the in-game world! The converts further went on to spread the religion even more.

Robert Yang, the founder of Altera, believes that the experiment is just the beginning of what he believes can be an "AI civilization" that can coexist alongside humans and work in the digital space.

But are these AI agents alive?

Altera co-founder Andrew Ahn explained that such human-like conscious behaviour is only possible due to "a sophisticated enough model of human social dynamics [to] mirror these human behaviours." That means it was made possible due to a specialized training data set which translates context-aware human behaviour into quantifiable data.

Alive or not, this experiment marks exciting times ahead for AI's foray into the world of gaming. Besides, popular gaming studios like Electronic Arts, Tencent and even OpenAI are racing against time to release their versions of text-to-game AI models. Elon Musk, did you just say something? Aah, You too.