ALL THE LATEST NEWS ABOUT THE BUSINESS OF PC GAMES

Interviews & Opinion

Meet the virtual gamers who test while you sleep: modl.ai unveils QA bots

Meet the virtual gamers who test while you sleep: modl.ai unveils QA bots

While tools like ChatGPT have dominated discourse recently, AI has a ton of behind-the-scenes benefits too. It isn’t all about drafting marketing guff for social media – AI can be a tool for core game development processes like Q&A and testing.

With that in mind, we sat down with Christoffer Holmgård on the show floor at GDC last month to discuss what modl.ai has lined up in this space.

The company delivers AI and machine-learning bots that play, grow and learn inside your game. You can have millions of (artificial) gamers playtest your game at speed. King, N3twork, Nitro and other big companies are already partnered with modl.ai. Holmgård co-founded modl.ai in 2017.

“We’re delivering AI to the games industry now!” says Holmgård, referring to how much speculation there is at GDC about the role of AI. “You have game engines, and you have audio engines, and you have physics engines. We think everybody should have an AI engine as well.”

The modl.ai team. The company is based in Copenhagen.

But this is something other than the wave of text and image generators that has been getting all the press. “We are good, old-fashioned AI,” he jokes. “It’s a behaviour. It’s things that control players inside of games. And what we tell people is: we’re the bot people. We built bots for games for different purposes.”

The growing need for more testing

The service from modl.ai uses bots to automate parts of the QA process. “I don’t think anyone has ever said they had too much testing or QA!” jokes Holmgård. “That’s what we’re trying to enable for studios of all sizes – enabling them to have unlimited virtual players to test out the game. You can also have multiplayer characters and machine-learning-based opponents that you can play against. We want to bring that to the whole industry.”

In this form, modl.ai has been running since 2018, but the core team have known each other for some 15 years. There were six people at the start, and now the company is over 30 people. Christoffer Holmgård has a background in indie game development and has a PhD following a stint as a machine learning researcher. All of this goes into the mix at modl.ai. “We taught the machine-learning people to be game developers, and we taught the game developers to be machine-learning people!” he jokes.

A bot inside a 3D location is about to fall through the floor and trigger an out-of-bounds error.

In 2020, modl.ai closed a $1.7 million seed funding round led by PreSeed Ventures, Saltagen Ventures, and Norwegian Propagator Ventures. Then in September, the company closed series A funding of $8.5 million with lead investor Griffin Gaming Partners and M12, Microsoft’s Venture Fund, with additional participation from Rendered.vc, PreSeed Ventures, and Transistormedia.

Its first version, modl.ai: Test (a platform that developers can licence and use independently) was launched during GDC week as a beta programme. He fires up his computer to show us, live on the show floor, what developers can experience. We see a Unity development environment with the modl.ai plug-in running. A bot starts running around in the level, pinging back data where it encounters an error and recording how the game performs.

“We’re using our new Unity plug-in,” says Holmgård. “We give it access to the player character. Outside of the game, our AI engine connects to it through a plug-in, and it takes control of the player. What starts happening is, essentially, it’ll start playing the game for as long as we want.”

Addition, not replacement

“I think that giving these kinds of tools to the QA teams out there will enable them to do more work and to ship with greater confidence,” he continues.

The idea is that you’ll upload a copy of your game to the modl.ai platform, and then “you can start 100 copies of your game, and then the AI sweeps through it, and then come back the next morning to find out what’s going on in your game, how did it perform, what kind of errors did the ‘player’ encounter? We’ll even record everything.”

There can be trepidation that AI will take away human labour. But [if] you give people better tools, you can do more
Christoffer Holmgård

Moment-by-moment information is reported to a dashboard. “Every little grey dot here is a place of interest for the bot,” he indicates. As we talk, it’s running for 30 minutes, covering half of the map.

Importantly, though, Holmgård sees this as an additive tool to help teams, not a replacement for testers. “This is a way that you get more out of the people you already have in your team,” he says. “I understand there can be some trepidation that this will take away all the human labour. But the way that we think about it is that you give people better tools, and you can do more testing. When spreadsheets came out in the ’80s, accountants didn’t go away! People did much more. They did more financial analysis and more simulation. And I think the same thing will happen with game testing.”

He recalls his time as a game developer: “You don’t have enough staffing or time. You probably don’t have enough time, even if you have thousands of testers. As a game developer, I often did so much testing myself, and we also sent it out to external parties. I still had a game crash on national TV. So to my mind, you can never have too much testing.”

The promise of artificial intelligence

The pros and cons of AI, not to mention the money it’s attracting, has been a theme all year. Many companies had AI projects at GDC. But the basic concept has been around for a while. Why is it the hottest topic in 2023?

“These things move in fits and starts,” he says. “We’ve had a technological watershed moment, where the models can suddenly do things they couldn’t do before. But I also think that OpenAI, in particular, has done a great job of making that available to people. AI was possible at a smaller scale before, but now everybody can experience it. That just drives the conversation and the whole narrative.”

Right now, modl.ai is focussing on this testing API. But Holmgård’s team plans to do more with intelligent bot behaviour in the long term. It has been showing studios some of their experiments. 

You can set up custom alerts using the modl:test plug-in for Unity.

“We’re using an AI guy trained on early-access data from players playing the game,” he says. “It has learned from them how to play the game. So imagine that you have bots that live with your game, learn from your player base, and can learn to operate the game from what we’re seeing out there, and maybe even personalise it. Maybe bots of different skill levels, and I can make one that drives like me. Any multiplayer game out there can have access to this stuff, which we want to bring to the industry at large. Much of the tech has been gated inside the larger companies because they have the resources to do it. But if you ‘modelise’ it, you can make it available to everyone.”

There is more information about the use of bots in testing, including some case studies, at the official modl.ai site. Our partner site BeyondGames.biz covers AI in games every week, and you can hear from game developers using AI at PGC Seattle next month.

COO, Steel Media Ltd

Dave is a writer, editor and manager who today is Steel Media's Chief Operations Officer. He gets involved in all areas of the business, from front-page editorial to behind-the-scenes event strategy. He began his career in games and entertainment journalism back in the 1990s when Doom came on floppy disks. You can contact him with any general queries about Pocket Gamer, Beyond Games or Steel Media's other websites, conferences and initiatives.