OpenAI’s AI-powered Dota 2 bot, OpenAI Five, has managed a near-100 per cent win rate since going public last week.
After beating pro players earlier in the month, OpenAI opened the doors to let the public take on its bot technology through the OpenAI Five Arena. Five won 7,215 competitive games during the week, compared to a measly 42 games won by human teams, giving the AI a 99.4 per cent win rate.
Letting players put Five to the test wasn’t just an experiment in stomping all over human players, however. Speaking to VentureBeat, OpenAI claimed the public Arena was meant to examine any potential exploits in Five.
“Arena was a massive-scale experiment to test whether OpenAI is exploitable, given the entire Internet trying to break it,” said OpenAI CTO Greg Brockman. “The Dota community teamed up, cataloguing every weakness. While Five has more to learn, no one was able to find the kinds of easy-to-execute exploits that human-programmed game bots suffer from,”
“This lets us have more confidence that future AI systems we deploy in the wild will be able to be made robust and hard to subvert. And perhaps even more importantly — we learned the value of having a community of people excited to pore over a system we’ve built in order to truly understand the limits and impacts of what we’ve built.”
OpenAI has been trying to crack Dota for a while now. A little under a year ago, Five first beat a team of human pros in a heavily-restricted version of Valve’s esport hit.
The version of the game played this time slightly opened up the hero pool, but Five is only able to play 18 of the 115 total heroes in Dota 2. However, the bot can now draft a team by itself in response to opposing team picks.
OpenAI will use the results from this limited test to further develop Five, investigating its reliance on macro decision-making over spur-of-the-moment plays.
While the main game is steadily bested by artificial players, who knows when we’ll see researchers try to take on the increasingly-popular game-within-a-game Dota Auto Chess.
Editor's Note: I recently decided to rewatch The Terminator films with my fiancée as she hadn't seen them and news stories like this give me The Fear. As you were.
