The last twelve months have been something of an "emotional rollercoaster" for Sergey Galyonkin.
The SteamSpy creator and Epic Games publishing strategy director moved country (again) and helped launch a little-known game called Fortnite.
Oh, and then there's the small matter of SteamSpy running into some serious trouble earlier this year after Valve changed the way profiles worked on Steam. Where before they were public by default, the firm switched them to be automatically private. This created some problems for SteamSpy, given that looking at the game libraries of public users was how the site collected its data.
Since Galyonkin has been developing some workarounds so that the site can still provide data for its users.
"I like the challenge," he tells PCGamesInsider.biz.
"The thing about the workaround is that the things I use SteamSpy for are still very much viable. I'm mostly looking into cross audience, trends and geography. Even without the exact numbers, the figures I have are still good enough for me, personally, in my day job.
"It's less useful to the general gaming audience because consumers like to see exact sales figures. SteamSpy is no longer accurate for them, which is why we have estimates and ranges instead of exact numbers. Most of the general audience do not understand the basics of statistics and I don't want them to feel misled. If they see the number and it says 10,000 copies but it could be anywhere from 1,000 to 50,000. It might feel less accurate but essentially, it feels fair."
This inaccuracy is a criticism that SteamSpy has fielded many a time - just last week Valve's business development chief Jan-Peter Ewert said the company was nervous about SteamSpy because of its inaccuracy. But, at the same time, Galyonkin has always been very open about the fact that his figures were estimates and should not be treated as gospel truth.
While he says he's okay with that criticism, he says that the recently shutdown algorithm that used achievement data to figure out sales was entirely correct.
"It felt like a fair criticism," he admits.
"But when we were using Tyler Glaiel's algorithm SteamSpy was 100 per cent accurate. It was accurate down to a single copy. I essentially rounded up numbers because I wasn't sure how I felt about having actual sales figures instead of estimates. But then it was accurate down to the copy. Several days later, they killed that again. I feel like they were misleading people. I don't feel that SteamSpy and accuracy was the problem. Accuracy was the problem."
Asked about SteamSpy's success, Galyonkin says it was a mixed bag. While he was happy with the information he was publishing, a lot of people were misunderstanding it.
"It was a double-edged sword. On the one hand, I felt extremely proud that people were using the data and that it was helpful. Every time I hear a story about somebody getting investors or getting funded or releasing a game because they researched the market with SteamSpy. I was a tiny, tiny part of the success, but I feel accomplished to some extent.
"On the other hand, I saw people misunderstanding the data. I wouldn't say I saw anyone misuse the data. But I saw people claiming a game was a failure because it sold x many copies and that other game sold so much more - not taking into account that the other game had been on the market for five years and the other had just launched; it was priced differently, had success in different markets and so on. Even with SteamSpy, you don't see the whole picture; you don't know how much money developers spent, what the targets were and so on. It's not apples to apples every time a game sells, and a lot of people forget about that. But as time goes on, I see developers and gamers understanding SteamSpy better and using it more efficiently."
Galyonkin goes on to say that it's important to have an idea of how well games are performing on Steam due to the vast array of titles that come to the store - and it's especially useful for the developers of smaller games.
"There aren't many marketplaces in the games industry that not only provide data but are home to a diverse set of games," he explains.
"A lot of modern multiplayer games have their own APIs; Battlefield has its own so you can check the numbers, so does World of Tanks. There are a bunch of titles where you can do that. But it's not very useful to the average indie - that's why I was never intending on developing a site that aggregates data from those big guys. An average indie developer will not be developing the next World of Warcraft or Battlefield because they're just out of reach. It's useful to the average developer to be able to look at indie and mid-sized games. They are, right now, either on Steam or the App Store.
"There are a bunch of tools to research the App Store - Sensor Tower, App Annie and SuperData to name just a few. There were no tools like that for Steam apart from SteamDB. That's great, but it doesn't give you a view of the whole picture. It was designed to give you precise information from Steam itself without any estimates or stuff like that. I feel like that's SteamDB's goal; to be as accurate and as semi-official as possible. Steam is unique as it is home to many, many diverse games. If you want to research the games market, you should start with Steam then look into other markets as well. Console developers are using SteamSpy and Steam as basic research for console games. Gamers are different by platform and country and gender, but they're not all that different."
It certainly is important to have some idea of how well games on Steam are doing, which is a challenge given that Valve is seemingly blocking third-parties trying to get a sense of the market. But the company is also apparently working on its own data solution - something Galyonkin is actually excited for.
"I would love it to come sooner rather than later; Valve has access to exact numbers and that's why they complain about SteamSpy not being accurate enough," he says.
"They have data that is exact enough. I would love them to release their own version of SteamSpy. I'm actually looking forward to that. If they do, it means that we'll have the first platform to talk about its data. It would be a really important addition to the games industry and would be really great, actually."
Moving forwards, Galyonkin says he is going to continue working on the machine learning algorithms that he hopes will soon be powering SteamSpy.
"Well, Valve doesn't reply to my emails. I don't think there's a conversation to be had to be honest. It's more like a monologue," he laughs.
"I'm working with some machine learning specialists on the new algorithm. There are some issues with that - it'll have to be retuned every now and again and maybe every year or so. It's not entirely accurate for big games - it's the exact opposite of the old problems. I have few data points for big games so it's less accurate for them as opposed to smaller titles where I have enough data. Or it's going to work with some degree of precision; people don't understand how imprecise other methods are. If you look at App Annie, that might be wrong by a factor of three. If you look NPD they can be off by a factor of two to three, I've seen it myself. That's fine because if you know there's a margin of error, you just work with that. That's how statistics work. You will always have margins of error - that's not an issue so long as you are aware of it. Even with less precision, it's still going to be good enough as I mentioned before.
"All the other stuff like geography and playtime and so on is still there and is still useful; to me at least."