GOOD Teams: Product Intelligence knows what players want
Anyone who works in game development knows how complex and challenging it is. For the best odds of success, a game team should include industry experts from various fields who work closely together towards a common vision: creating the best possible product. One such team of experts at Goodgame Studios is Product Intelligence, which studies data from our games to find out how we can improve our existing and upcoming titles. Product Intelligence analyse player behaviour to discover what keeps our users engaged and motivated, and provide these insights to the product team, game designers and developers. Their analyses generally contain recommendations on how certain behaviours can be incentivised and, ultimately, how the game can be made more rewarding for players. “Our team is really in the trenches, serving as the front line between the product and the 100GB of data we collect daily,” explains team lead Alexei Vink.
The Product Intelligence team at Goodgame Studios is made up of almost 20 data analysts and data scientists, who sit in sub-teams embedded within Goodgame’s individual development studios, whilst keeping strong links to the whole team every day. “Everyone does a bit of everything,” Alexei clarifies, but at a granular level, the data analysts tend to bring deeper game knowledge, while the data scientists lean towards heavier modelling skills. “The team is filled with brilliant and curious people from different fields – physicists, mathematicians, economists, computer scientists, even chemists. These are very bright guys and gals with great ideas on how to approach different topics, and it’s genuinely a pleasure to be working with them every day”, says Alexei. Alexei himself has been working in Product Intelligence for a little over a year, having previously worked in two different game companies after an early career in investment banking. He joined Goodgame Studios, he says, “because I wanted to be involved in a game company that was more serious about its analytics. Here we pay close attention to significance, to sample size, to statistical fallacies. We guard against confirmation bias and action bias. You have this feeling that you’re doing things properly; you’re telling the truth, the whole truth and nothing but the truth.”
In their day-to-day work, the team conducts a wide range of analyses of different scope and complexity. In general they identify problems or opportunities, undertake data-driven studies on these, and synthesise their findings into a recommendation. This recommendation could be specific for an individual game, or applicable for the whole company. Alexei explains: “We often optimise things – we may find or hear about something that can be improved in our games, so we study the data, generate hypothetical improvements and test them – maybe a feature is too hard, maybe loading times are too long, maybe we’re not giving the players enough to do. Experiments, where we change the game for some players and study how it impacts their behaviour, are usually the best way to be sure.”
For some analyses, the team takes a deeper dive into the numbers, crunching data to make predictions or determine causal relationships between things. “Sometimes we go outside the games and try to give Goodgame a competitive edge. Maybe we want to figure out which phones players will be using next year, so our artists know how complex their assets can be. Or maybe it’s something more novel: for example, we recently began using matrix factorization to work out what’s driving revenues in competitor games, so we’re backsolving retentions and lifetime customer value from AppAnnie data. That’s really interesting and can help us find and learn from hidden ‘jewels’ in the app charts which maybe don’t have King or Supercell-sized marketing budgets”, says Alexei.
At Goodgame Studios, the Product Intelligence team is part of a centralised Business Intelligence department, meaning that the game studios are internal customers rather than disciplinary managers of the team’s work. “This is incredibly important, this idea of protecting the company against its own confirmation bias,” Alexei says. “All of us suffer from cognitive biases as human beings: do you see the truth, or do you just see what you want to see? We absolutely want to guide the studios forwards, we love to make them happy with our findings, but having the freedom to say ‘sorry, but it looks like that feature just isn’t working like this’ is really important – it’s the only way we can all learn properly and move forward in the right direction. ”
According to Alexei, the best part about the job is knowing that you have created insight that helps studios to invest their resources in designing and implementing a new feature in the game. “That’s a very meaningful process, where you’re really feeling the impact of your work, proposing some change that makes the game better for millions of players overnight,” says Alexei.
