04 May Usability, Focus Testing and Playtesting
As you may know we are currently working on our Vertical Slice, the details of which were covered recently here
In summation, it is a small section of the game to near-finished standard that allows us to see if the game we imagined in our heads actually works! The vertical slice covers everything from combat, exploration, puzzles, visuals, audio, music…the full shebang.
I am very aware that we are all fallible human beings and one big weakness of the human ape is our optimistic outlook: to overestimate how good something we create actually is in the eyes of others.
When you’re too close to a project, you cannot trust your own judgement and so you need to actively work against your preconceptions and just assume that what you think is great is probably… a little bit shit.
You often see creators extolling the virtues of their upcoming project confidently proclaiming it to be amazing only to be genuinely shocked that the world doesn’t appreciate their efforts. From the outside it’s baffling. It’s just human nature and I’ve fallen into this trap myself.
So what can you do about it? Well, you put it in front of others and WATCH and LISTEN! In games, you’ll hear about focus testing, playtesting and usability testing. If you’re not sure what separates the three, you’re in good company, many seasoned developers cannot tell you the difference with any great certainty. To complicate matters, the terminology is often used interchangeably. Here’s my take on it based on my personal experience:
Focus Testing: Best for washing liquid packaging
Typically done by publishers, this is one to avoid like the plague. The process involves getting a bunch of random people off the street to examine a piece of concept art, or character, or gameplay description and give their judgement to a moderator.
A report is written up, the publisher panics and asks you to change said gameplay, character or concept art because someone who knows nothing about such things doesn’t like it. In effect, the design process is taken away from experts and put into the hands of a layman.
The worst offending publishers actually design their games through this process. I’ve witnessed had a room full of people at a publisher tell me that they designed the perfect game, rated off-the-charts through this process, if only they could find a developer to make it. The games designed this way are usually something like: GTA meets COD meets WOW. All the BEST GAME EVER’s combined into THE BEST GAME EVER!!!
And it’s not just games: both New Coke and the London Olympics mascots were all designed using focus groups:
Play Testing: Best for Balancing
This is where you get a large group of people to play your game. The players are vetted to be the types of people who you want to sell to. So if you are doing a driving game, you might find people who casually play driving games, and those that love driving games. The more people the better.
You then let them play the game and track data, either through questionnaires or in-game statistical gathering, or both. These days, it’s possible to make this group very large as you can do it online through closed and open beta testing.
Largely though, you do this when you have a game that is fully featured and you need to iron out the peaks and troughs of difficulty, progression, balancing issues and so on.
We experimented with a free to play game called Fightback a couple years ago. F2P is a much maligned concept for many but just forget about the monetization aspect for a moment. The ability to track millions of players’ progress through the game, where they drop out, where they get stuck, and so on using very sophisticated tools is playtesting taken to an extreme scale. There are vital lessons to be learnt there for all game developers.
Usability Testing: Best for Improving the Game
Usability testing is my favourite of the three types of testing and the one that has the most dramatic impact on the quality of the game you are making.
It’s simply watching a small handful of people play your game and gently asking them to talk about what they are experiencing along the way. And you always make the full team watch the play through.
With about 3 people playing your game for around an hour each, you can usually identify 80% of issues you had no idea existed. You do this regularly throughout development until you have nailed the experience you wanted people to have.
This process only works if you start it early enough – a year or more before you are due to finish the game. If you wait until the back end of production, it’s probably too late to make a great game, but still worth doing nevertheless.
We first came across usability testing when working with Microsoft Games Studios who have it down to an art form with massive dedicated usability labs often run by psychologists.
Once you experience a project that goes through good usability testing, there is no going back. It’s brutal, sometimes shocking but always honest and useful. Best of all, it doesn’t take a lot to do this yourself.
We always ask our publishers to run extensive usability sessions but most aren’t too familiar with the process and, to my amazement, some are against it, citing cost, distraction or confidentiality concerns.
Given we are independently making Hellblade, we no longer have to ask permission to do this so we have renovated a tiny room set up with a sofa, a camera, mic and a little water tank with what admittedly looks like a dead fish in it. It does the job.
The Vertical Slice Sessions
We now have a first pass Vertical Slice up and running so we asked a few of our internal staff from other projects to have a go while the full Hellblade team sat in a meeting room and watched. It was horrific to watch. So many assumptions we inadvertently made were utterly destroyed by this first test. The team was very despondent at this point. In fact, developers generally fear this kind of testing because it shatters your best laid plans and with it, your confidence.
But however brutal the process is, it is nothing compared to how vociferous and critical your players will be once the game is out (believe me I know!).
We made some big changes after that session and were pretty confident for our next round. Once again, everything unravelled before our eyes but not nearly as bad as the first time. The mood was less bleak this time and, after a pub meeting, we had a new plan of attack. The next one will be the one!
Well, no it won’t but this is exactly what game development is all about: incremental iterative improvements that puts the player front and centre. Nothing focuses the team on the player experience more sharply than usability testing.
We will be doing this every month until we ship. However painful it feels at the moment, I know it will get better and better and that, by the end of it, we will have created a game we can feel proud of and one that will hopefully meet your expectations.