The AdMogul – Usability Testing

The AdMogul Logo

We’re currently working on The AdMogul and last week, we had finished enough of the game so that we could play it from start to end. We still need to dot the i’s and cross the t’s – but the major gameplay was working. So, this Saturday, we invited some of our friends to put the game through its paces and see how it was doing – usability wise.

Getting a good sample segment for a facebook game is relatively easier compared to testing out some niche or enterprise software as everyone today has played some or the other facebook game and know what to expect. We had a good mix of people to test the game out with in the age range of 16 to 27 years with 6 men and 2 women.

This is our first ever facebook game and having no prior game design experience we did not know what to expect.
So on a rainy Saturday morning, with warm donuts, hot Startbucks coffee and packets of Pure Magic biscuits, we started our usability testing.

As soon as we got started, the design issues started to become apparent. We ourselves would have never imagined people getting stuck or confused with some of the aspects that they were — because the game is so close to us — but just by hearing and observing people try out the game within five minutes, it was so obvious to what was simple to do and what wasn’t.

We had prepared two sheets to fill.
One sheet had the twelve odd tasks written that the users had to perform. The tasks were generic and could be performed in more than one ways. We did not suggest how the task was to be performed — just what needed to be done. This allowed us to see which paths were easier to take.

The other sheet had a questionnaire which the observer filled in while observing the user.
We did not ask many questions and let the users go through the application themselves. The two questions that we did ask on each page / popup were:

  1. What is the first thing that strikes you when you see this page?
  2. What are all the things that you think you can achieve on this particular page?

The things that we noted down were:

  1. How the user went about completing a particular task
  2. How they navigated across a particular page.

Some important lessons that we learnt were:

  1. Each and every piece of consumer facing software should undergo usability testing.
    It is completely foolish to assume that your design will completely ring a bell with your target audience.
  2. Eight people – though good, was a tad too large for what we were testing. Four people should have been just right.
  3. After four people, the bad design symptoms just reoccur. I would say that 90% – 92% of the flaws can be exposed by testing it with four people. After that, the flaws increment very slowly (maybe 0.5% for every user after four).
  4. A concise, task based approach works really well if there are multiple screens and the concept is different.
    Every game has its own mechanics and ours was no different. So instead of asking people to just play the game, it was helpful to  give some definite tasks and see if they could achieve them easily.
  5. A test cycle should last anywhere between 25 to 45 minutes. After that, users will lose focus and interest.
  6. It is very important to make your users feel at ease and be able to think out loud. This will significantly helps the note taking process.

In hindsight, we realised that it is best to do this as early as possible. Probably once the screens are done, some click-thrus can be put together and checked. Another round can be performed with later — once the application is ready — with different people.

Luckily in our case, most of the changes that we will be incorporating are superficial – so there are no major issues as such.
However, it could have easily gone the other way.

If you plan on doing usability testing yourself, there is this good article that explains what needs to be done quickly. Also check out this tool – for quick testing of individual screens.

The AdMogul launches officially on 4th of August. If you would like to try out the game before others, sign up for a beta invite here: http://theadmogul.com

2 Comments

  • Pingback: The AdMogul Blog » The AdMogul Usability Testing

  • Amay Dighe
    August 1, 2011 - 12:28 pm | Permalink

    Hi,
    Very well written article and nice expression of the importance of usability testing. Yes as you said usability testing should be an integral part of the development and designers should get sufficient time to work on the point and issued raised by the group of audience representatives. Thanks for the other article link and even for the tool. I hope every one will make most of your work.
    Cheers

  • Leave a Reply

    Your email address will not be published. Required fields are marked *