How to set up User Testing Guidelines


How to set up User Testing Guidelines

You probably heard it already several times, but I think you can never hear it often enough: User Testing is one of the most important parts of successful User Interface Design.

It doesn’t really matter how “user-centered” your design approach is and how much you “think” you know of your users – if you do not talk to them and test your concepts with your real users, chances are high that you’ll overlook some obvious improvements.

So what do you need to get started with user testing?

Actually not much! You need:

  1. A prototype
  2. Testing goals
  3. Testing guidelines

This blog post outlines how to setup proper user testing guidelines.

First things first, your prototype & testing goals

Those two parts are related to each other. You need to prepare a prototype based on your testing goals. An example: Let’s say you want to test a new workflow of signing up for a service. As you want to test the workflow itself you can probably test that with a low fidelity paper prototype. When you want to test new UI elements, you probably want to create a higher fidelity prototype where you design the UI components. Once you decided on the fidelity of your prototype you can start writing your testing guidelines.

User Testing Guidelines

I already did a lot of user tests, but I still prepare user testing guidelines for every user test I do. The reasons are simple:

  • They force me to properly prepare myself for the test
  • They force me to think about every testing scenario in detail
  • They ensure that I prepare all relevant states and screens within my prototype
  • They ensure that all tests follow the same structure and thus make the results better comparable

Part 1: Introduction

First of all, if the participants don’t know you yet, introduce yourself and tell them what you do.
Introduce the following testing structure to the participants. Tell them what you are going to do. Make clear that they do not need to be nervous, because it is not them who get tested – it is the application itself. If you record the interview (which I highly recommend – so you can focus better on the test itself) – let them know that you do so and that you won’t use the recordings for anything else. Lastly, the most important thing: tell them that you would like them to say their thoughts out loud, so you can better understand what’s going on. This is especially important when testing remote.

Example:

“My name is Lisa, I am the User Interface Designer at Cropster and I will lead you through this test. Before we start, I will give you some information about the procedure of this test.

We will test a new part of our application (be specific if possible). Not you as person will be tested, but the prototype itself. You can be honest, because every feedback (positive & negative) will help us to improve our application.

At the beginning I will ask you some questions about your personality. Right after that you get 5 short tasks to solve. After finishing the tasks, there are some further questions I will ask.

The test will be recorded. These files help us to analyse the problems later and to improve our application. These recorded files are anonymous and won’t be used for other projects.
If you have questions during the test, you can ask me anytime. For a better analysis of the test, please speak out loud your thoughts during the tasks.

Do you have any questions now?”

Start recording

Part 2: Personal Questions

Those questions are very important for two purposes. First of all, to get to know your participants better, although you probably know some details – as you’ve recruited them. I would recommend to add some questions which puts them already into the right mindset. So let’s say you want to test this new Sign-up workflow, you can for example ask them: For how many online services have you signed-up for in the last month? Starting with those questions also get’s you started with a conversation and thus leads to more open feedback when you are starting with the next part – Completing tasks.

Example:

We start with some questions about your personality.

  • Tell me a little bit about your job :____________
  • Which digital devices do you have (smartphone, tablet, laptop, ….):________________
  • How often do you usually do XY? _________
  • Do you use any other service than XY? _________________”

Part 3: Completing Scenarios (Tasks)

That’s where you need to get back to you testing goals. So what do you actually want to test? Of course you want to test a specific workflow; but you should always break it up in smaller task. I recommend 3-6 tasks. When you have tasks you can more easily measure specific key metrics or success criteria on every test, which makes it easier to compare the results to each other.

When you start with you tasks, give the participants a starting situation.

Example:

Your starting situation:

You heard of a new web service where it is possible to buy green coffee. You are interested and so you open their website.

Please open: [domain]

Please explain your first impressions & the elements you see in the application.

Task 1:

The service got your interest, so now you want to sign up for a free account. Where do you think you can do that?

Success criteria

  • Where there wrong assumptions (about interface elements)?
  • Did the user need help to complete the task?
  • Where there murmurings of discontent?
  • How many tries did the user need to complete the task?
  • How long did the user need to complete the task?

Part 3: Open questions

The open questions usually depend a little bit on the interview. I’ve a set of pre-defined open questions but I only ask the ones that weren’t answered during the interview anyhow. Sometimes I also ask different questions, if the conversation leads to something else. The open questions are a chance to clarify things you noticed during the interview.

Example Open Questions

  • How was the orientation within the application for you?
  • Was there something you really liked or that you really disliked?
  • Do you think this kind of _____ would help you in your daily workflow?
  • How did you like _____?
  • Do you think there is something missing?

Now we are finished. Thanks for your support.

Stop recording

Conclusion

As you can see, user testing guidelines are not only for the participants, they are also for the person who is conducting the interview. That’s the reason why I include internal reminders in my guidelines, like “start recording” and “stop recording”. I would recommend to line out a template for user testing guidelines, that works for you and start based off this template every time you need to setup a new test. This saves time, but also makes sure that you stay consistent. Of course my template is not static – every time I find out something works better, I will adapt it. I hope that helps you getting started with your User Testing Guidelines. I am happy to hear your experience on user testing.

Lisa Gringl

http://lisa-gringl.com

I‘m Lisa, a Visual Designer, with a passion for responsive Webdesign, UX and Typography living in Innsbruck. I love creating something visual, I like coding and I love the board game „settlers of catan“