Usertesting: test 5 users one by one OR all 5 at once?

usertesting
usability

#1

Hi!

It is for think-aloud protocol.

Which approach do you think the best and why? Have you tested them?

A)

Test 5 users - analyse and improve prototype based on 5 users tested.

OR

B)

  1. Test 1st user - analyse, improve prototype

  2. Test improved prototype with 2nd user - analyse, improve prototype

  3. Test improved prototype with 3nd user - analyse, improve prototype

and so on for 5 users.


#2

If I were to do a usability test I think I’d go for option C: first test with 3 users, then iterate with the remaining two.

The first user you try may be a weirdo with esoteric preferences. Or maybe user two is the esoteric one. The third user will be the casting vote that tips the scale if the first two give different results.

Improve, then test with fourth user.

Improve again, then test with fifth user.


#3

I think this is a good idea, and a better option than doing testing on individual users and reiterating after each test.

However, the first test you describe is a perfectly valid form of UX testing, known as a focus group, but you do have to be aware of the strengths and weaknesses of the process. Primarily, focus groups will give you a good insight into what your target audience might be while giving you good ideas on concepts and patterns that are the basis of your UX. Focus groups are also great for reaching a larger number of people at once.

They cannot, however, give you very good detailed feedback, nor solid feedback on usability in general. What’s more, focus groups will inherently give more biased feedback, as humans will bias there responses based upon the group surrounding them. Removing additional participants allows for more in-depth answers, a better insight to general usability standards, and less (though not a complete absence of) bias in responses.

Depending on your goals, one option might be better than the other.


#4

Oh, I didn’t mean that the first three should do the testing as a group. Rather I meant that no conclusions or changes should be drawn/made until three users have tested the product.

Sorry that I was unclear and thanks for teaching me what a focus group is! :slight_smile:


#5

No worries! Sorry if that comes across as a little condescending; I certainly don’t mean it to sound that way. You never know who knows what 'round these parts :slight_smile:


#6

It’s good that you told me because I didn’t know. :slight_smile:

(what focus groups are)


#7

If you have to choose between option A and option B, which one would you prefer? (Assuming that the people in option A test individually)