Good baseline figures to track before and after Usability Testing


#1

Hi everyone,
I’m going to be conducting some Guerrilla Usability Tests on our website classifieds.advertiser.ie over the next few weeks.
[B]Does anyone have some suggestions as to a baseline figure I can track and compare before and after testing (and subsequent changes).[/B]

I will be pretty much on my own with these tests as the concept of UX hasn’t been embraced fully by the stakeholders.
Also the external company which developed the site did not use any UX principles to date.

I’m thinking of simply comparing visitors numbers to ads placed as a percentage of this.

And if we can identify some sticking points in the User Journey and fix these, hopefully we can increase the number.

Any suggestions much appreciated.

Paddy


#2

Hey Paddy,
This isn’t something that I can answer personally, but I’ll send some UXpert eyes this way.


#3

Brilliant question. Thanks for asking!

The phrase that runs through my head when planning before & after metrics is: “If it can’t be measured it can’t be monitored, and if it can’t be monitored it will be ignored.” So, we must focus on the most important aspects of the design changes, and how these affect the the people (and business) we’re aiming to support.

Essentially: what are the measurable success factors behind our need to (re)design?

Here are five steps to working this out:

  • [B]Work out what the critical user pathways are[/B] (the most frequent and critical activities that also reflect user and business objectives), i.e. “Place an advertisement”
  • [B]Give the activity some life and context[/B] by assigning it a persona and a user story. This is important for keeping our thinking grounded. i.e. “My name is Dylan. As [an advertiser] I want to [get my classified in front of the most relevant searchers] so that I can [achieve a higher conversion rate for my effort].”
  • [B]Determine what success looks like[/B] for that user story. Consider things like task completion rates, time taken and overall satisfaction. Try and stay away from generic statements like “easy to use” or “enjoyable” as they’re too abstract.
  • [B]Assign minimum and/or target values[/B] to these metrics by measuring the existing system. If you don’t have a current system, benchmark it against a competitor, or against your revenue projections.
  • [B]Monitor these metrics throughout the project[/B] to get an idea of how well you’re achieving your design goals. Test early, and test often, at the end of each iteration. Remote user testing tools are ideal for this kind of measurement.

I also find it really useful to map the design goals against the user’s journey using an experience map. It goes up on the design wall right beside the user stories.

If you want some more reading, here are some great articles about UX metrics:


#4

Thanks Luke, Great feedback, I’m really conscious that if I can’t measure a definite base figure and subsequent change. Then it becomes difficult selling the idea that the usability tests are vital. The figures need to be beyond doubt. Thanks Paddy


#5

Hi just a quick update on this, had a very good meeting last week and we have narrowed down our top three metrics. As a percentage of Unique Visitors we are now tracking (1) Ads placed (2) User accounts created (3) Sellers Contacted We are now tracking these monthly and hope to ad 1 or 2 more once we can work out a few more things on the analytics side. Now we are going to run some inhouse tests on staff to record good base time for some of these tasks. And soon we will sit some real users in front of the site and record them on the same tasks. P