Brilliant question. Thanks for asking!
The phrase that runs through my head when planning before & after metrics is: “If it can’t be measured it can’t be monitored, and if it can’t be monitored it will be ignored.” So, we must focus on the most important aspects of the design changes, and how these affect the the people (and business) we’re aiming to support.
Essentially: what are the measurable success factors behind our need to (re)design?
Here are five steps to working this out:
- [B]Work out what the critical user pathways are[/B] (the most frequent and critical activities that also reflect user and business objectives), i.e. “Place an advertisement”
- [B]Give the activity some life and context[/B] by assigning it a persona and a user story. This is important for keeping our thinking grounded. i.e. “My name is Dylan. As [an advertiser] I want to [get my classified in front of the most relevant searchers] so that I can [achieve a higher conversion rate for my effort].”
- [B]Determine what success looks like[/B] for that user story. Consider things like task completion rates, time taken and overall satisfaction. Try and stay away from generic statements like “easy to use” or “enjoyable” as they’re too abstract.
- [B]Assign minimum and/or target values[/B] to these metrics by measuring the existing system. If you don’t have a current system, benchmark it against a competitor, or against your revenue projections.
- [B]Monitor these metrics throughout the project[/B] to get an idea of how well you’re achieving your design goals. Test early, and test often, at the end of each iteration. Remote user testing tools are ideal for this kind of measurement.
I also find it really useful to map the design goals against the user’s journey using an experience map. It goes up on the design wall right beside the user stories.
If you want some more reading, here are some great articles about UX metrics: