How do you measure UX

ux
usability
research

#1

Hi all,

Been a while since I have been active on the forum, but today I’m back with an intriguing question: Do you attempt to quantify the user experience of your products or service, and if you do, which metrics do you use?

I know that a user experience is impossible to quantify because of the emotional aspects, but I’m looking for certain aspects of UX that can be quantified. Think of as aspects such as:

  • how many form validation errors does a user make?
  • how long does it take them to complete a form?
  • how many steps does it take for them to complete their goal?

These are pretty straightforward metrics, but I’m curious if you quantify any other aspects of your product or service. And does anyone, by any chance, use the Customer Effort Score? Love to hear your experiences!


#2

I’ve never heard about it, can u share some links about it?

I don’t agree on this. I struggle every day explaining to my stakeholders that UX (and UI) has to measureable.

UX is measurable if you define the metric and the KPIs based on the product goals and the customer satisfaction you would like to reach.
An example could be the A/B test, this is a precise metric that provides a way how to develop the UX.
If we are talking about qualitative data then another example could be, as you said, to measure how long it takes for a bunch of users to fill-in the registration form for a service.

In my opinion is mandatory to have a precise view of

  • the goals the company wants to achieve
  • the status of the product (new, re-launch, mature phase)
  • the gap between the current status and the target of the Customer Statisfaction

#3

The thing I’m struggling with, is that my stakeholders want to measure the UX (which is not the same as the UI, as you know), but I want to keep them away from using vanity metrics. One of such potential vanity metrics, is the CES: https://hbr.org/2010/07/stop-trying-to-delight-your-customers
I actually don’t know whether CES is a vanity metric, so that’s why I’m asking for experiences.

I agree that setting the KPI’s, and assigning actionable metrics, is essential to measure the performance of the product. But I still do not believe that the overall UX (which goes beyond just the UI) can be measured objectively, there will always be the stain of subjectiveness, and context. On the other hand, is it that much of an issue?


#4

thanks for sharing!

I don’t agree here.
According to my experience (corporate intranet, e-commerce and fintech) I was involved in projects where the UX was measurable.
Defining UX KPIs is a good starting point IMO and you have also data to lean on for measuring the UX strategy performance.

We measure UX KPIs such conversion rates, time to accomplish the task, the user requests such the password recovery, user issues addressed to the customer care (call-centre, emails, twitter etc). Those figures provide a clear overview of what are the UX strategy performance.
Of course, you have also other KPIs to add such the environmental ones (eg if the product is shipped in different countries), technical ones (eg if the product is designed for cross-device targets), but this, according to my experience, is not a blocker to measure the UX.

Superstar designers like Luke Wroblewski are pushing a lot this topic (https://www.youtube.com/watch?v=xiyOTcVHqHw)


#5

Again, this is focusing an awful lot on the UI of your product. Which is not a bad thing, since it contributes to the overall experience. But conversion doesn’t necessarily tell you that users had a good experience interacting with your product. Neither is time to accomplish a task. Or at least, these metrics by itself aren’t very good measurements of UX. They have to be set in the context of the bigger picture. Otherwise they become the before mentioned vanity metrics, and that’s what I want to avoid. I’m confident you assured that in your projects, the metrics you used were not vanity metrics, I’m simply curious how you did that? And what are some of the KPI’s that you have been using in your projects? And how do you link them to those metrics you mentioned before?


#6

Some of this is CRO fluff but it adds a couple more potential metrics to the list in your OP. (I’m also a big fan of Peep Laja and I think his ideas are pretty solid.)

https://conversionxl.com/blog/8-effective-ways-of-measuring-ux/


#7

Is not mandatory (at least for me) to find an agreement on that.
I’d like to share my experience here and how this approach had an impact on the overall customer experience(s) for the projects I worked on.

The KPIs we (KPIs have to be defined by all product team not only by the designers) choose were exactly the one I mentioned before.
Again, a lot of designers (you can find a bunch of videos as the one I already sent to you and books) are focusing on that set of KPIs.