Sell benefits of User Research and Usability Testing

culture

#1

I’d like to know about the various ways a usability consultant can sell/pitch the benefits of user research and usability testing. The startup team does not have access to end users. There isn’t a UI designer and the requirements are gathered through the following steps:

  1. Usually the technical team meets few prospective customers and shows them a demo. Then, their wish list is noted and prioritized.
  2. Compile a list of the requirements shared by the marketing folks and prioritize them.

Have you faced similar situations before? Suggestions and tips would be much appreciated.


#2

I suspect this is something that lots of people face, and would probably make for a decent article on uxmastery.com. Before I jump in with my own ideas, I’d love to hear from others. It would be great to collate them and create an article from people’s tips.


#3

Technical team’s decisions are influenced by logic so I have a feel that quantitative results might influence them. The technical team creates the UI screens with emphasis on functionality. They want an UI designer to make the screens look *pretty."


#4

Last year, I saw Will Evans speak on ‘sensemaking’ at the Cynefin Melbourne meetup. One point he made about ‘Build > Measure, Learn’ is that UX types look at it and ask: build what? He then went on to discuss the alternative approach of 'Think > Make > Check.'
The difference is not just semantic. How do we know what we’re building before we build it? In the example you gave, how do we verify that what a client wants actually solves the problem that they have? What is the problem? Perhaps a better metric for a logical, data driven technical team to track is which features are being paid for up front by a client? (Check out Steve Blank’s The Four Steps to the Epiphany)
To me, the problem with quantitative data is that it backwards looking - we don’t have it until something’s happened, or, until something is built.
When I’ve pitched the benefits of user research, I began by giving the 5 Whys treatment to colleagues and other internal stakeholders on specific features. But before I get to the product, I start with something more existential: What are we doing? Why are we doing it? Once people have done some looking inward and consider their own context, they usually find it easier to apply the same process to their end users.


#5

Hi

I see a similar relationships every day. I’ve worked for a lot in Graphic Design (print advertising), and often the designers don’t meet the customers,
only the Sales Reps do.
The customer asks for something specific that they like, but don’t necessarly need. Then pressure is put on design to be compromised in favor of “Giving them what they want”.

Its ‘The Customer is always right’ scenario.

What has to happen, is the design team in any area graphic/ux etc, must to take the lead. And really push for answers to the questions that will get to the best solution.

What I’ve always found is [B]‘The Customer is always open to meeting the person designing for them’[/B]
And meeting them is best chance to probe and find out what problems are the looking to solve.

P


#6

My two recommendations for instituting a user research culture:

  1. Start with the bottom line: If you want to pitch to decision-makers about usability research and testing, you need to demonstrate business value (whether it’s in a startup or a huge, hierachical workplace - both of which I’ve worked in). I’d look at it from the product manager’s or even the CFO’s point of view. Their ultimate goal is to maximize profit. So the more you can show that usability affects revenue (getting people to like and engage with your product for the long term) and expenses (small iterations now instead of large overhauls later when it’s too late), the better your case will be. If you start googling, there are tons of case studies where user testing/research has proven to be a strategy for profitability, not just usability, so I’d suggest selecing a few relevant ones to your industry for presentation to The Elders of your company.

  2. Just do it: ask for permission to just start VERY small and come along to a couple meetings or calls with potential users. Write a very concise script and have whoever usually does this see you doing it, or record it if that’s possible. It might mean a bit of extra work for you, but it becomes a much easier sell when people actually see how it’s done in reality and and feel the “aha” moments themselves.


#7

Hi Hammer, Paddy, and JollyZ, Thanks a lot! Sorry for the delay in responding.

A quick question. To evangelize the benefits of ROI, what are the UX metrics usually set and what is the benchmark for UX metrics? Are they set against the existing competitors?


#8

Hi arunjm: To measure against your competitors, you would need access to their data to make a comparison. You have access to your own data, so what you are measuring is the difference between the performance of something for a given period of time before and after you intervened. A good metric is actionable - so even before you measure anything, you need to know why you’re measuring it, what result is a pass or a fail and what action you’ll take as a result of that data.

​But that’s only half the story. ​JollyZ is right - you need to connect those results to the goals of the business. And your CFO doesn’t care about CTR. She cares how much more $$$ it brings in. So while the raw results of an A/B test on, say, the CTR of a product category given its location in a list might tell you how high in the list it should be, to your money person, it won’t be clear what that means in dollar terms.

To borrow an example made by Carl H from Country Road (who spoke about conversion testing at Melbourne Geek Night a few weeks back), it’s all about 2 things: 1: Just doing it (by getting friendly with some devs) and 2: framing the results in terms of how much the company will [B]lose[/B] over 12 months without the change. If I remember correctly, Carl noticed that people who navigated to the shoes tab often made significant purchases - but the tab itself was buried in the list. He A/B tested the position of the tab and confirmed his hunch that putting shoes closer to the top of the list would make increase the profitability of those sales.

When he presented those findings, he extrapolated his data from the fortnight he ran the test out to a year. Instead of saying how much the the company would make, he showed how much money the company was leaving on the table by burying such a profitable area of the site.

To come back to your original question: instead of evangelising on what you could do with your data, consider the data that you have and what you’re trying to do. Is acquisition a primary concern? What are your conversion funnels for signup? Are people signing up, but not doing anything? How are you measuring activation? How regularly are users logging in? For how long? What do you know about your users?

In my opinion, there isn’t really a playbook for ‘UX metrics.’ It depends what you and your business are trying to achieve. If you haven’t watched it, Dave McClure’s Startup Metrics for Pirates AARRR! is a great macro view of everything(?) that matters to a business that’s growing. Also worth looking at is [URL=“http://leananalyticsbook.com/”]Lean Analytics by Alistair Croll and Benjamin Yoskovitz. They’re main concern is identifying ‘the metrics that matter.’