Hi All,
Do you work at SaaS or Enterprise kind of companies? Or at least companies that produce a software that is used in other businesses?
If you do, and if you use analytics, what do you measure there? Obviously, not the bounce rate (because it does not give much data). If not this, then what?
I work for a company that produces a software product (this one) that is used in other businesses.
I’m confused by your question though. Are you asking if we measure how our customers users use our product?
Bounce rate would indicate web analytics, right? Are you asking about measuring teh website’s success or something else? I am also a little confused.
Yes, kind of.
Do you measure anything? If you do, what it is? I don’t mean measuring the marketing website but the tool itself.
In more detail:
The tool is used for project management. Companies buy the tool to manage their projects more easily. The tool is purchased by ppl at a top level and used by simple PMs, who rarely have impact on the decision. They use the tool every day, they rely on it, and they don’t have much of a choise - all this affects the behavior.
The reason for my question: I was given a task to clean up the analytics (Piwik / Matomo), which was managed by the previous UX Designer. I entered the account and got mesmerized. The tool has many options. I know what I want to achieve, but I’m open to any other hints.
I definitely want to see the usual user flow. But is there anything else that I could measure here?
I’m still confused TBH.
We build the product and we make our money by hosting it (it’s open source) but we don’t collect data around how it is used – that is up to the organisations that use it.
e.g. Acme Corp have a Discourse community which we host on our servers. They can choose to track user journeys by integrating GA or they can pull data via the API into segment.io (just a couple of examples) but we don’t access that data.
To clarify what I think @mag_sobieszuk is asking:
When you work at a company that provides software or services that other companies use, a lot of the traditional measures of UX success go out the window.
Take for example the product I currently work on. I’m a designer for a site that works as a stock trading and client management platform for registered financial advisors. We also serve as a custodian, meaning we hold onto our clients’ money for them.
The way we gain users is by signing up advisory firms to use our service. This is done entirely through our sales team, and while great UX design certainly helps, there’s no organic way that we attract new business. Someone can’t come to my site and just sign up to use it - you must be part of a firm that has agreed to buy and use our services.
The upshot of this is that you have a captive userbase that’s forced to use your product.
This means that many of the usual metrics you’d use to measure UX success - particularly in e-commerce situations - go entirely out the window. Some examples of common UX metrics we don’t track include:
So now, hopefully, you can begin to see a bit of the problem from a UX metrics perspective. Most of the metrics you might track have more to do with your sales team getting out there and landing key accounts than your skill as a UXer.
On the other hand, a well-designed product makes the sales team’s job way easier.
My advice: ditch the metrics analysis and spend your time doing as much testing as you possibly can. . It’s generally a much better way of spending your research time.
Personally, instead of focusing on metrics analytics, I spend most of my research time doing moderated and unmoderated testing. It’s much more useful for me to focus on whether or not a new or potential feature is easy to use and provides the required value to the user.
When I do analytic evaluations, they more focus on the task success rate for new features. The how-to behind that is much more in-depth than there’s scope for in this post, but suffice it to say that some of these metrics come back into focus on a micro-level when pertaining only to one page or a single flow.
How do you know that the tool meets requirements or needs of the users? Or do you not care about it?
By word of explanation, what people do and what they say are two different things (obvi). We do user research, surveys, questionnaires, etc but I need to know their actual behavior within the tool. A simple fact that they’re confined to use the tool, doesn’t have to mean we stop caring about user experience.
@dougcollins
Thank you a thousand times. As always, your help is priceless.
You got it right when it comes what I mean by metrics.
But also, I wanted to explain WHY I want to introduce metrics at my company.
The tool is old-ish to say the least. It sometimes uses old technologies that dev team refuses to work on (such as JFS). In general, the old tech, the old look, and non-intuitive and counter-logical behavior of those parts of the system are pain in the butt. For everybody at the company, including Sales who try not to show those functionalities that are ugly and not the best working.
Long stroy short: we’re in a very deep UX debt (and technology debt). It’s time to get out of it. It’s time to clean up the featuritis that has grown over the years. For that reason, I need to know which parts of the system are used, which are not. I need to know how people interact with the tool, how they use it - and understand it. I cannnot rely only on what they say. Our clients are hundreds of companies, and each is uisng system differently. They are spread out across the world and I have limited access to them. I can only do remote tests, provided their bosses agree that I take out 1 hour of their working time. This is a hard situation and to ease it a little bit I wanted to use Analytics Tools that I have at hands.
Anyways, I recon enterprise tools do not really measure anything. Or am I wrong? If you do have metrics applied, what do you actually measure?
Hi, this may not be the answer you’re looking for, but here goes.
I was in a similar situation and what worked was auditing the actual tools and basically setting the metrics from the ground up. If analytics tools don’t work, replace them.
Good luck, hope this helps!
PS: as always, aim for the low-hanging fruits and easy wins first.
Hi there, I don’t have that much experience in this so thanks for creating this topic I’m learning a lot. I would focus on:
As I said, don’t have that much experience in this area yet so I hope this helped.
There are at least few pieces of valuable information. Thanks a lot!
Audit is what I do right now. 
I have drawn out the whole (I hope) information architecture of the tool - an imperfect AI, I must say, but the tool is too big, and my goal right now doesn’t need it detailed right now.
I want to analyze how many hits each of these pages gets.
And then I think I want to ask our Users whether the data is accurate and they don’t use some of the features, and why they don’t use them, and how I could make them better to make their work more effective.
And then there’s this business part, which would probably mean going to the CEO and convincing him we should shut down some of the functionalities, and most desirably - change the technology and redesign few of the modules, which need new design from scratch. If it doesn’t work, then I’ll take @mr_dugi’s recommendation to simplify or flatten the AI.
What are your thoughts, based on the info you have?
We certainly care – and many organisations that use our tool do testing and pass on information.
We also seek feedback from our (open source) community and implement changes based on that info.
Hello!
It might be too specific vs the needed whole picture now, but maybe it could help. Is there a way to check the form filling quality? There are applications which have forms for actions like opening a new task or record some changes. Unused or unrecognizable elements or even the whole forms could be found and investigated.