Which rating scale do you use for Severity Ratings of Usability Problems?


#1

[SIZE=15px]For my master’s thesis I am interested in which rating scale you use for rating the severity of a usabilty problem when you apply a heuristic evaluation or an expert review. Jakob Nielsen’s four-step scale (0 = "No usability problem at all ", 4 = “Usability catastrophe” ) often seems to be the standard, I am interested in whether that is really true in practice.

Do you use Nielsen’s rating scale? If not, which other approach do you use, and why?[/SIZE]


#2

Welcome to the community @nabaumga

Here is a great article outlining and comparing the most common scales. Personally, I think that Nielsen’s is the simplest and easiest to understand/apply.


#3

Honestly, I don’t believe it’s that important whether to adhere to any kind of standard. The most useful information for fixing usability issues is qualitative (why is it an issue?), not quantitative, so understanding the context and the nature of the usability problem is more important than what number it gets when you find it. Sure, I get you have to put metrics around things sometimes in order to plan the effort to address them, but being consistent in how you rate them is the main thing.


#4

I’ve done user testing last year, and simply rated problems as 1-5, minor to serious. But looking back the consequences of not fixing the problems should be highlighted more that giving it a rating.
For example instead of saying “5 major problem” It would be better to describe the issues. “Users couldn’t find the advance search options”, and the consequences "This means that our products will be invisible to the users"
paddy


#5

Thank you for your feedback :slight_smile:


#6

I am interested in one more aspect: which factors of a usability problem do you consider as important when coming up with the rating? Factors like frequency? Impact?