Participants Needed: Online-Survey about Severity Ratings of Usability Problems


#1

Hi Community,

For my master’s thesis I am looking for participants for my online survey about severity ratings of usability problems: https://www.soscisurvey.de/severity-ratings/
The survey investigates how the severity of usability problems (websites, apps, …) can be perceived and assessed in different ways.

May I ask you to participate in my study?
Everybody can participate, no particular level of experience with usability is required.

It should take approximately 45 minutes to complete the study. Of course, it is anonymous and confidential.
Your input will be very useful for me. I am looking forward to your expertise and feedback.

Thank you very much in advance for your help!
Nina


#2

Hey Nina,
Welcome to the community. I can tweet that out to our audience for you. I confess that you may have some trouble finding takers, as 45 mins is a fairly big ask. Worth a try though!
What is your thesis about?


#3

I agree Nina. 45 minutes is a massive ask for people in this age when everyone is so time-poor. Unless you’re paying them, or there’s some big drawcard at the end (“go into the draw to win a car!”) I think you’ll struggle to get reasonable numbers. Revisit the questions and pull out any that are nice-to-have. Just keep the most important ones.


#4

Thank you!
My thesis is about the effect of different scales and training on severity ratings of usability problems. To see if simple scales or more “elaborate” scales affect rating accuracy, inter-rater agreement, etc. It is such a broad and interesting topic.

Thank you for your input, unfortunately I cannot change anything about the study anymore. So, let’s hope there are a few who are still willing to participate. Thank you for your support so far, I really appreciate that :slight_smile:


#5

Hi Nina,

I understand what is like trying to get research for your masters, and having to go through an ethics committee to get things approved. I also understand that your supervisor liked your questionnaire and thought it was good.

I tried to go through and answer it (for a fellow community member), but I’ve actually given up. I do not feel like the response I give is actually going to be very helpful at all. A lot of the ratings you are asking us to give are making huge assumptions.
We don’t necessarily know who the target audience is for each of these things.
We also don’t know what the workflow around what they are trying to achieve is, or how often they use it etc.
We don’t necessarily know the cost of fixing all the usability issues.
The static examples are harder to evaluate as you don’t necessarily know how they work i.e. in the first one, how does the person actually change their details? Single screens aren’t just the problem with usability, it is behaviour as well, as you know as you have asked about behavioural problems.

The scales of very low - very high also do not necessarily translate well to the factors asked.

I very highly suggest you go back to your supervisor and suggest changes, as with this length of survey, you want to at least try and get valid results, which I’m not sure you will get with this current one. You don’t want to use up your pool of participants on results you can’t use, and then have to create another survey that people will not want to do a second time. I would question your supervisor as to why he wouldn’t let you shorten it, as for the results to be helpful you will need as many participants as possible. Is there a reason why you need it to be so long, a love of the questions is not enough to risk participants.

Did you test the questionnaire before posting it out there? what were other people’s feelings around answering it?