What research method to chose for my master thesis project?

Process & Techniques
#1

Hi, I’m writing my master thesis and I’m currently structuring my methods section. From a previous pilot study, I have developed an information visualization tool, a hi-fi prototype in other words. I want to further develop this prototype based on findings (user requirements and needs) from some type of user study. But I can’t seem to find an appropriate UX approach for my work.

Alot of UX research methods/processes are based on the notion of identifying problems/needs of users to ideate on, in most cases the said users have previous experience of a software or product.

However, in my user study, I mean to introduce a totally new scenario/situation for the participant (the user). The user group is department managers within a certain organization.The new scenario would imply new responsibilities/tasks in their current work, which they have no previous experience of. This prototype would serve as an example of a solution to address the proposed scenario. I want to gain insights based on this prototype but I’m not sure how to proceed.

Would mean the world to me if someone could nudge me in the right direction. Thank you!

#2

You can always use your prototype to test how users react to the new functions. After they completed the action, you can ask some questions about how safe they felt in accomplishing their new tasks, if they think it will help the to tackle those new tasks.

If you don’t have any prototype for your scenario, you can do a card test for the user to order the hierarchy of steps.

1 Like
#3

What you’re doing is called formative research. There’s a good NNG article about it here:

I’ve done formative research when launching a new product or service. I normally structure it as follows:

Interview

Before you show the prototype, talk to the participant about the scenario. Ask lots of open questions. How do they feel about this new responsibility? How would they expect to approach it? Do they think they’ll find it difficult or easy? Do they have the skills and training that they need? Etc… You’re collecting data about their thoughts, feelings and preconceptions before they see your solution.

One really important thing to do during the interview is not to guide them. Be careful not to ask things like: ‘Do you think approach X [which I use in my prototype] would help with this issue?’ because it will skew how they think about your prototype. Keep the questioning neutral and open. In commercial UX, this interview is often done by someone who didn’t work on the prototype, to ensure neutrality.

Usability test

Introduce the prototype as one possible solution to this scenario and ask them to use it. Get them to talk about their experience as they use it. Again, keep any questioning neutral and be careful to avoid leading questions like: ‘Did you find that process easy to do?’ (planting idea of easy-ness in the user’s mind).

There’s lots of great articles about usability testing on NNG, like this one:

In my experience, this combination of interview and usability test works really well to gather the information you need quickly in one focused research session. You normally need to allow 1 - 1.5 hours per participant, and 5 participants is the ideal number to get a good range of feedback.

I hope that helps!

3 Likes
#4

Thank you! The scenario is more of a situation that hey have to consider in their current work.

#5

Thank you alot, that made it much clearer!

#6

@AMG Since you are launching a new product from my understand first understand the problem and write down the problem statement. Your problem lies in the gap between what is already existing and what you intend to do (basically, where you want to go). There is a gap in the present and future and your answer to the problem lies in this gap. Once you understand the problem, then think if you want to go about with a deduct model or an inductive model (if you want to deduce the hypothesis or induce hypothesis). Based on this you can decide upon the research methods and go further with your thesis. Without analysing the problem you can get to prototypes. It is analysis>Design>Prototype>Evaluation ( where you do usability tests). That will be my approach to tackle your question.

1 Like
#7

Hi again, I’m planning my usability study, and currently writing user tasks for think-aloud. This got me thinking about what method I should use to analyse the recorded data. The type of data collected (audio recordings, I guess) and the user tasks have to be supported by the analysis method. I’m still a bit confused what method/s I should use for analysing the qualitiative data…

Thanks

#8

I’ve generally collected screen recordings of what the user is doing alongside the audio of them talking. On a Mac, you can do this with QuickTime (File > New screen recording, check that the mic is set correctly to capture what the user is saying). I’m sure there are plenty of similar screen recording apps for Windows.

In terms of analysing the data, there’s a good article about this here:

The data you collect is basically screen recordings of behaviours and user comments, which you can then collect together into themes and report as conclusions. Here’s an extract showing what this looks like from a real report that I put together a few years ago:

2.0 Search

2.1 Two-step approach

[2 x screen shots of tester doing broad search and then refining results]

When searching for events and projects, testers typically used a two-step approach:

  1. A broad search to start with (e.g. for all projects in London or all Education projects)
  2. A scan of the results followed by a refinement of the initial search

Testers felt the presentation of search results would be very important in helping this process to work smoothly. The list of search results should contain the key information required to allow results to be assessed.

A range of filtering and search refining tools are seen as very important. Reaction to the search refining tools in the left hand menu of the prototype was very positive.

“It’s great that you can filter results like this. I really like being able to choose a place and a topic to cut down the results list.” (Tester LE)

2.2 Searching by time is important

“I’m not interested in projects from a few years back. I need to see the projects that are happening now.” (Tester CF)

Searching and filtering by time was seen as an important feature. Testers wanted to see which projects were currently most active. They also wanted to be able to list events by date, including future dates.