*Remote* mobile user research

For me and probably for others, it’s been a how-has-this-not-been-figured-out type of pain point for UXers who want to conduct usability tests with users REMOTELY while being to:

a) see what they are doing on their screen
b) see what gestures they used (bonus)
c) record what they do on their screen on video (bonus)

What trips up the process seems mostly to be the amount of installation work the user has to do. And my philosophy is also to reduce the prep a user has to do since they are helping you out already.

I tried Reflector, but it seems to be very buggy and hard to install. Plus, it’s only good for iOS and requires the user to be connected to their computer.

I’m sure there has been progress made on this, so I’d love to ask the community what they’ve found to be the best solution(s). Thanks!

Jolly

I have never attempted it, but I wonder if a Google Hangout would be able to capture some of that?

JollyZ I wrote a review of an app called MagiTest a while back, which may be worth trying? It doesn’t require a computer connection, but is iOS only. Also getting the recording off the phone is likely to be a hurdle. But it’s something!

UserTesting.com recently released a new capability to perform remote unmoderated usability testing on mobile devices.

I do a lot of multi-channel strategic UX work, and I spent a lot of 2014 trying to figure out a solution for remote moderated user research on mobile devices. As far as I know, there aren’t any really great solutions for this yet, but there are passable solutions. Cross your fingers that a vendor figures this out soon!

I’ve been using a solution that is only good for fairly tech savvy participants. We tried it with “average” consumers and had a lot of headaches. We tried it with very tech savvy young affluent males and it worked ok, but we had to pay them for the tech setup (extra incentive $50) and over-recruit knowing we would lose some (they get frustrated about how long it takes). that’s the caveat.

For iOs, you use airplay to mirror the participant’s tablet or smartphone to their computer monitor. Sounds like others have used Reflector successfully. I use Adobe Connect to screen share so I can see them & their mobile device. Their webcam (not the device) provides an image of their face. You can see a perfect image of the screen, but you cannot see their fingers touching the screen. It doesn’t work on all iOs devices (older ones, I think, are the issue). Since I’m doing strategic UX and not usability, I opt for seeing their face; if I was doing usability testing, I’d be more interested in seeing their fingers. For Android, there’s a cord involved and some temporary settings changes. It doesn’t work on all Android devices. Setup is more complicated than with iOs, but it works well once it is setup.

Another way to do it is an iPevo camera (or some other document camera) pointed down at mobile device – that way you can see the participant’s fingers on the screen but you can no longer see their face because you cannot use the document camera & a webcam simultaneously. It’s a compromise – what’s more important: finger action on screen (use iPeevo) or user’s face (use iOs mirroring & android cord solution).

I’ve also heard recently from people who are using Team Viewer & Bomgar, but they seem expensive and not very interested in working with researchers; those tools are designed for remote IT support.

None of these are ideal setups, but as far as I know they are the best we’ve got so far. I have clients who need a robust geographic mix for multi-channel studies, and this is the only way to do it without breaking the bank with travel. However, given that you can only use these approaches with fairly tech savvy participants, it may be wise to use these “remote” solutions as part of a hybrid study design that includes in-person interviewing. There’s a real risk of skewing your results by skewing towards tech-savvy participants who can handle the setup (unless tech savvy is your audience anyway)

I’m no tech guru so I brought on Civicom to help me with the tech support; they handle tech checks" with participants. I wouldn’t try it without a vendor to handle the tech setup – it’s a massive time suck to help participants setup their devices for this. and again, we learned that we need to pay people to go through the setup process and over-recruit.

Hope this is helpful. good luck! and please let me know if you’ve come across any other solutions! I’m presenting on this at UXPA in June and I’d love to hear what solutions other people are using.
Julie
Founder, BellaVia Research