The challenges of remote testing

This excerpt is Chapter 10 of Remote Research by Nate Bolt and Tony Tulathimutte. For remote user research tools and detailed articles, also see

You’ve seen how remote research deals with the problems of traditional in-person research (geographical distance, recruiting, task validity, etc), but it raises plenty of its own problems, too. We’d like to wind down this discussion with a review of the biggest challenges of adopting remote research methods: the doubts, concerns, and pains in the neck that seem to come up in study after study, even for seasoned practitioners.

Swipe to scroll horizontally
Screen sharing is interrupted/malfunctions

Check to see whether your internet connection is stable. Check to see whether your user's internet connection is stable; if possible have him/her switch to a wired connection. If it's still not working, try a different screen sharing tool.

Recordings come out corrupted/glitchy/truncated

Test the recording tool. If test recordings don't work, check recorder settings to ensure that they are recording to the correct format and quality. If test recordings work fine, most likely the computer was running too many processes during the recording. Close down unnecessary programs to fix this problem; if it persists, you may need to upgrade your computer with more RAM.

Also check whether you have sufficient hard drive space to store the recordings.

For corrupted files, use a video editing program or converter to attempt to convert the file to a different format. For certain file formats, there are also utilities that are able to fix minor problems.

Phone connection malfunctionsCheck your phone connection. Use an alternate phone line, if one is available. Ask users if they're using a cell phone. Ask if there is an alternate line to call. Ask users whether you can call back on a different line, at another time if necessary.
Microphone headset/sound input malfunctionCheck whether the headset is muted. Check the mic input volume in system settings. If you're using a VOIP service like Skype, check the software settings to see if it's not muted.
Internet connection seems choppy or breaks

The problem could either be your connection or the user's.

If it's yours, postpone the study and switch to any alternate internet connections you may have in your office. As a last resort, call your internet service provider and see if the service has gone down.

If it's the user's connection, ask the user if he/she is on a wireless connection; if so, ask if he/she is able to switch to a wired connection. If that doesn't work, attempt to reschedule the study to a time when the user will be at a different computer.

User's firewall does not permit the screen sharing tool to functionSwitch to an alternate, preferably browser-based screen sharing solution. If no available solutions, attempt to reschedule the study to a time when the user will be at a different computer.
Recordings have no soundCheck the system sound input volume and settings (make sure it's not muted) and recording software settings.
Swipe to scroll horizontally
Your website's traffic volume isn't high enough to bring in six qualified recruits an hour

Increase the screener display rate if it's below 100%. Place the screener on multiple pages or a higher-level page in the IA. Schedule qualified recruits in advance to supplement the users you're able to obtain. Increase the incentive, but not too high (or else you attract more fakers).

Lengthen the duration of the study (with healthy traffic, it's possible to do about six users in a work day.

Your recruiting criteria are too strictIf you're filtering your results, disable the filter to see if any of the filtered recruits are acceptable participants. Ask stakeholders if any recruiting criteria are negotiable and relax the lowest-priority ones. Increase the incentive.
The wording or length of your recruiting screener turns people offRevise the wording to feel less like a deal or an offer. Omit needless words and questions. Be specific about the incentive.
Fakers are filling out your recruiting formReview the "Why did you come to this site?" responses to determine whether the fakers were referred by a deals/bargain site. Add sneaky questions to the screener to trick fakers into tipping their hand. Add open-ended questions that can be answered plausibly only by your legitimate recruiting audience.

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

*Read 5 free articles per month without a subscription

Join now for unlimited access

Try first month for just £1 / $1 / €1

The Creative Bloq team is made up of a group of design fans, and has changed and evolved since Creative Bloq began back in 2012. The current website team consists of eight full-time members of staff: Editor Georgia Coggan, Deputy Editor Rosie Hilder, Ecommerce Editor Beren Neale, Senior News Editor Daniel Piper, Editor, Digital Art and 3D Ian Dean, Tech Reviews Editor Erlingur Einarsson and Ecommerce Writer Beth Nicholls and Staff Writer Natalie Fear, as well as a roster of freelancers from around the world. The 3D World and ImagineFX magazine teams also pitch in, ensuring that content from 3D World and ImagineFX is represented on Creative Bloq.