Remote, unmoderated usability testing is so fast and easy that some teams make it their only method for evaluating their experiences, but don’t shy away from its more robust alternative, the remote moderated usability test, which can give you more information and is also inexpensive. Let me explain. Remote, moderate and testing’s benefits by first explaining the downsides of remote unmoderated testing. Remote unmoderated testing usually involves using one of the many services, such as user testing, dotcom or user Zoome, setting up some tasks and waiting for the data to be collected. Many teams love it because it’s a lot faster, easier and cheaper than traditional in-person testing. But it has some downsides. Part of the reason why unmoderated testing is so fast and easy is because the testing tool essentially does the job of the facilitator. It administers tasks and instructions to participants, so the research team saves time. But the flip side of that is that you don’t get the substantial benefits of a facilitator. So imagine that something interesting happens during the test. Maybe a participant says something like, well, that’s cool, but then fall silent. If a facilitator is involved, she might ask a probing question. She might say. What’s cool, though? Those simple follow up questions can yield detail and insight. Without a facilitator, you lose those opportunities. Now imagine that the participant misunderstands the tasks instructions without a facilitator, there’s nobody to correct those misunderstandings or to answer questions that the participant might have about what he’s being asked to do. Additionally, without a facilitator asking him to do things in real time, the participant may not feel as motivated to try the tasks.
You may even get some cheater’s people who are just trying to get the incentive money with as little effort as possible. In contrast, remote moderated usability testing actually does use a facilitator so it doesn’t suffer from these same problems, although you still can run across some unmotivated participants in moderated studies. In fact, in many ways, remote moderated testing can run almost exactly the same way as an in-person study. The only difference is that the participant and the facilitator aren’t in the same room. There are two major downsides for remote moderated usability testing as compared to unmoderated. First, you do actually have to have a facilitator schedule and spend the time to conduct each session. And second, you will need to choose and set up screen sharing meetings, software, and that can take some advance time and planning. But that might be worth the extra work in order to have a facilitator running the session that’s more likely to result in higher quality, more detailed findings. And compared to in-person testing, it still does save time in terms of travel for both your researchers and your participants, that can be an advantage when you’re recruiting. It might be easier for your participants to find an hour where they can be at their computer in a quiet location versus traveling into a testing facility. If your team always relies on remote, unmoderated usability testing, give moderated a try.
You might be surprised it could be well worth the extra time.