The usability testing tool marketplace is booming—every day there’s some new service or app that allows you to plug in your tasks and run a test on a panel of participants anywhere in the world. You can get hundreds of responses for quantitative research or watch video recordings of a handful of participants taking your test and thinking aloud into the camera. With all of these options, why would you still usability test the “old” way by moderating your own test with five to 10 participants from your user base?
Here’s why:
Moderated usability testing brings you closer to the user.
You can dig deeper into issues and adjust the test as you go.
It’s easier than ever to remote usability test.
Moderated Usability Testing Brings You Closer to the User
When was the last time you actually talked to your website’s users? Not the business stakeholder or the client, but the person who visits your site or uses your tool, or who is a prospect in your sales funnel. Even if you did thorough user interviews and research at the start of the project, chances are it’s been a while since you’ve checked in. Inviting these people to participate in a usability test is a good way to make sure you’re on the right track, and to uncover unknown needs or unexpected usage patterns.
Another benefit of moderated usability testing is that you can control the quality of the participant panel. With some tools that provide panels, you don’t get to build a relationship with the participants, and worse, sometimes you find out after the fact that they weren’t exactly the right fit for your product. When you recruit the panel, you can curate your own list of high-quality participants. In the future, you can return to these folks for additional rounds of usability testing, user interviews, focus groups, and more (especially if you were generous with incentives and giveaways).
You Can Dig Deeper Into Issues and Adjust the Test As You Go
Perhaps the biggest UX benefit to moderating usability tests is probing more deeply into the user’s behavior or motivations. Why was the task so easy? What made it difficult? The user may also spontaneously admit to using the product a different way or reveal a use case that the team did not anticipate.
Moderating also makes it easy to employ one of the best usability test questions of all time, which is asking the user—before clicking—“What do you expect to see after you click that?” This question is sometimes followed by “So that isn’t what you expected to see. Tell me more…” when their expectations were not met. This line of questioning is often very insightful, yet difficult to reliably capture in an unmoderated tool.
The ability to dig deeper is critical when you don’t have frequent feedback from users. If you’ve never talked to the “real” users before, or if it’s been a while, you can use some of the test session to interview them about their needs and wants for future features, or get their reaction to something you have in progress. You are not limited to standard usability testing tasks or protocols.
Sometimes, a task flops. Perhaps the first few users are confused by a piece of functionality. In an unmoderated test, you can’t adjust your study on the fly. But when you’re the moderator, you can make changes. For example, I had eight usability test sessions lined up for a product, three on the first day and five on the second. The first three participants all had problems successfully completing an important task. That afternoon, we regrouped as a team, and based on what we’d learned from the first three participants, I made some tweaks to the prototype for the second day of testing. Rather than watch a task fail five more times, we were able to fix the issue during the test, then test the solution. This is impossible without moderation.
It’s Easier Than Ever to Remote Usability Test
The good news? You don’t need a usability lab or fancy recording software to moderate usability studies. You don’t even need office space. Video/phone conferencing apps like GoToMeeting, WebEx, and Zoom all offer the capability to have a voice or video call with screen sharing. You can even record through these services or have silent observers dial in to take notes. Though remote moderated mobile device usability testing is still difficult, companies like Lookback.io are building tools to meet that need too.
Finding participants is also easier than ever by reaching out via social media posts, blog posts, email newsletters, and even callouts on your website itself. There are also many services that help you run website intercepts to invite your site visitors to volunteer to be part of a participant panel. Gone are the days of contracting with a market research recruiting firm to find users!
Value in the Traditional Approach
So, the next time you’re thinking about running usability tests, don’t forget to consider the old standard. Moderating usability test sessions builds empathy with the user and enables you to directly observe their behaviors and listen to their thoughts. The value you gain from the experience is reason enough to give the traditional approach a try.
