Moderated usability tests

Moderated tests by nature involve more risks. They take more time to organize and mistakes are more costly compared to unmoderated tests. However, you're safe if you know these pros and cons in advance.

Pros of moderated usability tests

1. Better test quality 

With moderated usability tests, users are less prone to quitting, staying confused or silent, or getting distracted. Moderated tests are well planned ahead, supervised, and involve deep discussions.

2. Tests are under control

Under the moderator's guidance, users are unlikely to get sidetracked off the course. At any given moment, the facilitator can emphasize what is most important, adjust, switch devices, and react to technical problems.

3. Questions & discussions

The biggest advantage of the moderated test is the ability to follow up with questions. You can ask questions during the test as well as at the end to wrap up. Simply asking “Why do you think that is?” can uncover valuable insights.

4. Test paper prototypes

Moderated in-person tests bring the opportunity to test low-fi paper prototypes. These prototypes can be easily explained to the user as well as handed out and conducted.

5. Discover unexpected issues

The facilitator’s presence supports the spontaneous discovery of usability issues. These issues can be discussed immediately or left aside to test at another time. 

Cons of moderated usability tests

1. Time-consuming

Moderated tests take time to organize. For a start, there are at least two people to coordinate. In addition, time and place should be agreed upon, tests planned, equipment set up and tested, and participants briefed. All of which require a notable time investment.

2. No-shows

Considering the effort it takes to organize moderated tests, it’s costly when users don’t show up. Imagine, you show up 5 minutes early, wait for the tester for 15 minutes assuming they are late, and realize it's a no-show. 

3. Testers are too nice

Human presence advocates for less honesty and more politeness. An issue that might lead to testers not being straightforward when they should be. Which in turn affects the quality of test findings.

4. Tests are less streamlined

Moderated tests are kept in structure by the facilitator, not the tool. They are usually less standardized and more loose compared to their unmoderated counterparts. The lack of a step-by-step process might result in inconsistent findings or a complicated analysis.

Pros of unmoderated usability tests

1. More efficient

Unmoderated tests are a shorter, more cost-efficient alternative compared to moderated tests. It takes less time to coordinate appointments and recruit participants. Most tools also offer helpful automation, flexibility to run tests whenever convenient, and more. 

2. Asynchronous testing

Time and location are not an issue. Basically, you can schedule tests with participants across the world anytime. Running tests at your own pace and schedule is a big plus, especially if you need to run them immediately.

3. Natural user behavior

While at home, users are more likely to act naturally. Doing tests from their own computers and state of comfort often makes them more open to sharing their honest opinion. This is a hidden advantage of unmoderated testing.

4. Easier recruitment

Most usability testing tools feature extensive databases full of testers. Users can be screened and picked based on demographics, interests, and even their ratings.

Cons of unmoderated usability tests

1. No follow-up questions

Unmoderated tests don’t allow in-test discussions with participants. The format usually fails to remind users to think aloud and therefore leaves researchers relying on their hunches. This is why Solid has created a friendly avatar that reminds users to voice their thoughts.

2. Getting to know technology

Unmoderated tests are done with the help of a usability testing tool. Every tool on the market takes time to learn and has its pros and cons. The inability to use tools correctly, setting them up the wrong way, or having testers install recording software might cause a number of issues.

3. Lower result quality

Exploring the interface and completing tasks on their own makes testers more prone to confusion. Users often give up, get stuck, or misunderstand what they are being asked to do. So, having no moderator can influence the overall quality of test findings.

What method should you choose?

First, it depends on where you or your company stand and what you are trying to achieve.

If you work fast-paced, the remote unmoderated might be your go-to method. If you are looking for in-depth results and are an experienced moderator, then you know your first pick.

Since unmoderated tests are usually less time-consuming, we recommend them in most cases - especially if you are new to the world of usability testing.

Spend that time you save with unmoderated testing to run more tests. Instead of typically running with 5 testers moderated, double down with 10 testers unmoderated.

Just try it out - you will be surprised to see results with little to no effort!