When doing usability testing, I recommend testing with a very small number of test participants, typically five, because when you’re testing with the very first test user for sure, everything you see will be brand new and interesting. And that’s why we do user testing in the first place to see how our customers actually use our design. But when you test with the second person, a lot of what you see is going to be the same as you already saw with the first person.
Now, of course, people are different and different people do very different things. There’s also going to be a lot new, you see, with test user number two when you test person number three. Well, some of what that person does is going to be the same as person one, some of what user three does. It’s going to be same as user two. And some things you are now seeing for the third time when you test using them before there’s going to be even more overlap with what you already saw and even less new, but still kind of an OK amount of new information for use of five relatively little new information, a lot more overlap. And by now it’s getting to be pretty boring to sit through this.
Testing and testing with more than five users is typically pretty wasted because you’re not going to learn very much more from those subsequent users. You’re much better off stopping your test. They’re taking all the information you’ve learned and using that to improve the user interface design, make a new version, a new iteration, and then test that with another group of five users. Take that information, improve the design again, test a third iteration with five more users and keep going like that as long as you have budget. So if you have budget for, let’s just say, 20 users for a certain design project, the best comes from splitting those 20 people up into four tests of four different design iterations, each tested with five users. Right. And spending kind of all of your twenty test users on testing one design iterations really well, really thoroughly. Yeah. You’re going to get a little bit more, but not much more. And you’re going to get much more improvement of the design, which is, after all, the reason we do user testing in the first place.
You’re going to get much more improvement by driving up that quality of the user experience through four test iterations, each tested with five people. Now, there are some types of research where you want to test with more users, like quantitative usability studies, where the goal is to derive metrics there to get good statistical significance. You need a much larger and a much larger sample size. But here the goal is a better number, not better insight. You don’t get better insights with more users, just a better number.
Also, methods like a card sorting a certain types of eye tracking require a larger number of users. But far most have standard normal types of usability testing. But the goal is to drive up the sign quality. This is more less users in each round of testing means you can do more rounds of testing, more design iterations within your budget. That means more final improvement, more total improvement and therefore more profitability of that resulting user experience.
Imagine that you’re writing an email to a large group of people, you need to send roughly the same email with a few small changes, you could write a brand new message for each person or you could reuse most of the same message. To do that, you could highlight the similar text, click, edit in the top menu and then copy. Maybe you could highlight the passage, right. Click and choose copy or for maximum quickness, just select and command that simple copy action can be successfully carried out in a number of different ways. It’s a flexible and efficient process.
Heuristic number seven of the 10 usability heuristics deals with this idea. Flexible processes can be carried out in different ways so that people can pick whichever method fits them. That flexibility helps people be more efficient. If a system lets me choose my method of interaction, I can get things done in a way that’s quick and comfortable for me. That keyboard shortcut, the handy command see is an accelerator. An accelerator is any option or action that speeds up an interaction. Accelerators are there for more experienced users who know a system well and want to make routine tasks go by more quickly. Highly usable systems are flexible enough to be efficient for experts and friendly to newbies. It’s important not to overwhelm first time users with too much information. Don’t try to teach them every action on the system in the very first session of use. That’s too much to learn all at once. Accelerators are often seen in the wild in work and productivity related applications. If you work in design, you’re likely familiar with the many keyboard shortcuts that help you work in your interface design program of choice more quickly. If you work extensively with data analysis, you’re likely very familiar with macros in Excel or Google sheets. But accelerators can show up just about anywhere. For most of us, Instagram isn’t a work related product, but there’s still a hidden accelerator there. If you’re a frequent user of the social media app, you may know that you can quickly like a photo by tapping twice on a photo that’s quicker and slightly more efficient than finding and tapping the heart icon. When the seventh heuristic is done, well, expert users won’t be slowed down by a system that’s only built with basic use in mind. New users won’t be overwhelmed by an information overload of actions to learn.