Formative vs. Summative Usability Evaluation

Have you ever been asked why you want to test with five ease’s and why are you not testing a product with hundreds of users and giving them a survey after they’ve used a product? Won’t that be more accurate? Your clients or coworkers are asking you in order to really explain to your team or your clients why you’ll do one and not the other is down to a distinction between what is called formative and summative usability evaluation.

They both have different goals and as a result, there are differences in how and when you would carry out each type of valuation. Formative evaluation tells you what aspects of your design work or don’t work and why. While summative evaluation tells you overall how usable your designs are compared to your competitors or to a given benchmark. Let’s talk about both. There are many types of formatter valuation methods, such as heuristic reviews, cognitive walk, Theroux’s. But the most common one is your bread and butter usability testing that we carry out throughout the design process. We create tasks for a user to complete on a design. We ask the user to think aloud and we observe the user’s behavior and explore when they struggle or make an error. The idea here is to understand what specifically is working or not working and why you don’t need a large number of users to do this. Five years is enough. By learning about what doesn’t work in our design, we can go back and rework that aspect to improve the experience. Formative evaluation is used frequently throughout the design process to support the iteration to a better product. Overall, summative evaluation is normally carried out. When you have a complete design or shipped product, again, you perform tasks with your product normally unassisted, like they would in real life. And then they grade the experience on a predefined scale against predefined criteria, which could include things like satisfaction, ease of use, net promoter score and so on. A summative evaluation will tell you how usable or satisfying the experiences. Summative evaluation can be performed at various points during the product lifecycle. It will help you and your team to understand how your iterations compare, but be aware in order to get reliable ratings, you need a large representative sample. This takes time if your product isn’t life because you’ll need to recruit that sample to evaluate your design. Summative evaluation isn’t helpful if you want to learn quickly what you need to do to improve the design. But it is useful if you want to see overall how your product compares to its competitors or to your previous iterations. So use formative evaluation as part of your early design process and employ it frequently. Use summative evaluation.

When you can obtain a large representative sample, your product is complete and you want to know how it’s performing. Generally, make sure you utilize the right method of evaluation for the maturity of your product and to meet your goal.

Leave a comment

Your email address will not be published. Required fields are marked *