Are there any definitive answers when it comes to you, X, how do you balance advocating for the user against business goals and how many users do need to test with? In a Q&A session during our New York virtual news conference, Jacob Nielsen addressed these questions and his famous five years ago.
It is definitely a balance because you can only advocate for the user, even though I always like like to say that that’s our job. Right. We as a user advocates and the only stakeholder not at the meeting is the user. Everybody else is the marketing, the engineering guys. Everybody else is in the meeting. But the actual customer is not in the meeting that that’s our job, to represent the voice of the customer. And that is true. But at the same time, you can just only drive to that and say, well, who cares about the company is going to make no money and go out of business next week. If it go out of business next week, you’re not going to have a quality product to deliver to these customers. So so it’s clearly extremely important that we deliver business value. That’s also what we’re being paid for. More pragmatically speaking, whether we are employees or we’re consultants or whatever, we’re being paid to go and deliver some business value. So from all these different perspectives, we had to deliver business value. But I think a lot of the conflict between the two customer advocacy and business value comes from your your time perspective, because if you have a very short term time perspective, you may think that we can just mess with the users and cheat them, maybe even and do a bad, bad user experience. And we can squeeze out a little more and more money out of them next week. And that’s probably true, to be honest. Quite often you can’t do that bipartisan. You can make more money on a short term basis. But the long term basis and this is what I think what you can flip the perspective, the longer term perspective, a lot of value comes from customer loyalty. And we know as interest in this business studies is already that you make much more money from loyal customers than from the first time customer. And how do you create customer loyalty with the variety of ways, but basically by delivering what people want and need in a nice way that they like you and they like using you and they do see the website and using your product and having good experience, all the various things that are really what we would advocate from a strict usability perspective, a strict purely experience perspective. Those are some of the things that actually drive the longer term value. So over the next year or the next 10 years, whatever long, long term perspective, you will get more value from delivering a good user experience. And so I think that’s the counterargument quite, quite sometimes. But I mean, all scale it all back to kind of pragmatics, which is, yeah, you’ve got it. All these things are tradeoffs. And you can’t just only drive for one thing or you’re going to get an unbalanced product out at the end. So it’s about it has to be a balance. And we have to I think we can proudly say we will advocate for financing if we can, because we also advocate for this, the balance going this way. But we have to remember the balance can go. I mean, you tip over, right? Clearly, you cannot do that either. So you can advocate for one direction while remembering the other direction has is justified as well as income to some reasonable middle ground.
Thank you, Jacob. I loved you falling over. So actually, I see you here today.
Charles Fasano had asked, are there then any definitive answers when it comes to Web usability or are is it all subjective? If he’s looking at a lot of articles, thinking about things that we in our course, like so many places, have different answers. So what do you do?
Well, I think it’s a little bit of a false dichotomy to say that this either everything has one single answer or everything is subjective and whatever. I think neither of those two is really true in user experience because it’s such a broad, multifaceted field that there’s really a single solution. I mean, we will often say that something is a usability guideline if we believe that it’s true. And about 90 percent of cases, that’s my kind of like a guideline or guideline for guidelines is that if something is true, the vast majority of cases will make that a recommendation. But that doesn’t mean that it’s always true. I mean, there will be those, roughly speaking, 10 percent of cases where you do something different because the circumstances of context require something different. And now I do want to say here that people always think, oh, I’m different, I’m special, and therefore they should just ignore the usability guidelines. And that’s wrong because probabilities are in favor of being in the 90 percent of unusual cases. So you’re most likely only a little different than enough different to violate the guideline. But even so, I mean, the guidelines only guideline, it’s not like one hundred percent truth. That’s always the case. But we had the other extreme is completely false also, which is we know nothing about user. Ability, any design is as good as any other design, however I prefer. So when we should ship, that’s a prescription for disaster. I mean, like a terrible, terrible product. There’s a lot we do know. I mean, we just don’t there are few, like really hardcore psychology rules, but they’re relatively few at the FETs law and a few other rules of that nature that are relatively firm. But most of us have a little bit a little bit amorphous because we talk about the human brain and not just that, but but the organizational behavior in psychology and social behavior and the very broad systems that don’t have a single, very clean answers. But that definitely are answers. And there definitely are things to know that can guide us for saying this is a better design than that design. So it’s definitely not a matter of anybody’s opinion, as good as any other opinion, because I definitely think there’s a lot of things we know. And yet we don’t know the one and we know them, let’s say, 90 percent as a rough estimate.
Thank you, Jake. And actually, I noticed there was kind of a good follow up question to this that came through the Simchat.
So Rylan had asked around, does that five user rule still stands even though it was formulated in about 20 years ago? I guess why does it still stand or are there any changes there that you were thinking about?
I think it actually very much does stand. So the rule is that if you test with about five users, it is about but a test with about five users, you will have learned a very large percentage of what you will ever learn if you test that same design. And so if you have budget for more than five users, which I would always recommend because I like doing more user research, you’re better off spending it on several small studies and doing it and testing it and doing something new rather than spending 30 users, 50 users, one hundred users and keep testing the same thing. And the reason this continues to be true is that it’s because people are different, but they’re not 100 percent different. In other words, when you if you have one design and that’s what we’re talking about, if you have a specific design, you want to know about it and you’re testing it with some users, well, different people will use it in different ways. And this is why it’s not enough to just test one user, because the next person will do something sort of different. But people are not like a hundred and different either. I mean, there’s a lot of commonalities. And so the second user will do some of the same as the first user. And then experience just shows that if you keep adding people who do a little bit something differently and other things the same, you know, there’s going to be more and more of a similarity building up as you test more people.
And so as you get to see that around five, it’s not as if you couldn’t do six. But usually from person number six, you’re sitting there yawning and saying, I’m seeing the same thing again. And why keep doing that? Rather, go back to the drawing board, use what you learn from those five people, change the design to make it better, hopefully better. Right. And maybe you did. Maybe you failed because not all redesigned. So for the better. And so you test that with another five users and you discover whether it’s better or worse. And also you discover that some new things that should be fixed, you just keep doing that. Another lesson that has been true for my entire career is an iterative design. The more iterations, the better. You can always make it better. There is no perfect user interface. There’s just so many dimensions of interface design that we will never design the perfect user interface. It can always get better every single time. You can try something out and discover whether it’s better or worse and keep improving, you really can always keep improving. So that’s the thing to really drive for is, you know, as many iterations as possible. And since it is a fact of life that there’s only a limited amount of budget, I mean, nobody, even the richest company and not infinitely rich. So there’s always a budget available. And so whatever budget you have. Right, you’ve got to allocate in such a way that you can do as many iterations as possible because that drives the design quality up and that means a small studies and many of them.
And that’s sort of based on some, I would say, kind of fundamental fact of how it is to go about studying stuff with humans. And the stuff changes whether you’re testing a mainframe interface or PC interface or a website or mobile app or a phone, the watch interface or wall interface or virtual reality or the voice interface. I mean, for all of these type of things, it’s the same general point, which is that they can always get better. But if you have multiple people use, then they will sort of tend to use them in approximately the same way. Because also remember that. Usability studies is it is a debarking methodology. It’s actually sort of similar to if you have a piece of programming code in trying to find a bug, a programming error, you were trying to find a design error. And so you kind of hit and this interface a number of times. And at some point of time, you’ve sort of learned about you haven’t done everything because it’s infinitely many things that you could learn, but it’s just worth it to find that last minute little little thing, rather to the redesign and to test again.