Vanity is defined as taking excessive pride in one’s appearance or achievements. Similarly, in analytics, a vanity metric is data that perhaps looks good on paper and makes the website or app appear successful. But that doesn’t actually say anything about how well it’s truly performing. Vanity metrics tend to be evergrowing numbers like that. Billions served sign at a McDonald’s. A stand alone data point is impossible to interpret, and it’s a clear warning sign of a vanity metric. Context is key. Is it impressive that ten thousand people have downloaded your app?
Not if eight thousand of them never opened it again. So rather than tracking a single ever bigger number, focus on a rate or ratio instead to provide some context, specifying the time frame like a daily or weekly rate would allow you to track the trend to see whether things are getting better or worse as you continue to make design changes. Tracking a metric on a per user basis can also work well to give you insight into what proportion of your users are completing certain actions, or perhaps how often an average user might be taking that action. Or it could make sense to compare one metric to another if that provides some meaningful context to understanding how well something is performing. When determining what rate or ratio makes the most sense to track, look for something that would be relatively stable to provide a good baseline, but still be sensitive to tell whether the design is improving or not.
After making a change, you want to rate whose changes can be attributed to your design decisions rather than something that fluctuates constantly on its own. Tracked metrics should help you gauge Design’s performance so you can make data driven design decisions. A vanity metric might make you feel good, but if it doesn’t help you take action, then it’s not worth tracking.