Recency is a dangerous bias

As many companies did, we just finished product planning (at least for 1H) before the holidays. There are a lot of inputs that go into a roadmap – customer problems to solve, customer satisfaction to deliver, company strategy, product strategy, and beyond.

Understanding how to prioritize and weigh these inputs is a critical part of being a successful product manager.

When these inputs come in the form of metrics, it tends to be easier to look at things as black and white. As a simple example, releasing feature A to a few customers helped us move critical metric M. We need to add component C1 and C2 to get to the next 10 customers, and then we’ll be able to move M even further.

However, when they come from anecodotes, then things are less black and white. Worse, there’s a tendency to look at the most recent anecodote, and give it greater weight than it deserves. Just like when you buy a red car, and all of a sudden every car you see is a red car , observational selection bias can bite you when you’re doing product planning.

This is where bringing in a framework like RICE can help, but the challenge is that if the weighting is biased, then the framework will fail you.

So how do you weigh the weighing system without introducing recency bias?