Making product decisions

Go back to My Knowledge Base

Do you always follow what data says?

We’re blind. It doesn’t matter what any individual person thinks about something new. Everything must be tested. It’s feature echolocation: we throw out an idea, and when the data comes back we look at the numbers. Whatever goes up, that’s what we do. We are slaves to the numbers. We don’t operate around innovation. We only optimize. We do what goes up.

Anonymous Facebook employee

Business vs User Experience

Let's see an example from Facebook. They were going to launch this new gorgeous timeline.

But there was a serious problem: the numbers. Facebook found during testing that users who were switched to the new News Feed tended to spend less time on the site.

Facebook’s data team discovered that the new News Feed was performing too well. It was performing so well from a design standpoint that users no longer felt the need to browse areas outside of the News Feed as often, so they were spending less time on the site.

Unfortunately, this change in user behavior led to fewer advertisement impressions, which led, ultimately, to less revenue.

This is truly a nightmare scenario for any CEO: do you take the risk and proceed with the better user experience/product at the expense of short term numbers–with no promise that the better design will actually lead to long-term benefits–or do you scrap the new design and start over? The answer to this question is where company culture and identity dramatically peek through the curtain and express themselves in very raw form. Because it is impossible to answer this question rationally–both sides can be argued very successfully–it has to be answered from a cultural point of view.

Facebook didn't launch it at the end. In fact, with the new News Feed you can see more ads.

Articles I've used

Comments powered by Disqus