The presentation opens with a chart. The business case includes a metric. The product review features a dashboard. Everyone nods. The decision is made.
Nobody asked where the data came from, what it excludes, or whether it measures what they think it measures. The organization is not data-driven. It is data-decorated.
⸻
The Selection Problem
Data does not arrive neutral. Someone chose which data to collect, which to display, and which to omit.
A product manager presenting a feature's success will show adoption rate. They will not show the support tickets the feature generated. They will show time-to-engagement. They will not show the churn rate among users who engaged.
The data is real. The selection is editorial. And the editorial decision is almost always in favor of the conclusion the presenter wants to reach.
Calling this "data-driven" gives a subjective editorial process the weight of objectivity. The data is the costume. The decision was already made.
⸻
The Metric Proxy
Most of the metrics organizations track are proxies, not measurements.
Net Promoter Score does not measure customer loyalty. It measures a customer's answer to one question on one day. Monthly Active Users does not measure product value. It measures the success of the engagement loop. Revenue does not measure health. It measures the ability to extract payment.
Every proxy is a simplification. The system forgets this. It begins optimizing the proxy directly, and the distance between the proxy and the thing it was supposed to represent grows silently.
When the proxy moves in the right direction, nobody investigates further. The number confirmed the narrative. Investigation stopped.
⸻
The Confirmation Engine
Data-driven organizations have a predictable failure mode. They use data to confirm decisions that were already made, not to challenge them.
The hypothesis comes first. The data search comes second. If the data supports the hypothesis, it is presented. If it contradicts the hypothesis, it is reframed, recut, or excluded as "noisy."
This is not analysis. It is prosecution. The conclusion is predetermined. The data provides the evidence.
⸻
The Honest Standard
Being data-informed means accepting data that contradicts your position.
Before presenting data, state the hypothesis. Then show the data that supports it and the data that does not. If you cannot produce contradictory data, you did not look hard enough. If you looked and found none, state that explicitly.
A decision supported only by friendly data is a decision supported by curation, not evidence.
End.