Attribution is a ludic fallacy

There is a slide in most media reviews that splits every sale across channels, down to the decimal. Paid search, 34%. Retargeting, 22%. Email, 11%. It adds to a hundred and it looks like measurement. It is closer to a set of casino odds, and the market it describes does not run like a casino.

Nassim Taleb has a name for this mistake1 . He calls it the ludic fallacy Taleb's term, from The Black Swan: treating real-world uncertainty as if it were a game with fixed, knowable rules and computable odds, like a casino. Real markets have neither. : taking the neat, bounded odds of a game and applying them to the open world, where the rules are not fixed and the next event may be one the model never contained. A roulette wheel has knowable odds. A market has customers who change their minds, competitors who move, and a hundred causes that never leave a trace. Attribution takes the second and treats it as if it had the fixed odds of the first.

What the model measures, and what it claims#

Last-click attribution records the final touch before a sale. It then presents that record as the cause, which is a different and much larger claim.

Start with what an attribution model actually records. Last-click, and most of its more sophisticated cousins, log the touchpoints on the path to a sale and hand out credit by a rule. That record is real. The trouble is the sentence people say next to it: that these are the things that caused the sale. Recording the last few clicks before someone buys is not the same as explaining why they bought. The model saw a click. It did not see the two years of preference that made the click happen, because preference does not click.

The demand with no click#

The customer who was always going to buy you, and searched your name to do it, looks identical in the data to a customer performance created. Attribution cannot tell them apart.

Here is the case the model handles worst. Someone has wanted your product for months. When they are ready they search your brand name, click the ad at the top, and buy. Last-click hands that sale to paid search. On the slide it becomes evidence that paid search drives revenue. What paid search did was stand in the doorway and collect a customer who was walking in anyway. The demand was built somewhere the model cannot see, so it does the only thing it can: hand the credit to the nearest click.

Taleb’s word for the rest is silent evidence Taleb's term for the failures, near-misses and non-events that never make it into the data, so we reason from the survivors and the visible cases only. . The demand your brand created that never resolved into a trackable click leaves no row in the table. Read the table and you would conclude that demand does not exist, for the same reason a survey of lottery winners would conclude that buying tickets makes you rich.

Recording the last few clicks before someone buys is not the same as explaining why they bought.

Why the false precision wins anyway#

In a budget meeting, a number to the decimal beats an honest “we cannot attribute this exactly”. It wins by looking rigorous, not by being right.

None of this would matter if the model lost the argument. It wins it. In a budget meeting, the person pointing at the dashboard, paid search drove 34% of revenue, beats the person who says the brand work probably built a chunk of that demand but cannot pin it to a click. Every time, and not because they are more likely to be right. A decimal reads as rigour. An honest “we cannot measure this precisely” reads as an excuse. So the room backs the number it can show, and cuts the brand work first.

The honest version is messier#

Incrementality testing gives up the decimal and asks a cruder question: run the media, hold a group out, measure the gap. It admits uncertainty, which is why it is closer to true.

There is a more honest tool, and it is less impressive on a slide. Incrementality testing A test that measures how many conversions your media actually caused: run it in some regions and hold it out in others, or use ghost ads, and measure the gap against the customers who would have converted anyway. gives up the per-channel decimal and asks a blunter question. Run the media in some places, hold it out in others, and measure the difference. The answer comes back as a range, not a point, and the range is often uncomfortable. That discomfort is the tool working. It admits the uncertainty that attribution hides, and that is why it lands closer to the truth. I made this case in full in the first essay; most shops avoid the test because the result undercuts the last-click story.

Where this sits on the triangle#

Attribution can see Activation and cannot see Equity, so the spend follows the corner the model can count. People make that call and feel rigorous doing it.

On the model, attribution is one of the main reasons brands starve the corner that compounds. Activation leaves clicks, so the model can count it. Equity leaves demand with no click, so the model cannot. Feed a budget decision with a tool that shows one of the three corners and the spend follows the tool. Nobody in the room chose to cut equity. They let the number decide, and felt rigorous doing it.

So the next time someone shows you attribution to the decimal, ask one question. What does it do with a sale that had no click on the way in? If the answer is that it files it under “direct” or “organic” and moves on, you are looking at a casino model of a market that is not a casino, and the brand it cannot see is paying for the confidence it gives you.