How Numbers Lie
At the beginning of 1910 Giuseppe Morello went off to federal prison, and the Secret Service could celebrate. Its chief, John Wilkie, claimed that the Morello gang was responsible for sixty per cent of all the Black Hand extortion in America over the previous ten years. Now the ringleader was inside and the numbers, as expected, collapsed. According to the Service, Black Hand–style crime among the Italians of New York had fallen by more than 75%.
A splendid report. It has just one snag. Critchley drily records that in 1911 seventy Black Hand bombs went off in the city. In 1910 there had been thirty-five.
Extortion down by three quarters, and twice as many bombs. Strictly speaking, the one doesn’t rule out the other: there could have been fewer threats, while the people who didn’t pay got blown up more often. They could have counted in different ways and within different boundaries. But a pair of numbers like that is a reason to pick up the report and check it before you celebrate. Critchley explains what was going on: the Morello gang was above all in the counterfeiting business, and the Black Hand was a sideline for it. The Service wanted its chief villain to be the cause of all the evil. They locked up the villain, and in the report the evil obediently shrank. Out on the street, the evil never heard about it.
Numbers almost never lie by themselves. The liars are the people reading them, and usually they lie to themselves. Here are a few of the ways it’s done.
A Number with No Neighbours
In The Godfather—mind you, this is a film—Michael Corleone walks through the little town whose name he bears, sees the closed shutters and the funeral notices, and asks where all the people have gone. His bodyguard answers that they’re all dead, from the vendettas. He says it as if vendetta were the plague.
Dickie went and looked at what had really happened. In the years when the Michael of the film was strolling round Corleone, the townspeople were suffering less from the Mafia than from typhus: in the summer of 1947 about forty people died of it. As for murders, the town had eleven in 1944, sixteen in 1945, seventeen in 1946, eight in 1947 and five in 1948. Nothing like the cinema’s plague. But for a small town, Dickie writes, it was an astonishing number.
See what happened? One and the same number, “eight murders in 1947”, can sound like nothing or like a nightmare, depending on what you put next to it. Next to the film, it’s nothing. Next to forty dead from typhus, it’s less than an epidemic. Next to an ordinary peaceful town of the same size, it’s a reason to run.
A number with no neighbours means nothing. “We sold ten per cent more”—more than what? Than last year, when there was a crisis? Than the competitors, who grew by thirty? Before you celebrate or panic, find the number some neighbours: last year, the next district over, a similar firm.
One Witness
In 1963 Joseph Valachi, a soldier in one of the New York families, told a Senate committee how the Mafia was organised. The committee decided he had revealed organised crime to be real. Critchley writes that Valachi’s testimony, as it was interpreted, became the empirical foundation of every model that followed: a single, rigid, centralised empire.
But from 1969 the results of long-running FBI surveillance began to surface. They painted a different picture: fragmentation, no overall command, businesses of modest size. One soldier saw his family from the bottom. From what he said, they drew a conclusion about the whole country.
That’s a sample of one. The commonest mistake in any office. One loud client complains: “the clients are unhappy”. One salesman says the product isn’t moving: “the market’s fallen”. One acquaintance tried it and liked it: “there’s demand”. Before you draw a conclusion, ask: how many people told me this, and are they like everybody else? Or am I only hearing the ones who shout loudest, or the ones who happened to be standing nearby?
Give or Take
In 1987 the carabinieri counted forty-six Mafia families in Palermo and its province. A 1990 report already gives forty-eight. Agrigento had thirty-six, and then thirty-eight. Trapani had fourteen, and then twenty-one. Gambetta gives both sets of figures and remarks that they differ somewhat.
From fourteen to twenty-one is growth by half again. Did the Mafia in Trapani grow by half in three years? Maybe. Nobody can check. More likely different people counted in a different way from different information: families don’t register and don’t publish accounts. So the honest figure here isn’t a point but a range: “somewhere between fourteen and twenty-one, maybe wider”.
Or take this. By the end of 1995 a hundred and twenty-five million pounds’ worth of property had been confiscated from Totò Riina. A huge figure. Dickie immediately adds that it almost certainly doesn’t reflect the true size of his fortune. They confiscated what they found. How much they didn’t find, nobody knows.
A scientist would call this a discrepancy between counts and incomplete observation. The phrase “confidence interval” doesn’t fit here: you get one of those by a particular calculation from a sample, at a given level of reliability, while the carabinieri simply counted twice in two different ways. Simpler words will do for us: “give or take how much?” When someone brings you next year’s sales forecast accurate to the last cent, don’t admire the accuracy. Ask how big the error could be. If the honest answer is “give or take a third”, then all those cents were decoration.
Share and Cause
In 1990 Sicily had nine per cent of Italy’s population. Yet it accounted for twenty-four per cent of all the robberies in the country. The conclusion suggests itself: Mafia protection is useless at best, and a con at worst.
Gambetta doesn’t accept that conclusion. The protection works, he writes. It just protects the wrong people. Thieves don’t like being cheated either, and they can’t go to the police. So they need protection more than anyone, and where there’s a Mafia, stealing is safer.
There are two lessons here. First: ratios are powerful things. “Nine per cent of the people, twenty-four per cent of the robberies” says more than just “lots of robberies”. Second: even an exact ratio doesn’t tell you what causes what. There’s a Mafia and there are lots of robberies. That doesn’t mean the Mafia can’t protect. It may mean it protects thieves. It may also mean there’s unemployment there, and the Mafia has nothing to do with it. Two things that walk together aren’t necessarily holding hands.
Morello and the bombs, by the way, are the same story the other way round. They locked up Morello and the number fell. So Morello was the cause? The bombs gave their own answer.
The Average Boss
Gambetta gives one more figure. In 1987 the median age of Sicilian bosses was fifty-seven, and of Neapolitan bosses thirty-five. A difference of twenty-two years.
In a footnote he explains why he took the median rather than the mean: it’s steadier against unusual values at the edges. If one boss in Naples lives to ninety, the mean jumps, while the median barely stirs. The median is simply the man in the middle: half are older, half are younger.
And then Gambetta does something worth repeating to everyone who reads numbers. He offers two explanations for the difference and weighs them against each other. The first is culture: Sicilians, so the argument goes, are more inclined to obey their elders. The second, in his view, is better: in Sicily the firms are more solidly built and men rise to the top in turn, while in Naples bold young thugs force their way up without waiting their turn. A boss’s age tells you nothing about a people’s character and a lot about how the firm is built. Where the boss is grey, nobody is breaking the ladder.
Once they brought me a report: the average order from our restaurants had gone up by a quarter over the year. I was pleased. Don Vincenzo asked only: “Whose?” It turned out it was one hotel on the seafront, which had opened a second restaurant and started taking oil by the barrel. The rest were taking as much as before, and two of them even less. The average went up. The business didn’t. One hotel went up. After that the Don always asked to be shown not the average, but the one in the middle and the one at the edge.
The Bottom Line
Give every number some neighbours and ask “give or take how much?” And when a report announces a victory, look out of the window. If the bombs are still going off out there, the report isn’t necessarily lying. But it’s time to read it more carefully.