The CIA Program That Started with a Dead Ox

The wisdom of crowds is a concept that arose in 1906, when British statistitian Francis Galton observed a competition at a fair where 800 people guessed the weight of a dead ox. No one got the right answer, but when he tallied all the guesses he was shocked to learn that the average — or maybe it's the median — was 1,197 lbs, just one pound short of the actual weight.



This is also how colonies of bees and ants appear to do quite intelligent things even though each individual insect is totally oblivious.

A few years ago the CIA started a program to use Galton's finding. "The Company" is infamous for weird and sinister programs. For example, MK Ultra, where they tried to use drugs like LSD to produce "Manchurian Candidates." Or Remote Viewing, where they had psychics using ESP and clairvoyance to spy on the Russians. Or waterboarding, where they tortured prisoners with simulated drowning, a tactic used by the Spanish and Flemish Inquisitions, the Gestapo and WWII Japanese war criminals, who were hanged for torturing Americans.

The program that came from the dead ox is not so sinister. Called the Good Judgment Project, it attempts to use the wisdom of the crowd to forecast world events. It's been running for three years now.

It consists of 3,000 ordinary people who answer questions on a website to estimate probabilities of future events. The astonishing thing is, this program is better at making forecasts than the professional CIA analysts who have access to classified documents.

Even more amazing, some of the individual participants are 30% better than the CIA at doing the CIA's job. How do they do it? Elaine Rich, a sixty-something pharmacist, says, "Usually I just do a Google search."

Even though that may sound impressive, it really isn't. The CIA -- and organizations in general -- are notorious for groupthink. They know what answers their bosses want to hear, because their bosses have already decided what they want to do, and they just want ammunition to back up the decision they've already made. This was especially true of the Bush administration in the run-up to the Iraq war.

But there are less sinister reasons why a group of 3,000 folks from across the country might be better at doing this than the CIA.

First, they have no skin in the game, while the CIA is responsible for the safety of the American people. If a CIA analyst misses something a lot of people may die. Analysts feel that pressure, and will tend to perceive potential threats to be greater than they actually are. Also, the number of analysts devoted to each area at CIA is fairly small, and they all talk to and influence each other. It's only natural that they would drift toward a consensus, and all too often consensus is driven not by mutual agreement but by whoever shouts the loudest.

Sometimes there is just too much detail. With all those classified documents, analysts can get bogged down in minutiae that are much less important than they might seem. They can't see the forest for the trees.

Finally, no one cares if a CIA section predicts 10 doomsdays an hour and none of them ever happens, but everyone will be all over them if four guys die in an embassy attack that they failed to predict.

By the way, if you're interested the Good Judgment Project is accepting applications!