Wisdom of the crowd.

I watched a few episodes of a series called The Code on the Science channel recently.  The Code, by the way, is mathematics.  It's a GREAT series to show to math students of high school or college age!  It talks about mathematics that occurs in nature and that can be applied to numerous areas of life.  And, I learned a few things! 

One of the things that intrigued me was an experiment that the narrator/host conducted about the wisdom of the crowd.  He filled a large jar will jelly beans and asked a rather long list of people to guess how many beans were in the jar.  There were about 4,000 beans in the jar, but he got guesses that ranged from 400 to 50,000.  Very few were even close.  Yet, when he took the average of all the guesses, he got a number that was within 5(!!) of the actual total.  He says that this kind of result is not that unusual. 

I would write this off as a coincidince except that it goes hand-in-glove with another phenomenon called open source software development.  This is a form of software development in which hundreds or thousands of programmers work on the same project at the same time and communicate very little, if at all, with each other.  According to traditional software engineering methods, it shouldn't work.  I would have bet the farm it wouldn't work.  But it does, and it produces really good software!  I learned about this from a book called The Cathedral and the Bazzar by Eric S. Raymond. As a person who was engaged in software engineering for a living for the better part of 20 years, it blew me away. Software engineering is supposed to work best in small teams of 5 to 8 people who are tightly controlled and in constant communication.  Open source is the exact opposite. 

Does this phenomena apply to other areas?  The underlying philosophy of democracy is the same -- the broad-based electorate is wiser than any small group.  There is another book that I would recommend - Never At War by Spencer Weart.  In that book, Dr. Weart makes a good case that two countries that are true demoncracies never go to war with each other.  He does spend some time defining a true democracy, which is crucial.  For instance, the American Civil war is not a counter-example because the slaves could not vote in the Confederacy.  But, if you use his measuring stick, you will see that though democracies do, unfortunately, find it necessary to go to war from time to time, it is never with another true democracy.  The crowd is even wiser than we knew!

Food for thought...does this mean that the outcome of most, if not all, elections, is the best outcome possible, given the individuals who are candidates?  No matter what you political leanings, you are probably thinking "What?  No way!  What about the time that [insert name] was elected in [insert year], given that he was a(n) [select one or more from: idiot / nut job / spawn of the devil] ?"

Again, food for thought.