Recently we wanted to put Beste.st, and our friends, through their paces - how well could a (largely uninformed) crowd predict the results of the 2010 World Cup? When it comes to major sporting events, it seems everyone (even cephalopods) has an opinion.
So we set up a quick survey of all 32 World Cup teams on Beste.st and asked - Who is going to win the 2010 world cup? We then sent the survey out to our friends, who being mainly Australian, didn't know too much about soccer (sorry guys!).
The survey worked by taking two of the teams at random and asking the user to choose which of them they thought would be more likely to win in a match. As people voted, Beste.st collected the results and calculated a leaderboard. After the first week, here's what Beste.st came up with, based on the armchair opinions of about 40 visitors.
- Argentina
- Brazil
- Netherlands
- Spain
- Germany
By comparison, the popular website www.worldcupbuzz.com tried a more conventional means of gauging opinion using a straw poll method of suggesting a winner. Their predictions, from more than35,718 votes:
- Spain
- Brazil
- Argentina
- Italy
- France
CBS.com tried a similar method to find out who their readers thought could win. Their results?
- Brazil
- Germany
- Spain
- USA
- Argentina
Dr Ian McHale of the University of Salford tried a more sophisticated approach. He used an ordered probit model using the data from 18,000 previous match results between the 32 teams.
His results?
- Spain
- Brazil
- France
- Netherlands
- Argentina
Now of course, we know the final results of the 2010 world cup:
- Spain
- Netherlands
- Germany
- Uruguay
- Ghana (according to FIFA rankings)
So - which was the best predictor? A detailed statistical analysis of 18,000 match results? 30,000 World Cup Buzz fans? Or 40 uninformed Aussies? The results of this meta-competition were:
WorldCupBuzz.com correctly predicted 1 entrant (Spain) out of the top 5.
CBS.com visitors predicted 2 out of 5 (Spain and Germany)
The statistician who analysed 18,000 games also predicted 2 out of 5 (Spain and the Netherlands)
and
Beste.st visitors predicted 3 out of 5 (Spain, the Netherlands and Germany)
Not a bad outcome!
Of course there are many elements of our assessment that are pretty unscientific, and both WorldCupBuzz and Ian McHale deserve credit for correctly predicting the eventual winner - Spain.
But we're pretty stoked so far with our little project, and it's a great demonstration of how, with the right tools, the crowd can be a powerfully accurate predictor.
