Predictive Markets - Is the Crowd Consistently Wise?

By: John Kearon - Based on the Award winning Paper "Best Methodology" - Esomar 2005

CEO of Europe’s leading online research agency. John was Ernst & Young’s ‘Entrepreneur of the Year 2005’ and the company’s innovative approach to research has garnered a number of awards including; ‘The Most Innovative Use of IT’ and ‘Service Business of the Year’ 2004. John’s unusual and refreshing approach makes him a popular speaker at research and marketing events. John’s recipe for entrepreneurial success is; creativity, resilience, determination, perseverance, stamina, drive, imagination, resourcefulness, courage, self-belief, commitment, ability to go without sleep and a touch of madness.

Two years and seven experiments on from the paper in 2005: Re-engineering Research to take advantage of the ‘Wisdom of Crowds’


Introduction

The original 2005 ‘Predictive Markets’ paper shared the challenging supposition that a large diverse crowd of people, buying and selling shares in ‘ideas’ will be at least as accurate as and possibly more discerning than a classic, scientifically sampled research approach. We decided to torture test the supposition on a classic concept test, with some intriguing results.

Predictive Markets, originally from experimental economics and popularised by James Surowiecki’s book, Wisdom of Crowds puts forward the challenging assertion is that crowds make better decisions than experts, when they are diverse, independent and faithfully aggregated. What he puts forward also suggests another possibility. Namely, a crowd of people operating through a ‘Market’ mechanism could be just as accurate as our best research approaches.

Surowiecki’s evidence makes a strong case for crowd based markets being uncannily accurate, so we decided to test the thinking by evaluating 10 new product concepts in a crowd-based-market and the same concepts in 10 separate (monadic) concept test.

Effectively we were testing the following:

1. Could a diverse crowd be as accurate as a carefully sampled target audience?
2. Could buying & selling ideas prove as accurate as a classic monadic concept test?
3. Could judging on behalf of the market be as accurate as answering for oneself?

The Predictive Market did prove uncannily accurate; selecting the same top five ideas as the monadic with no significant difference in purchase intention scores on four out of five and the fifth was only just significant. Helpfully the market was also much more punishing of the weaker five ideas, with significantly lower scores for four out of the five, offering the additional benefit of sorting ‘wheat from chaff’ more than in monadic tests, giving clear guidance for which ideas to drop.

To read more:

No hay comentarios.: