Minority Report — the 2nd Revolution in Crowdsourcing
There were three breakthrough openings in idea creation last year. The first one – in the beginning of the 2017, and the last one – in December. We kept passion and in the early 2018 we introduce you the finals of the research for now.
These great news were brought to Russia by Sergey Karelov.
Part 1: Revolution in Crowdsourcing is Already Here
Part 2: Minority Report — the 2nd Revolution in Crowdsourcing
Source: Optimal incentives for collective intelligence, by Richard Philip Mann and Dirk Helbing.
Collective intelligence is the ability of a group to perform more effectively than any individual alone. Diversity among group members is a key condition for the emergence of collective intelligence, but maintaining diversity is challenging in the face of social pressure to imitate one's peers. Through an evolutionary game-theoretic model of collective prediction, we investigate the role that incentives may play in maintaining useful diversity. We show that market-based incentive systems produce herding effects, reduce information available to the group, and restrain collective intelligence. Therefore, we propose an incentive scheme that rewards accurate minority predictions and show that this produces optimal diversity and collective predictive accuracy. We conclude that real world systems should reward those who have shown accuracy when the majority opinion has been in error.
The equilibrium distribution of attentionis the same no matter whether agents initially attend to arbitraryfactors or initially favor the most obvious ones. The convergencetime to equilibrium depends on the magnitude of rewards; in oursimulations, we normalize rewards, such that the mean rewardper agent is one at each time step.
Minority rewards lead to a far higher accuracythan any of the investigated alternative reward systems, regardless of system complexity, and achieve close to 100% accuracy.
The Mathematical analysis shows that minority rewards will continue to produce near-perfect accuracy forany problem size if the population of agents is large enough. Researcher's analysis of ﬁnite group sizes shows that minority rewards outperform other reward schemes for problem dimensions up to 10 times bigger than the population size, assumingbest response dynamics.
The different levels of collective accuracy across reward systems are a reﬂection of the differing equilibrium distributionsof the proportion of agents attending to each factor. Minority rewards outperform both market rewards and unweightedapproaches, because attention is automatically redirected if thecollective prediction would otherwise be wrong; only those outcomes where the majority opinion is wrong contribute to agents’rewards. Under minority rewards, the system converges toward astate where the number of agents paying attention to any factor is proportional to factor importance. This optimal distribution is both a stationary and a stable state of the minorityrewards system.Additional analysis shows that varyingthe cutoff value for minority rewards (for example, by rewardingthose voting with less than 40% of the group or 60%) invariablyreduces collective accuracy.
Authors suggest that individuals should not be rewardedsimply for having made successful predictions or ﬁndings andalso that a total reward should not be equally distributed amongthose who have been successful or accurate. Instead, rewardsshould be primarily directed toward those who have made successful predictions in the face of majority opposition from theirpeers. This proposal can be intuitively understood as rewarding those who contribute information that has the potential tochange collective opinion, because it contradicts the currentmainstream view. In our model, groups rapidly converge to anequilibrium with very high collective accuracy, after which therewards for each agents become less frequent. We anticipatethat, after this occurs, agents would move on to new unsolvedproblems. This movement would produce a dynamic system inwhich agents are incentivized to not only solve problems collectively but also, address issues where collective wisdom is currently weakest. Future work should investigate how our proposedreward system can be best implemented in practice from scientiﬁc career schemes to funding and reputation systems to prediction markets and democratic procedures. Researches suggest experiments to determine how humans respond to minorityrewards and additional theoretical work to determine the effectsof stochastic rewards, agent learning, and ﬁnite group dynamics. In conclusion, how best to foster collective intelligence is animportant problem that we need to solve collectively.