A mathematical BS detector can boost the wisdom of crowds – George Musser | Aeon…
A mathematical BS detector can boost the wisdom of crowds
Wisdom of crowds is an old concept. It goes back to Ancient Greek and, later, Enlightenment thinkers who argued that democracy is not just a nice idea, but a mathematically proven way to make good decisions. Even a citizenry of knaves collectively outperforms the shrewdest monarch, according to this proposition. What the knaves lack in personal knowledge, they make up for in diversity. In the 1990s, crowd wisdom became a pop-culture obsession, providing a rationale for wikis, crowdsourcing, prediction markets and popularity-based search algorithms.
That endorsement came with a big caveat, however: even proponents admitted that crowds are as apt to be witless as well as wise. The good democrats of Athens marched into a ruinous war with Sparta. French Revolutionary mobs killed the Enlightenment. In the years leading up to 2008, the herd of Wall Street forgot the most basic principles of risk management. Then there was my little Skittles contest. It was precisely the type of problem that crowds are supposed to do well on: a quiet pooling of diverse and independent assessments, without any group discussion that a single person might dominate.
Dražen Prelec, a behavioural economist at the Massachusetts Institute of Technology (MIT), is working on a way to smarten up the hive mind. One reason that crowds mess up, he notes, is the hegemony of common knowledge. Even when people make independent judgments, they might be working off the same information. When you average everyone’s judgments, information that is known to all gets counted repeatedly, once for each person, which gives it more significance than it deserves and drowns out diverse sources of knowledge. In the end, the lowest common denominator dominates. It’s a common scourge in social settings: think of dinner conversations that consist of people repeating to one another the things they all read in The New York Times.
Dražen Prelec, a behavioural economist at the Massachusetts Institute of Technology (MIT), started from the premise that some people’s judgments deserve greater weight than others. By no longer averaging everyone’s judgments equally, you can avoid overcounting redundant or otherwise extraneous information. You already do this all the time, whenever you trust opinions that are expressed with confidence and spurn diffident-sounding ones. There’s something to be said for that kind of trust. In psychology experiments, people who are more accurate at a task – say, remembering a list of words – tend to express more confidence. Unfortunately, the converse isn’t true: confident people aren’t necessarily more accurate.
The solution, Prelec suggests, is to weight answers not by confidence but by metaknowledge: knowledge about knowledge. Metaknowledge means you are aware of what you know or don’t know, and of where your level of knowledge stands in relation to other people’s. Metaknowledge functions as a powerful bullshit detector. It can separate crowd members who actually know something from those who are guessing wildly or just parroting what everyone else says.
The article expands a lot further around this idea and is fascinating to read. It has implications too for many assumptions we make during the daily course of our lives, and decisions we make (including around political corruption, choice of music, political and economic forecasts, climate change, etc). The most interesting, is the test that could be used to determine who you should give extra weight to in terms of their knowledge.
Read the full article at https://aeon.co/essays/a-mathematical-bs-detector-can-boost-the-wisdom-of-crowds
#metaknowledge #climatechange #economicforecasting #wisdomofcrowds
https://plus.google.com/+DanievanderMerwe/posts/WqbMw1yEEy1
