It is not people who need to be counted, but independent information
Ten forecasts repeating one note are not equal to ten independent observations. Aggregation can reduce individual noise, but does not have to eliminate overall error. To discuss the “wisdom of crowds,” first ask what people value, what data they saw, whether they responded independently, and how the responses are combined. Market price, average questionnaire, and majority voting are three different mechanisms, even if there are many people involved throughout.
In Lorenz et al.'s experiment, social influence may have reduced judgment diversity without a corresponding improvement in collective accuracy and increased confidence. But the subsequent scientific discussion specifically draws attention to the difference between collective and individual results. This is not a reason to declare any communication harmful: the result depends on the task and evaluation criterion.[1][2]
Fictional measurements of one indicator
Suppose you need to estimate an unknown number, which after verification turns out to be equal to one hundred. Three independent preliminary estimates - eighty, one hundred and one hundred and twenty; their average is one hundred. If participants first saw the erroneous clue "more like one hundred thirty" and all reported one hundred and thirty, the disagreement disappeared and the error grew to thirty. The agreement between the answers became stronger, but the information about the truth did not increase.
This is a deliberately simplified example, not the result of an experiment or a quote model. It shows that a small dispersion of opinions is not independent proof of correctness. Likewise, a group may be unanimously mistaken due to an incorrect conversion rate, an outdated report, or the same interpretation of an indicator. First, the primary fact is checked, and only then the estimates resulting from it are averaged.
How to collect opinions to test the benefits
Invite participants to first independently write down the estimate, range of uncertainty, and source. Save these answers. Then open up the discussion and put together a second version explaining what exactly has changed. After revealing the fact, compare the accuracy of the two stages. This protocol makes it possible to distinguish learning from a more informed participant from simple convergence of answers under the influence of a confident voice.
Don't assign weights just for volume and position. Reasonable weights would require the accuracy available prior to the current historical question on comparable problems. Even then, it is worth testing the robustness to excluding one expert and to common data sources. If two analysts use the same database and the same algorithm, their formal affiliation with different organizations does not yet ensure information independence.
Why Consensus Doesn't Capture All the Uncertainty
Zarnowitz and Lambros's study distinguishes between agreement between point forecasts and uncertainty within each forecaster's probability distribution. People can name close central values and simultaneously accept a very wide range of outcomes. Therefore, a small spread of central numbers may hide significant uncertainty rather than eliminate it.[3]
In practical terms, this means: ask not only for “average expected growth,” but also a list of scenarios and probabilities. When combining, do not erase different bases. Two identical numbers, derived from opposite premises, can react to new information in completely different ways. Testing [forecast calibration](/guides/forecast-calibration/) is more useful than simply showing how many experts agreed with each other.
Why the stock market is not an ordinary quiz
In an educational quiz, the correct answer often exists regardless of the participants' behavior. In the market, their actions can change the price, liquidity and restrictions of others. In addition, opinion weights are determined not only by the number of participants, but also by the amount of available funds and transactions. Therefore, it is impossible to transfer the laboratory result directly to the profitability of the purchase. The issue of interactions is discussed in [analysis of complex systems](/guides/complex-systems-markets/).
When reading the original source, check the type of problem, the method of aggregation, the availability of other people's answers, and the success criterion. For your own discussion, it is useful to maintain independent initial estimates and separately indicate the overall input data. The result should be a list of verifiable reasons for agreement and disagreement, and not the belief that a large group automatically knows the future price.
An exercise with the origins of grades
Make a small map of the five forecasts and draw arrows to the sources used. If all arrows converge to the same original table, increasing the number of retellings did not increase the number of observations. Then note the independent transactions: who checked the perimeter, who recalculated the currency, who used a different data set. Diversity comes at the level of the work being tested, not just the names.
For aggregation, determine in advance whether the mean, median, or other method is used and why. The median is more robust to a single extreme response in a certain sense, but it does not eliminate the common error of all participants. You can show the results of several pre-selected methods without choosing the winner as the one that happens to be closer after the fact is revealed. This is how the organization of the process is tested, and not the ability to retroactively select a beautiful association rule.
Sources and limits of verification
[1] [How social influence can undermine the wisdom of crowd effect](https://pmc.ncbi.nlm.nih.gov/articles/PMC3107299/). Laboratory assessments and social influence; carryover to prices is limited.
[2] [Social influence benefits the wisdom of individuals in the crowd](https://pmc.ncbi.nlm.nih.gov/articles/PMC3169111/). Distinction between individual and collective accuracy.
[3] [Consensus and Uncertainty in Economic Prediction](https://www.nber.org/papers/w1171). Consensus spread is not equal to the uncertainty of individual forecasts.
Ant optimization: memory is on edges
In Marco Dorigo's description of ant optimization, the route is built sequentially: the probability of transition depends on the trace of the previous search and the local attractiveness of the edge. Once routes are built, the trail is updated and part of it evaporates.[RCW1] This is an insect-inspired algorithmic design, not a claim that a natural colony solves any problem optimally.
Let's set our minimum option. From vertex i, vertices j that have not yet been visited are accessible; transition weight wᵢⱼ=τᵢⱼ/dᵢⱼ, where τ is the positive trace, d is the given length. The probability is equal to the weight divided by the sum of the available weights. After a full tour, each used edge receives Q/L, where L is the length of the tour; the old trace is multiplied by 1−ρ. At the first fork with equal tracks and distances 2 and 4, the weights are 1/2 and 1/4, the probabilities are 2/3 and 1/3. Let Q=12, ρ=0,5, the original trace is 1. The tour edge of length 6 will receive the trace 2,5, the unused one will remain with 0,5. This is verifiable positive feedback coupled with forgetting.
For a traveling salesman, cities and the cost of each connection are specified. The tour must visit each city exactly once and return to the beginning. In the artificial graph AB=2, BC=2, CD=2, DA=2, AC=5, BD=5, route ABCDA has length 8; ABDCA - 14. Comparing the two lengths is easy; Finding the best tour among a large number of options is much more difficult. The algorithm is stopped based on a predetermined budget of the constructed tours and compared with a simple random search with the same budget. The best result found is not declared a proven global minimum without a separate lower bound or exhaustive search. By reinforcing the early route too much, variety disappears; evaporation and randomization reduce but do not eliminate the risk of getting stuck.
Labyrinth and search for a specialist: combine information, not answers
In a maze, it is not the average length of other people's paths that is useful, but a map of proven transitions. Author's example: the first participant discovered the edges S→A of length 1 and A→T of length 9, the second - S→B of length 4 and B→T of length 4, the third - A→B of length 1. Their individual routes can be longer than the path S→A→B→T of length 6 in the merged graph. A new route is acceptable only if each edge was actually observed, the directions are the same, and the state of the maze has not changed. It is impossible to average the names of turns: you will end up with a path that no one has tested.
Here variety provides coverage of space, and not necessarily compensation for errors. If everyone is exploring the same corridor, additional people hardly expand the map. If the cost of checking edges is high, you need to compare the gain with the costs of reconnaissance and communication. By analogy, creative inquiry begins with two independent explanations, then seeks an observation that distinguishes them; The mechanical accumulation of links of the same origin does not create diversity.
Finding a rare specialist is another challenge. Instead of averaging opinions, we need to accept applications and verifiable decision criteria. In the toy model, the probability that a randomly selected independent candidate is able to solve the problem is 0,1. For five candidates, the probability of at least one suitable candidate is 1−0,9⁵≈40,95%. This is not a measurement of the effectiveness of open competitions: dependency of competencies, self-selection and cost of verification change the calculation. The terms of the competition should define in advance the reproducibility of the test, the rights to the result and payment; the number of applications does not replace the verification of the solution.
Market predictions and educational printer sales forecast
A “yes” contract pays one notional unit if a predetermined event occurs, and zero otherwise. Given a price of 0,65 and a subjective probability of 0,70, his expected monetary outcome before expenses is 0,05. Interpreting price as a probability requires additional assumptions about the risk attitudes, wealth, liquidity, and capabilities of participants. This is not an average of votes and is not a guarantee of calibration. The source of the fact, the deadline, the procedure for resolving the controversial result and the inability to change the outcome in personal favor must be determined.
Chen and Plott's study of internal sales forecasting at Hewlett-Packard describes a specially organized aggregation mechanism, not the universal superiority of any crowd. [RCW2] Retain the general mechanism, but do not transfer product name, sample, and historical accuracy as ready-made evidence.
In our own exercise, we forecast monthly sales of fictitious printers: below 80, from 80 to 120 inclusive, above 120. Three mutually exclusive contracts pay one each only in their range. Prices 0,2; 0,5; 0,3 are summed to one; purchasing a complete set for one unit gives one unit for any outcome at no cost. The values 60, 100 and 140 can be designated as range representatives and 104 can be obtained as a weighted estimate. However, open extreme intervals do not define their own means; 104 is the result of an additional proprietary model, not a mathematical expectation, unambiguously extracted from quotes.
To test usefulness, first record official forecasts, market probabilities, and when they were produced, then compare errors across the same set of months. Participants from sales, warehouses and production provide different information; a general corporate bonus and the fear of arguing with management can destroy independence. The control also includes unsuccessful months, the cost of the mechanism and changes in the rules. Even an advantage in such a test does not prove the effectiveness of stock prices: the subject of calculation, payments and the influence of participants on the outcome are different.
[RCW1] Marco Dorigo, Mauro Birattari, Thomas Stützle, “Ant Colony Optimization: Artificial Ants as a Computational Intelligence Technique”, IRIDIA, Université Libre de Bruxelles, Technical Report Series, TR/IRIDIA/2006-023, September 2006, sections II.A–II.C, formulas (2)–(5), source cr-aco-dorigo-http. Formal scheme for probabilistic search and trace updating; The graphs and numbers here are the author's.
[RCW2] Kay-Yut Chen, Charles R. Plott, “Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem,” source cr-sales-markets. Primary research of a specific corporate mechanism; educational printers and spacing are an example in their own right.
How sequential actions hide private signals
In the information cascade model, a person observes the actions of predecessors, but not their personal information. Reviewing independent assessments and such sequential learning are different tasks. Primary work by Bikhchandani, Hirshleifer and Welch shows the possibility of a situation where the action ceases to reveal private information.[1](#source-cp-cascades) The following example is constructed anew; it does not reproduce market quotes.
There are two states of a conditional project: good and bad, prior probabilities in half. Participants take turns choosing “accept” or “reject,” receiving a one for correctly identifying the condition and a zero for an error. The price and payments do not change due to other people's decisions. Everyone receives a signal that is independent for a given state: it is true with probability 75%. Everyone knows the rules and sees previous actions, but not the signals. If the final probabilities are equal, we will agree to follow our signal. It is this draw rule that is important for the example.
The first one sees a positive signal and accepts it. His action reveals this signal. The second one also sees the positive and accepts it. After two such actions, the ratio of the probabilities of a good and bad state is (0,75/0,25)² = 9; the probability of a good condition is 9/(1 + 9) = 90%.
The third sees a negative signal. It reduces the odds ratio by a factor of three, but leaves it at three; the final probability of a good state is 75%. Therefore, the third one also accepts, although his private observation is negative. If the signal were positive, it would also receive. Its effect is the same for both signals: it no longer tells the next participant what exactly he learned.
The fourth has no right to consider three acceptances as three independent positive signals. He knows the third action didn't reveal anything and keeps the public odds 9 to 1. His own negative signal again leaves the advantage of a good condition. This way, subsequent steps can be repeated without adding independent information. The number of consonants is growing, but the information content of the sequence is not.
The cascade can be flawed and fragile
In a really bad state, the first two partial signals can still turn out to be positive: the probability of such a particular start is equal to 0,25² = 6,25%. Under our rules, it triggers false acceptance. This is the conditional probability of one path in a bad condition, not the overall probability of all erroneous cascades, much less the frequency of market bubbles. To calculate the overall indicator, you need to take into account all possible beginnings, negative cascades, draws and a priori probabilities.
Now let's introduce a new publicly available test that gives the likelihood ratio of 20 to 1 in favor of the bad state. The public odds of good will become 9/20, that is, below one. New information has the potential to destroy what seemed like a massive consensus. Therefore, a sudden reversal does not necessarily mean that each participant simultaneously received an independent new observation. At the same time, the model does not claim that any reversal is explained by a cascade.
Separate informational imitation from reputational imitation: a person may follow others not for the sake of a better conclusion about the project, but in order not to differ from the group or not to lose a client. The desire to have “no worse than those of friends” adds another motive; it does not improve signal accuracy. The popularity of a product or the fame of a company also does not indicate whether the price already paid is profitable. The pleasure of participating in a discussion and the expected monetary result are different goals; the high price tag of a compelling story may leave little room for subsequent income.
Practical verification consists of restoring the origin of information: who made an independent observation, who saw only the action, who retells one author. Saving the original estimates helps detect lost signals, but does not guarantee correctness. The financial market has changing prices, unequal payments, restrictions, partially informative volumes and new public news. Therefore, this model explains one mechanism of information dependence, but does not predict price direction, the scale of contagion, or the moment of reversal. Analogies to animals, fashion, and infections are no substitute for testing these conditions.