The AI Journal

In one experiment, people could pay a lottery ticket to learn how accurate their adviser had actually been at estimating a stranger’s weight from a photograph. The more confident the adviser sounded, the less they wanted to pay. That is the puzzle of confidence in a single result. Confidence is usually described as a stand-in for competence, a signal we use when we cannot cheaply check the real thing. But it does not behave like a stand-in, waiting to be exchanged for the genuine article. It behaves like a tax on looking. The more confident the source, the fewer people bother to find out whether the source was any good, and the premium survives because nobody collects the evidence that would end it.

The status premium itself is well documented. In six studies, Cameron Anderson and his colleagues found that overconfident people attained higher social status than their equally able but better-calibrated peers, in both short-lived and longer-term groups, and that the mechanism was perceptual: overconfidence produced a behavioural signature (speaking more, a confident and factual vocal tone, more task-relevant information, a calm and relaxed demeanour, and answering first) that made the person look more competent than they were (Anderson et al., 2012). The effect did not require the confident person to be right. It required only that the audience have no way to tell.

You might expect such a premium to be temporary, a first impression corrected on acquaintance. It is not. In three further studies, Jessica Kennedy and her colleagues told groups what they had not known the first time: that the confident member’s self-assessment had been inflated. The groups did not turn on them. Members revealed as overconfident were still viewed positively; the status they had won with bravado outlasted the disclosure that the bravado was baseless (Kennedy et al., 2013). This is the finding that should unsettle anyone who trusts the long run. The long run, in these studies, had the information in hand and declined to use it.

Part of the reason is that audiences do not merely tolerate confidence; they prefer it. In a set of experiments by Paul Price and Eric Stone, people chose between two advisers, one well calibrated and one overconfident, judging the likelihood that shares would rise. Participants tended to prefer the overconfident one, and treated his certainty as evidence of greater knowledge and of more correct judgements, even though it was not (Price & Stone, 2004). Confidence was read as a report about the world when it was only a report about the speaker.

But here the picture has to be split, because the premium is not one thing. Status, the general rank a group gives a person, is sticky in the way just described. Credibility for a particular claim is far more fragile. In two experiments, Elizabeth Tenney and her colleagues asked people to judge witnesses who answered questions with varying confidence and accuracy. Confident witnesses were believed more, but only until they erred: an error damaged the credibility of a confident witness more than that of a diffident one, and after a mistake the less confident witness could end up the more credible of the two (Tenney et al., 2007). The audience, in other words, does track calibration, but only when it is given something to track. When a specific claim is visibly wrong, confidence becomes a liability rather than a shield.

So the premium is not a fixed property of confident people. It is a property of how much the audience can afford to check, and it is strongest exactly where checking is expensive. Sunita Sah, Don Moore and Robert MacCoun ran both sides of that comparison. When participants received feedback about an adviser’s accuracy, confidence stopped paying: accurate advisers benefited from sounding sure, while confident but inaccurate advisers were rated as less credible. When feedback was unavailable or costly, the same confident advisers held sway regardless of whether they had been right (Sah et al., 2013). Confidence did not have to be earned to be effective. It only had to be uncheckable.

And confidence keeps itself uncheckable. In the same experiment, participants who heard a more confident adviser showed less interest in buying the performance data that would have revealed whether the advice was sound. The signal did not merely replace the evidence. It lowered the demand for it, which is why the belief that hubris is eventually found out is less a description of how things work than a hope. Disclosure does not undo the premium, because disclosure is not verification. What undoes it is an imposed, cheap, unavoidable check, and that is rarely what a group volunteers.

Here is the strongest version of the objection. These are laboratory studies of status among students, with small stakes and short horizons, and the claims prove too much. In the institutions that actually carry risk, hubris is punished, and rightly so. Auditors, regulators, rival firms, liability, and the mere passage of time discipline confidence, because they attach a record to it. Bankers who oversell their models are fired or ruined; surgeons who overclaim lose referrals; politicians who bluff are fact-checked within the hour. If overconfidence survived in the wild as well as it survives in these experiments, the world would be run by blowhards, and it visibly is not.

The objection is right about institutions, and wrong about what it proves. It is right that where a check is forced, the premium collapses: the feedback condition in Sah’s experiments is exactly such a world, and there confidence stops working. But notice what that concedes. The discipline comes from the machinery that imposes the check, not from the natural revelation of overconfidence. Kennedy’s groups were handed the information and still did not penalise. By the objection’s own account, confidence is disciplined only where verification is made compulsory and cheap, which is a claim about institutions, not about people seeing through one another. And the machinery is needed precisely because audiences will not seek the check for themselves. Where the record exists but consulting it is optional, confidence does the work.

The confusion underneath the folk belief is that overconfidence is a single thing. Don Moore and Paul Healy showed that it is at least three: overestimation of one’s own performance, overplacement of oneself relative to others, and overprecision in one’s certainty (Moore & Healy, 2008). They have different causes, and, I want to suggest, different fates. The punishment people have in mind attaches to a measured error on a specific claim, which is the Tenney case. It does not attach to the looseness of certainty in general, which is the Anderson and Kennedy case. A society can be full of people whose overconfidence is never docked and still punish the particular ones whose errors are caught, and that is the more likely state of things than the tidy one in which truth always collects.

Which changes the question worth asking of a confident voice. Not whether the person is right, but who is paying for the check, and whether we would have bought it. The confidence premium is set by the cost of verification, and confidence pushes that cost up while pretending to lower it. It is not a verdict on the confident, and it is not evidence about the world. It is a description of the position of everyone listening. It persists for as long as looking is optional.