*My Bias against Certainty*

That’s Scott Adams (of Dilbert fame):

Advocates – for anything – generally present their arguments as absolutes, in the form of “This is 100% right and the alternative is 100% wrong.” That might make sense for some topics, but does it ever make sense for a complicated issue, such as economics?

We see this in the public sphere all the time. One of Philip Tetlock’s observations about foxes and hedgehogs is that the more accurate foxes seem less convincing than the less accurate hedgehogs. Media consumers don’t find it entertaining or compelling to listen to pronouncements from pundits who hedge their bets and calibrate their confidence levels. They’re biased towards certainty.

I have a personal bias that only idiots have certainty about complicated issues. (The exception would be skeptics who don’t believe in magic, religious or otherwise. I give them a pass for being 100% certain.) So when [economist Paul] Krugman, who is brilliant, displays certainty on the economy – with his Nobel Prize and all – my brain automatically conflates him with idiots, and it weakens his argument.

So I wonder if it’s just me. When you hear an argument about a complex issue presented as a certainty, do you reflexively downgrade its value? Or does the certainty mixed with a credible source make it more persuasive to you?

Personally, yes to the former. My default response to a claim of certainty is to assume that I’m hearing an advocacy claim rather than an analytical claim. In making that judgement I’m responding to various cues, some of which I can account for consciously. I’m assuming, for example, that the Paul Krugman who writes a column titled The Conscience of a Liberal for the Opinion section of The New York Times is in some sense a different Paul Krugman to the one who publishes in the International Tax and Public Finance academic journal, even when both are talking about macroeconomics. But how do I know I’m reading these cues correctly?

I’ve written previously about ambiguous language in Business Analytics. Software vendors mean different things when they talk about “analytics”; Finance departments run three substantively distinct processes in parallel, all called “forecasting”. Such language either exploits, reflects, or generates uncertainty. This may or may not be conscious or intentional.

As an analytics practitioner communicating analytical claims, then, I need to try to factor in two layers of uncertainty:

1. The uncertainty of the claim itself.
2. The uncertainty surrounding communications of the claim.

The second will reflect my understanding of things like the communication context (e.g. a board meeting), the analytical literacy of my audience, my own ability to communicate clearly to them, and their expectations of and assumptions about me. I need to build models for all of these things too.

Related Analyst First posts:

• *What’s Wrong with Expert Predictions*
• *What’s Wrong with Expert Predictions* – Commentary
• Hedgehogs are foxy when they’re right
• Vendor worldviews evolve
• Forecasting, goal setting, planning