That’s Scott Adams (of Dilbert fame):
Advocates – for anything – generally present their arguments as absolutes, in the form of “This is 100% right and the alternative is 100% wrong.” That might make sense for some topics, but does it ever make sense for a complicated issue, such as economics?
We see this in the public sphere all the time. One of Philip Tetlock’s observations about foxes and hedgehogs is that the more accurate foxes seem less convincing than the less accurate hedgehogs. Media consumers don’t find it entertaining or compelling to listen to pronouncements from pundits who hedge their bets and calibrate their confidence levels. They’re biased towards certainty.
I have a personal bias that only idiots have certainty about complicated issues. (The exception would be skeptics who don’t believe in magic, religious or otherwise. I give them a pass for being 100% certain.) So when [economist Paul] Krugman, who is brilliant, displays certainty on the economy – with his Nobel Prize and all – my brain automatically conflates him with idiots, and it weakens his argument.
So I wonder if it’s just me. When you hear an argument about a complex issue presented as a certainty, do you reflexively downgrade its value? Or does the certainty mixed with a credible source make it more persuasive to you?
Personally, yes to the former. My default response to a claim of certainty is to assume that I’m hearing an advocacy claim rather than an analytical claim. In making that judgement I’m responding to various cues, some of which I can account for consciously. I’m assuming, for example, that the Paul Krugman who writes a column titled The Conscience of a Liberal for the Opinion section of The New York Times is in some sense a different Paul Krugman to the one who publishes in the International Tax and Public Finance academic journal, even when both are talking about macroeconomics. But how do I know I’m reading these cues correctly?
I’ve written previously about ambiguous language in Business Analytics. Software vendors mean different things when they talk about “analytics”; Finance departments run three substantively distinct processes in parallel, all called “forecasting”. Such language either exploits, reflects, or generates uncertainty. This may or may not be conscious or intentional.
As an analytics practitioner communicating analytical claims, then, I need to try to factor in two layers of uncertainty:
- The uncertainty of the claim itself.
- The uncertainty surrounding communications of the claim.
The second will reflect my understanding of things like the communication context (e.g. a board meeting), the analytical literacy of my audience, my own ability to communicate clearly to them, and their expectations of and assumptions about me. I need to build models for all of these things too.
Related Analyst First posts:
- *What’s Wrong with Expert Predictions*
- *What’s Wrong with Expert Predictions* – Commentary
- Hedgehogs are foxy when they’re right
- Vendor worldviews evolve
- Forecasting, goal setting, planning
About usAnalyst First is a new approach to analytics, where tools take a far less important place than the people who perform, manage, request and envision analytics, while analytics is seen as a non-repetitive, exploratory and creative process where the outcome is not known at the start, and only a fraction of efforts are expected to result in success. This is in contrast with a common perception of analytics as IT and process.
Tags in a CloudAIPIO analyst first Analyst First Chapters analytics analytics is not IT arms race environments big data business analytics business intelligence cargo cults collective forecasting commodity and open source tools complexity data decision automation decision support educated buyer EMC-greenplum forecasting HBR holy trinity human infrastructure incentives intelligence model of analytics investing in data lean startup literacy management culture MBAnalytics operational analytics organisational-political considerations Philip Russom Philip Tetlock prediction markets presales R Robin Hanson Strategic Analytics tacit data TDWI Tom Davenport uncertainty uneducated buyer vendors why analyst first