Currently viewing the tag: "uncertainty"
Eric Ries, Harvard Business School’s ‘Entrepreneur-in-Residence’ and the founder of Lean Startup, has been interviewed a number of times recently following the launch of his Lean Startup book. As we have argued before at Analyst First, the ‘unknown problem / unknown solution’ domain—in which and for which the Lean Startup approach was developed—reflects the world of Business Analytics. In the 12 to 18 months since giving the last interview we linked to, Ries has significantly enriched the ideas of Lean Startup. Two new interviews are highly recommended. The first is from the Commonwealth Club of California’s Inforum program. Details here and audio file here, or here. The second is from the ITConversations network’s Tech Nation program. Details and download here.
Ries takes it as axiomatic that enterpreneurs seek to create “institutions of lasting value” in an unstable environment. He defines a startup as any “human institution designed to create something new under conditions of uncertainty”. The role of the Lean Startup toolkit is to help entrepreneurs navigate the best path in this context. Most contemporary management tools have their heritage in twentieth century manufacturing and are based on forecasting and planning. They assume that the world is stable enough to be predicted such that plans can be reliably devised and executed. As Ries points out, and should be obvious, this assumption simply doesn’t hold in the environments in which many of us now work.
In the twenty-first century we can build almost anything that can be imagined. The challenge is not to build more stuff. It’s to build the right stuff. Most startups fail, says Ries, because they make the wrong things. The key activity of a startup should therefore be learning, not building. What creates value for a startup is it determining whether or not it’s on the path to a sustainable business.
Lean Startup is a scientific approach to new product development which treats everything a startup does as an experiment. The goal is to collect data (feedback about what customers want) with minimal cost, not to build to a pre-determined product specification (which assumes what customers want) with minimal cost. In service of this, the Lean Startup movement is developing ‘innovation accounting’, an attempt to revolutionise the existing accounting paradigm so that it can operate under conditions of uncertainty and instability. The current planning-based paradigm (are we on time, on budget?) is unable to distinguish between the threshold of success and the brink of failure. The core question being asked by innovation accounting is, instead: are the experiments the team’s doing affecting customer behaviour?
Clearly there are implications here for Business Analytics. We’ve often written here of the uncertainty inherent in data-driven analysis, and of the unsuitability of the default IT project plan-based, build-centric, waterfall approach to what is inescapably an exploratory and learning-oriented set of activities. Much of the Lean Startup approach translates directly into the Analytics Lab. However, the constraints faced by those attempting to innovate from within already established organisations (who Ries terms ‘intrapreneurs’) are not the same as those which frame the entrepreneurial enterprise. Entrepreneurs operate until the financial capital provided to them by venture capitalists runs out. The ‘lean’ in Lean Startup seeks to maximise the number of experiments they can run before this happens. The entrepreneurs described by Ries all enjoy a large and fundamentally interchangeable prospective customer base. Many unsuccessful experiments can be run on different user populations in search of a loyal base, essentially without consequences. Unhappy users don’t stick around. This is not the case for intrapreneurs. Within an organisation there are only a limited number of prospective analytics customers, and disappointing any one group leaves a legacy. The scarce resource for intrapreneurs is political capital.
Related Analyst First posts:
Is Big Data a Bubble?
In case you’re in a hurry: Of course it is. And that is good.
Quentin Hardy in The New York Times Bits blog summarises the state of play regarding the business world’s interest in and utilisation of big data. As he recognises, big data is really about “the benefits we will gain by cleverly sifting through it to find and exploit new patterns and relationships.” Big data drives up the obviousness of the need for analytics. Its often invoked qualities of volume, velocity, and variety mean that we can’t pretend to analyse it without statistical and machine learning methods.
Big data analytics at the present moment, however, is characterised by uncertainty. What is big data, exactly? What’s its value? How is it analysed? Which technologies should be used? Which standards will prevail? Where to invest? There is, Hardy finds:
a common problem in the Big Data proposition: Often people won’t know exactly what hidden pattern they are looking for, or what the value they extract may be, and therefore it will be impossible to know how much to invest in the technology. Odds are that the initial benefits, as it was with Google’s Adwords algorithm, will lead to a frenzy of investments and marketing pitches, until we find the logical limits of the technology. It will be the place just before everybody lost their shirts.
This is a common characteristic of technology that its champions do not like to talk about, but it is why we have so many bubbles in this industry. Technologists build or discover something great, like railroads or radio or the Internet. The change is so important, often world-changing, that it is hard to value, so people overshoot toward the infinite. When it turns out to be merely huge, there is a crash, in railroad bonds, or RCA stock, or Pets.com. Perhaps Big Data is next, on its way to changing the world.
Such is technology. Hence the level of uncertainty, and also reasons for optimism:
There are an uncountable number of data-mining start-ups in the field: MapReduce and NoSQL for managing the stuff; and the open-source R statistical programming language, for making predictions about what is likely to happen next, based on what has happened before. Established companies in the business, like SAS Institute or SAP, will probably purchase or make alliances with a lot of these smaller companies.
Expect to see a lot more before it all gets sorted out.
TDWI’s recent Big Data Analytics report by Philip Russom, based on 325 sets of responses to a May 2011 survey, echoes this and provides a far more comprehensive overview of how businesses are coping with the realities of big data. One measure of how unsettled the field is right now is that, in response to the question ‘Which of the following best characterizes your familiarity with big data analytics and how you name it?’, 65% of the survey’s respondents replied, ‘I know what you mean, but I don’t have a formal name for it.’
The report is recommended in full, but in case you’re still in a hurry, its margin summaries can be skimmed through in a matter of minutes.
Related Analyst First posts:
Week 1, Day 5 of the CORTEX MBAnalytics program includes Tom Davenport’s ‘Rethinking Knowledge Work: A Strategic Approach’ from the McKinsey Quarterly of January 2011. In the essay, Davenport argues that productivity software hasn’t boosted the productivity of “knowledge workers” to the extent hoped for given the outlays of the last two decades. The primary method employed over this period has been what he calls ‘free-access’: providing knowledge workers with tools and information and leaving it to them to work out what to do with them:
In this model, knowledge workers define and integrate their own information environments. The free-access approach has been particularly common among autonomous knowledge workers with high expertise: attorneys, investment bankers, marketers, product designers, professors, scientists, and senior executives, for example. Their work activities are seen as too variable or even idiosyncratic to be modeled or structured with a defined process.
This approach suits when there is uncertainty, ambiguity, and contingency, each of which work against predictability. The upside is the ability of humans to adapt to these. The downside is that autonomy doesn’t come for free. Workers will execute variably, some poorly. The lack of standardisation leads to duplication and other kinds of inefficiency. Precise performance measurement and management is also a challenge. Typical productivity metrics in the free-access domain are rough and high level if present at all, and there is a trade-off between additional measurement and ease of information access.
The alternative model Davenport terms ‘structured-provisioning’, in which tasks and deliverables are defined and knowledge workers slotted in. Typical examples are workflow or ‘case management’ systems, which integrate decision automation, content management, document management, business process management, and collaboration technologies:
Case management can create value whenever some degree of structure or process can be imposed upon information-intensive work. Until recently, structured-provision approaches have been applied mostly to lower-level information tasks that are repetitive, predictable, and thus easier to automate.
The upside is efficiency. The downsides are worker alienation and resistance, and detrimental business outcomes resulting from complexity and poor specification—bad mortgages, for example.
Davenport believes that businesses should increasingly “structure previously unstructured processes”. That is, that the free-access domain should be progressively structure-provisioned. He uses a 2 x 2 matrix to frame his argument. On the x-axis is ‘Complexity of work’, ranging from Routine across to Intepretation/judgement. On the y-axis is ‘Level of interdependence’, ranging from Individual actors up to Collaborative groups. The resulting knowledge work quadrants are:
- Transaction model (Routine x Individual actors)
- Expert model (Interpretation/judgement x Individual actors)
- Integration model (Routine x Collaborative groups)
- Collaboration model (Interpretation/judgement x Collaborative groups)
The Transaction model contains most existing structure-provisioning, and the Collaboration model—consisting of “Improvisational work”, being “Highly reliant on deep expertise across multiple functions”, and “Dependent on fluid deployment of flexible teams”—is inherently free-access. Davenport sees the Expert and Integration models, however, as open to further structured-provisioning.
Martin Ford’s book, The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future, (free as a PDF download), further illuminates these trends. Ford identifies three categories of job vulnerable to displacement by technology:
- Hardware jobs, such as assembly line jobs, which become displaced by robotics—a process which is already well underway.
- Software jobs, such as radiology, which are first displaced by outsourcing, then by AI.
- Interface jobs, such as loan officers, which become displaced by telecommunications, digitisation, and data standardisation.
‘Rethinking Knowledge Work’ is an interesting change of direction for Davenport. His seminal ‘Competing on Analytics‘ essay, and the book that followed, profiled business effectiveness and adaptiveness powered by analytics. The arguments here, by contrast, are all about efficiencies.
[T]o date, high-end knowledge workers have largely remained free to use only the technology they personally find useful. It’s time to think about how to make them more productive by imposing a bit more structure. This combination of technology and structure, along with a bit of managerial discretion in applying them to knowledge work, may well produce a revolution in the jobs that cost and matter the most to contemporary organizations.
Given the vulnerability of so much knowledge work to displacement, it’s a good time to be an analyst. Business Analytics clearly lives in the “Expert model” quadrant. Further to that, Davenport sees it as playing a role in augmenting other expertise within that domain:
Expert jobs may also benefit from “guided” data-mining and decision analysis applications for work involving quantitative data: software leads the expert through the analysis and interpretation of data.
This further validates Analyst First principles, namely our insistence on the importance of human over electronic infrastructure, our conception of Business Analytics as an intelligence rather than IT function, and our focus on strategic in preference to operational analytics.
Related Analyst First posts:
That’s Scott Adams (of Dilbert fame):
Advocates – for anything – generally present their arguments as absolutes, in the form of “This is 100% right and the alternative is 100% wrong.” That might make sense for some topics, but does it ever make sense for a complicated issue, such as economics?
We see this in the public sphere all the time. One of Philip Tetlock’s observations about foxes and hedgehogs is that the more accurate foxes seem less convincing than the less accurate hedgehogs. Media consumers don’t find it entertaining or compelling to listen to pronouncements from pundits who hedge their bets and calibrate their confidence levels. They’re biased towards certainty.
I have a personal bias that only idiots have certainty about complicated issues. (The exception would be skeptics who don’t believe in magic, religious or otherwise. I give them a pass for being 100% certain.) So when [economist Paul] Krugman, who is brilliant, displays certainty on the economy – with his Nobel Prize and all – my brain automatically conflates him with idiots, and it weakens his argument.
So I wonder if it’s just me. When you hear an argument about a complex issue presented as a certainty, do you reflexively downgrade its value? Or does the certainty mixed with a credible source make it more persuasive to you?
Personally, yes to the former. My default response to a claim of certainty is to assume that I’m hearing an advocacy claim rather than an analytical claim. In making that judgement I’m responding to various cues, some of which I can account for consciously. I’m assuming, for example, that the Paul Krugman who writes a column titled The Conscience of a Liberal for the Opinion section of The New York Times is in some sense a different Paul Krugman to the one who publishes in the International Tax and Public Finance academic journal, even when both are talking about macroeconomics. But how do I know I’m reading these cues correctly?
I’ve written previously about ambiguous language in Business Analytics. Software vendors mean different things when they talk about “analytics”; Finance departments run three substantively distinct processes in parallel, all called “forecasting”. Such language either exploits, reflects, or generates uncertainty. This may or may not be conscious or intentional.
As an analytics practitioner communicating analytical claims, then, I need to try to factor in two layers of uncertainty:
- The uncertainty of the claim itself.
- The uncertainty surrounding communications of the claim.
The second will reflect my understanding of things like the communication context (e.g. a board meeting), the analytical literacy of my audience, my own ability to communicate clearly to them, and their expectations of and assumptions about me. I need to build models for all of these things too.
Related Analyst First posts:
- *What’s Wrong with Expert Predictions*
- *What’s Wrong with Expert Predictions* – Commentary
- Hedgehogs are foxy when they’re right
- Vendor worldviews evolve
- Forecasting, goal setting, planning
Broadly speaking all Business Analytics serves one of two goals: decision support or decision automation. One way to idealise these is as either reports (decision support) or algorithms (decision automation).
Algorithms reduce the need for humans to think. Picture the in-database credit scoring function embedded deep in your bank’s systems and firing thousands of times an hour. This kind of decision automation (or decision replacement) is a common operational analytics endpoint.
Reports, on the other hand, make decisions more difficult. The simplest decision support system is a coin toss, but a business relying only on heads and tails will not survive for long. Real decision support adds ambiguity, complexity, uncertainty, and necessitates human judgement. This makes decisions harder, not easier.
So what is Analyst First all about?
In a nutshell, it is about making analytics cheaper, more relevant and appropriate to business (which can includes government, NGOs and any other folks actually using analytics to do something other than research for its own sake). It is also about presenting a radically different model of analytics to the one currently seen by most of the market.
Does this mean that it is not being done well already? Well… Let’s say that it could be done a whole lot better.
The biggest problem is: most people think that analytics is about software, when it is actually about people.
What does this mean? It means that buying very expensive software that the buyers do not understand and do not have the staff to select appropriately – let alone use – is a lousy way to get going with analytics.
On the other hand, investing in people might just be the right idea. Investing in people does mean getting skilled analysts before software. Hence “Analyst First”.
But this is only the beginning, getting us to the first key principle of Analyst First:
Invest in Smarts: Build The Human Infrastructure First
This means getting highly skilled experts in analytics to advise, demonstrate and trial a range of techniques, mentoring the new team.
It means carefully building an appropriate team of analysts, business experts, communicators and data manipulators (yes, they are different skill sets).
More importantly it means establishing the right channels, expectations and incentives to gently educate executives about what they can ask of analytics, and what they need to provide to make it happen.
This may sound hard enough, but what does the team work with if there is no software? But there is:
Use Free, Commodity and Open Source Tools First
Tools on the desktop, such as MS Excel and Access, are more than enough for most analytics tasks attempted by beginners, or business areas trying out analytics.
If serious power is required, tools like R, Rapidminer, Knime etc will probably do. These tools are free, and industrial-strength enough for most applications. Certainly worth trying first, and perhaps sticking with.
Here is some evidence that this is catching on.
In our experience, commodity and open source tools are good enough 95% of the time – 100% percent of the time for a new analytics unit. In the latter case, the unit is not entirely sure how analytics may be applied in their business, and their first job is to find out, capturing executive support in the process.
This is a big ask, made all the bigger if big $$$ have been spent on software, and a small number of mediocre staff are hired as an afterthought.
On the other hand, commodity and open source tools are a great alternative, allowing the money to be spent on human infrastructure.
Analyst First is not against buying expensive vendor tools, but it is against spending a cent on software until the buyer is an Educated Buyer, having used commodity and open source tools extensively, found their limitations, and seen a specific need for an expensive vendor solution.
Educated Buyers cut their teeth on readily available, inexpensive tools first, and invest their money in people: staff, skills, consultants, mentoring.
The Practice of Analytics: Exploration, Learning, and Making Mistakes
Analytics is not a linear process, like most engineering projects. Its end product is discovery: you cannot determine what will be discovered ahead of time. Thus the outcomes of analytics, and the decisions based on them, cannot be made before the analysis has been carried out.
Further, analytics is inherently exploratory in nature: data is a treacherous beast, and there may be many dead ends. Not all analytics exercises end in brilliant findings, accurate models or actionable insights. Nor should they. Mistakes are how we learn. The trick is to make them quickly, minimise their cost, learn from them and move on.
This organic, exploratory approach fits perfectly with a view that Analytics is an Intelligence activity.
Where it fits less well is with the view that analytics is IT…
Analytics is not IT
While accounting, graphic design, journalism and medicine are not part of the IT function, they all use a heck of a lot of IT.
Analytics done right is no different. While analysts require some very smart software to do their jobs, the software is not the star of the show. A strategic intelligence function does not belong within IT.
The fact that analytics relies on highly sophisticated software should not serve to downplay the far greater role of people.
The exploratory, organic growth, human centric model outlined above is in stark contrast with the IT view of the world, which is all about pre-defined systems, known outcomes and large, top-down specified projects. An intelligence-based, human-centric model has no chance to flourish within such an environment.
Finally, most of the issues concerning IT in the context of analytics apply in the operational deployment of analytics results (eg campaign models), but apply far less in the context of analytics proper: the exploration, description and modelling of data.
Issues around security, real-time effectiveness, transfer bottlenecks etc are not real issues in most analytics contexts, though they are thrown around by IT departments.
More later.
Meanwhile, here is a video of the first A1 presentation, with slides.
Follow Us!
