Currently viewing the tag: "analyst first"
On Tuesday night I presented Getting started with Predictive Analytics in the Public Sector to a public meeting of Analyst First in Canberra.
The presentation itself is an update of one given in June to Canberra’s IBM Business Analytics User Group. For this version I added material describing how analytics supports the risk management cycle, and incorporating some insights from Jim Manzi’s excellent Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society.
Part 1 of the highly recommended Uncontrolled covers the evolution of the scientific method (from Bacon on experimentation, to Hume on induction, to Popper on falsification, to Kuhn on scientific paradigms, through to the present day). Part 2 looks at the development of randomised field trials in the latter half of the twentieth century and their applications in medicine and business (i.e. analytics). Part 3 advocates the more widespread and systematic use of randomised field trials to areas of public policy, learning from the business experiment revolution.
Our thanks to BAE Systems for providing the venue.
Eric Ries, Harvard Business School’s ‘Entrepreneur-in-Residence’ and the founder of Lean Startup, has been interviewed a number of times recently following the launch of his Lean Startup book. As we have argued before at Analyst First, the ‘unknown problem / unknown solution’ domain—in which and for which the Lean Startup approach was developed—reflects the world of Business Analytics. In the 12 to 18 months since giving the last interview we linked to, Ries has significantly enriched the ideas of Lean Startup. Two new interviews are highly recommended. The first is from the Commonwealth Club of California’s Inforum program. Details here and audio file here, or here. The second is from the ITConversations network’s Tech Nation program. Details and download here.
Ries takes it as axiomatic that enterpreneurs seek to create “institutions of lasting value” in an unstable environment. He defines a startup as any “human institution designed to create something new under conditions of uncertainty”. The role of the Lean Startup toolkit is to help entrepreneurs navigate the best path in this context. Most contemporary management tools have their heritage in twentieth century manufacturing and are based on forecasting and planning. They assume that the world is stable enough to be predicted such that plans can be reliably devised and executed. As Ries points out, and should be obvious, this assumption simply doesn’t hold in the environments in which many of us now work.
In the twenty-first century we can build almost anything that can be imagined. The challenge is not to build more stuff. It’s to build the right stuff. Most startups fail, says Ries, because they make the wrong things. The key activity of a startup should therefore be learning, not building. What creates value for a startup is it determining whether or not it’s on the path to a sustainable business.
Lean Startup is a scientific approach to new product development which treats everything a startup does as an experiment. The goal is to collect data (feedback about what customers want) with minimal cost, not to build to a pre-determined product specification (which assumes what customers want) with minimal cost. In service of this, the Lean Startup movement is developing ‘innovation accounting’, an attempt to revolutionise the existing accounting paradigm so that it can operate under conditions of uncertainty and instability. The current planning-based paradigm (are we on time, on budget?) is unable to distinguish between the threshold of success and the brink of failure. The core question being asked by innovation accounting is, instead: are the experiments the team’s doing affecting customer behaviour?
Clearly there are implications here for Business Analytics. We’ve often written here of the uncertainty inherent in data-driven analysis, and of the unsuitability of the default IT project plan-based, build-centric, waterfall approach to what is inescapably an exploratory and learning-oriented set of activities. Much of the Lean Startup approach translates directly into the Analytics Lab. However, the constraints faced by those attempting to innovate from within already established organisations (who Ries terms ‘intrapreneurs’) are not the same as those which frame the entrepreneurial enterprise. Entrepreneurs operate until the financial capital provided to them by venture capitalists runs out. The ‘lean’ in Lean Startup seeks to maximise the number of experiments they can run before this happens. The entrepreneurs described by Ries all enjoy a large and fundamentally interchangeable prospective customer base. Many unsuccessful experiments can be run on different user populations in search of a loyal base, essentially without consequences. Unhappy users don’t stick around. This is not the case for intrapreneurs. Within an organisation there are only a limited number of prospective analytics customers, and disappointing any one group leaves a legacy. The scarce resource for intrapreneurs is political capital.
Related Analyst First posts:
Ted Cuzzillo, writing at TDWI and citing Blake Johnson of Stanford, identifies 6 conditions for [or barriers to] the rise of business analysts:
1. The best analysts are skilled in three areas: First, they engage stakeholders and have an eye for business opportunity. Second, they inspire stakeholders’ trust with consistently excellent analysis. Third, “big data” requires skill with data management and software engineering.
This paints a similar but not identical picture to Drew Conway’s Data Scientist Venn Diagram. The key point of difference is that Conway places more weight on mathematical and statistical training, which is not the same thing as “consistently excellent analysis”, but is more important than is often assumed in enabling it.
2. Each analyst’s skills should be about 80 percent in data management and about 20 percent in business and analytics — but Johnson expects that to change over the next five or 10 years as tools make data management easier. Eventually the mix of skills will be the opposite: 20 percent data management and 80 percent business.
I have no strong view on this, but my intuition is that data wrangling will always consume far more time and effort than analysis. Analysis is a feedback loop and a read-write activity. Standardisation and automation continue to consolidate efficiencies but these tend to raise the analytical bar. That said, I’d be happy for future tools to prove me wrong.
3. Gaining a foothold within an organization is best done in small bites with an entrepreneurial approach. Forget trying for a “big bang,” he says. Instead, find a need and fill it quickly, then move on to others. Identify and solve one business problem after another — always making sure to keep your methods scalable.
This agrees with Analyst First’s contention, seconded by others, that the monolithic IT project approach doesn’t work, and that—within an existing organisation—a bottom-up Lean Startup approach is your best bet. The only exceptions to this are analytic-centric online startups and quantitative hedge funds.
4. Location of analysts’ workspace matters. They should work in a cluster for critical mass, which encourages sharing of best practices and support. If they sit within business teams, their work becomes more visible.
This makes sense. Isolated analysts are a problem whether they’re isolated from each other or from management oversight and direction. Generally speaking, senior executives need to be broadened while analysts need to be narrowed. Middle managers need to be skilled up to bridge between the two.
5. It’s an adjustment for everyone — on the business side but especially on the IT side. It means fundamental changes in the way data is organized and managed, and accessed and used, with both new technologies and skill sets.
6. Many IT pros deny access to data based on obsolete knowledge. Johnson reports that many don’t know about modern load-balancing and other technology that make such access safer.
Certainly true. I’ve written before about the data needs of analysts as distinct from traditional business intelligence consumers, and also observed that big data is at once driving up the need for advanced analytics and rendering traditional data warehousing approaches obsolete. But the odd part about the commonly invoked ‘IT vs business’ balance of power is the acceptance of IT as a ‘stakeholder’ as opposed to an enabler. It’s unquestionably the case that analytics doesn’t happen without software, but that’s just as true of accounting, graphic design, and most other activities conducted in front of computers in today’s workplace. It simply doesn’t follow that IT deserves, so to speak, a seat on the Security Council.
Cuzzillo closes well aware of both the future possibilities for Business Analytics, and the status quo political realities standing in its way:
You would think that both sides would sign up for the bargain the new middlemen [i.e. analysts] seem to offer. IT would cede control and concentrate on what it does best, managing the back end. Meanwhile, business stakeholders would get insights from these newly empowered, eager specialists. Analysts would be newly ready to answer business questions, conjure up new questions, and offer strategic options.
Analysts would colonize what had been the no-man’s-land between IT and business. Trouble is, the analysts may end up ruining the neighborhood for them. If the strategies Johnson suggests work, IT and business would find a new power growing alongside them. Analysts — simply from the position they would find themselves in, not from any wish to rule the world — would be indispensible, powerful, and well funded.
Who wouldn’t want that?
Related Analyst First posts:
- Analytics Education and Recruitment – Builders vs Finders
- Analysis is read-write
- Forrester on the need for agility
- Analytics Is… A Lean Startup Enterprise
- *Why Software Is Eating The World*
- The data needs of analysts
- Big data as an advanced analytics driver
- *Building for Yesterday’s Future*
- *The Elusive Definition of Agile Analytics*
I presented on Business Analytics and Analyst First to 80+ people at the Corruption Prevention Network‘s 2011 Annual Full Day Forum in Sydney yesterday. The first half of the presentation covered the role of advanced analytic methods in audit and investigation contexts, illustrating with some anonymised examples of link analysis, digital analysis, clustering and predictive modelling. The second half addressed ‘how to get started’ with analytics the Analyst First way. The slides are available on our Resources page.
UPDATE: All papers from the forum (including previous years) are available here.
Week 1, Day 5 of the CORTEX MBAnalytics program includes Tom Davenport’s ‘Rethinking Knowledge Work: A Strategic Approach’ from the McKinsey Quarterly of January 2011. In the essay, Davenport argues that productivity software hasn’t boosted the productivity of “knowledge workers” to the extent hoped for given the outlays of the last two decades. The primary method employed over this period has been what he calls ‘free-access’: providing knowledge workers with tools and information and leaving it to them to work out what to do with them:
In this model, knowledge workers define and integrate their own information environments. The free-access approach has been particularly common among autonomous knowledge workers with high expertise: attorneys, investment bankers, marketers, product designers, professors, scientists, and senior executives, for example. Their work activities are seen as too variable or even idiosyncratic to be modeled or structured with a defined process.
This approach suits when there is uncertainty, ambiguity, and contingency, each of which work against predictability. The upside is the ability of humans to adapt to these. The downside is that autonomy doesn’t come for free. Workers will execute variably, some poorly. The lack of standardisation leads to duplication and other kinds of inefficiency. Precise performance measurement and management is also a challenge. Typical productivity metrics in the free-access domain are rough and high level if present at all, and there is a trade-off between additional measurement and ease of information access.
The alternative model Davenport terms ‘structured-provisioning’, in which tasks and deliverables are defined and knowledge workers slotted in. Typical examples are workflow or ‘case management’ systems, which integrate decision automation, content management, document management, business process management, and collaboration technologies:
Case management can create value whenever some degree of structure or process can be imposed upon information-intensive work. Until recently, structured-provision approaches have been applied mostly to lower-level information tasks that are repetitive, predictable, and thus easier to automate.
The upside is efficiency. The downsides are worker alienation and resistance, and detrimental business outcomes resulting from complexity and poor specification—bad mortgages, for example.
Davenport believes that businesses should increasingly “structure previously unstructured processes”. That is, that the free-access domain should be progressively structure-provisioned. He uses a 2 x 2 matrix to frame his argument. On the x-axis is ‘Complexity of work’, ranging from Routine across to Intepretation/judgement. On the y-axis is ‘Level of interdependence’, ranging from Individual actors up to Collaborative groups. The resulting knowledge work quadrants are:
- Transaction model (Routine x Individual actors)
- Expert model (Interpretation/judgement x Individual actors)
- Integration model (Routine x Collaborative groups)
- Collaboration model (Interpretation/judgement x Collaborative groups)
The Transaction model contains most existing structure-provisioning, and the Collaboration model—consisting of “Improvisational work”, being “Highly reliant on deep expertise across multiple functions”, and “Dependent on fluid deployment of flexible teams”—is inherently free-access. Davenport sees the Expert and Integration models, however, as open to further structured-provisioning.
Martin Ford’s book, The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future, (free as a PDF download), further illuminates these trends. Ford identifies three categories of job vulnerable to displacement by technology:
- Hardware jobs, such as assembly line jobs, which become displaced by robotics—a process which is already well underway.
- Software jobs, such as radiology, which are first displaced by outsourcing, then by AI.
- Interface jobs, such as loan officers, which become displaced by telecommunications, digitisation, and data standardisation.
‘Rethinking Knowledge Work’ is an interesting change of direction for Davenport. His seminal ‘Competing on Analytics‘ essay, and the book that followed, profiled business effectiveness and adaptiveness powered by analytics. The arguments here, by contrast, are all about efficiencies.
[T]o date, high-end knowledge workers have largely remained free to use only the technology they personally find useful. It’s time to think about how to make them more productive by imposing a bit more structure. This combination of technology and structure, along with a bit of managerial discretion in applying them to knowledge work, may well produce a revolution in the jobs that cost and matter the most to contemporary organizations.
Given the vulnerability of so much knowledge work to displacement, it’s a good time to be an analyst. Business Analytics clearly lives in the “Expert model” quadrant. Further to that, Davenport sees it as playing a role in augmenting other expertise within that domain:
Expert jobs may also benefit from “guided” data-mining and decision analysis applications for work involving quantitative data: software leads the expert through the analysis and interpretation of data.
This further validates Analyst First principles, namely our insistence on the importance of human over electronic infrastructure, our conception of Business Analytics as an intelligence rather than IT function, and our focus on strategic in preference to operational analytics.
Related Analyst First posts:
I presented to 40+ members and non-members of Information Systems Audit and Control Association’s (ISACA) Sydney Chapter last night as part of ISACA’s professional development program. The presentation explained data analytics approaches to a group of mostly auditors, contrasting them with CAATS and providing some examples of how we apply them in audit and risk management contexts. The presentation also introduced Analyst First concepts as a means of guiding organisations interested in progressing further with analytics. The slides are available on our Resources page.
I’ve written before about vendor worldviews and their evolution as the competitive landscape changes. One of the interesting characteristics of vendor worldviews is their bias towards symmetric competition. Each vendor focuses most of its competitive attention on its nearest neighbours: those most closely matching its business model and product and service offerings. When I was working for a Comshare distributor a decade ago we worried most about Hyperion. When I was working for Cognos a few years later we worried most about Business Objects. In each case we accepted the paradigm we were placed in – by Gartner, for example – and focused our competitive energies on the minority of features which distinguished us from other occupants of our Magic Quadrant.
It’s easiest, and perhaps most comforting, to understand your competitors in terms of yourself. However, your most challenging competition is asymmetric. It typically comes from outside, it’s often unexpected, and it usually changes your paradigm. Australian newspapers fifteen years ago competed symmetrically with other newspapers for a slice of national, metropolitan or regional market share. Nowadays, as a result of the Internet, they must also compete asymmetrically: with global newspaper brands like The New York Times, and with alternative content generators (blogs, social media, Youtube) and delivery mechanisms (computers, smartphones, tablets).
Business Analytics today is a truly asymmetric marketplace. Megavendors compete with pure-plays. Commercial vendors compete with open source. Software competes with services. Inhouse functions compete with outside providers. The electronic infrastructure competes with the human infrastructure. Strategic focus competes with operational. Top-down competes with bottom-up. Bespoke competes with automated. The IT model competes with the intelligence model. The project-based approach competes with Lean Startup.
The Analyst First worldview recognises that each of these dimensions is its own continuum, that each matters, and that their interactions have substantive implications in terms of likelihood of success, cost and benefit trade-offs, and risk profile.
Related Analyst First posts:
- Same same but different
- Vendor worldviews
- Vendor worldviews evolve
- Measuring the Business Analytics software market
- Strategic First
- Solution buying
- Against best practices in Business Analytics
- Analytics is… Intelligence – The Podcast
- Forrester on the need for agility
- Analytics Is… A Lean Startup Enterprise
Tried and true best practices for enterprise software development and support just don’t work for business intelligence (BI). Earlier-generation BI support centers — organized along the same lines as support centers for all other enterprise software — fall short when it comes to taking BI’s peculiarities into account. These unique BI requirements include less reliance on the traditional software development life cycle (SDLC) and project planning and more emphasis on reacting to the constant change of business requirements.
That is from Forrester Research’s Agile Business Intelligence Solution Centers Are More Than Just Competency Centers report, just released. The full version of the report is paid (USD 499) but a free overview from its two lead authors, Boris Evelson and Rob Karel, is here. The case against the SDLC / project approach is summarised thus:
Earlier-generation BI support organizations are less than effective because they often:
- Put IT in charge
- Remain IT-centric
- Continue to be mostly project-based
- Focus too much on functional reporting capabilities but ignore the data
In response, Forrester advocates a a ‘flexible and agile’ approach to BI, and establishing “BI on BI” to explicitly learn from successes and failures.
This echoes much of what Analyst First advocates, namely that:
- Analytics is not IT
- IT risk management practices hamper Business Analytics initiatives
- It is prudent to assume bad data
- A Lean Startup approach makes more sense
- A good deal of Business Analytics is bespoke
Note that Forrester is making these recommendations at the Business Intelligence end of the Business Analytics spectrum. It’s arguing that, even where Business Analytics lives in an operational, repeatable, systematised, automated, decision automation context:
[No] repository can fully substitute for personal, qualitative knowledge; that’s often more art than science. Therefore, staff the BICC/COE [Business Intelligence Competency Centre / Centre Of Excellence] with individuals whose primary responsibility is to disseminate such knowledge above and beyond what’s available in the repository.
In other words, the human infrastructure is critical, and investments in electronic infrastructure which ignore it will be unsuccessful.
Related Analyst First posts:
Reminder: All welcome for Tony Corke’s presentation at The NTF Group, Suite 318, 5 Lime Street, Sydney. We will need to start at 5-30.
For interstate Members, please use webex link below:
Topic: Profitably modelling AFL football
Date: Tuesday, July 12, 2011
Time: 5:30 pm, Australia Eastern Standard Time (Sydney, GMT+10:00)
Meeting Number: 863 164 522
Meeting Password: Analyst1st
——————————————————-
To start or join the online meeting
——————————————————-
Go to Webex
——————————————————-
Audio conference information
——————————————————-
Please call: 03 8779 7440 (from Melbourne) or 02 9696 0774 (from Sydney)
Your guest access code: 12589350#
——————————————————-
For assistance
——————————————————-
1. Go to Webex
2. On the left navigation bar, click “Support”.
To update this meeting to your calendar program (for example Microsoft Outlook), click this link:
Analyst First argues that Business Analytics should be more prevalent than it is. Two types of barrier prevent organisations from experimenting with and adopting it as they otherwise might. Barriers to entry prevent Business Analytics initiatives from getting started. Barriers to exit prevent them from adapting once they have started by making it difficult or impossible for them to learn from experience.
Common barriers to entry include:
- Cost: If Business Analytics is immediately associated with specialised commercial software, particularly enterprise software, then getting an initiative off the ground will come with a price tag.
- Complexity: If cost drives up the initiating budget then invariably the business case for Business Analytics will increase in complexity, gather more stakeholders and hangers-on, and most likely attract competing and/or mutually exclusive needs and agendas.
- Compliance: A higher budget, higher profile, technically intensive Business Analytics initiative will mean more demanding compliance requirements with internal Procurement, IT, and Finance regimes and processes.
Barriers to exit are all forms of unwanted dependency:
- Capture: Software and technology lock-in, compliance lock-in (once internal compliance requirements have been met they will need to stay met), and skills lock-in. The latter often translates into an over-reliance on single individuals. This is typically presented as a risk for the organisation but it is also undesirable from the point of the individual because it hampers role mobility.
- Commitment: Pre-disposition to a particular outcome or application. Committing to ‘more accurate forecasts which will reduce inventory costs’ or to ‘better customer targeting which will increase retention’ is unwise. Business Analytics will probably be able to provide forecasting and customer insights, but it can’t guarantee contingent operational outcomes.
- Capital: Both sunk financial costs and, more importantly, the political capital expended. A sponsor who gets behind a particular Business Analytics initiative in a high profile way (as is usually required if its costs are significant) becomes politically exposed. The sponsor’s ‘political capital’ becomes tethered to the perceived success or failure of that initiative. The more complex and committed that initiative, the higher the risk of perceived or actual failure.
Analyst First argues that each of these barriers can be either avoided altogether or greatly reduced and made manageable. Software cost is best managed through exhausting the capabilities of commodity and open source tools. These also mitigate against complexity (they may be already familiar), compliance (they are in many cases already accepted and available), and capture (many of them are ubiquitous, open, or both). Capital and commitment prejudice against learning as an outcome. Both of these can be better managed through effective upfront framing. The inherently probabilistic nature of analytics results and their deep dependence on data (and not on fiat, will, assumption or authority) needs to be built in to the way a Business Analytics function manages and presents itself.
Further to this, starting with operational analytics is, perhaps counter-intuitively, harder than going with strategic analytics first. Strategic analytics is about searching for insights as opposed to implementing systems design to deliver gains at existing operational margins (operational analytics). If the primary job of a Business Analytics function is strategic – to enable the data-driven exploration and discovery of insights and intelligence – then the operational implementation of particular results becomes at most a secondary consideration. There is a more complex value chain involved in operational analytics – incorporating systemisation, automation, process re-engineering and change management – and therefore much more which can go wrong. The considerable challenges of operational implementation, many of which are not within the control of a Business Analytics function, need not and should not represent single points of actual or perceived failure for that function.
Follow Us!
