Currently viewing the tag: "why analyst first"
Let’s see a brand name magic black box fix these.
Meanwhile, also from Rexer:
R continued its rise this year and is now being used by close to half
of all data miners (47%). R users report preferring it for being free, open
source, and having a wide variety of algorithms. Many people also cited R’s
flexibility and the strength of the user community
.
Continuing with the big data meets big hype theme:
So you want to get into Business Analytics/Big Data/Predictive Analytics.
What areas, skills, tools, data should you focus on first ?
There are three rather big questions that you need to ask yourself:
1. How well do I really understand the problem(s) that I want Analytics to solve, and The roles(s) that Analytics would play ?
2. How well do I understand my data?
3. What data do I actually have, or can get ?
Each question explores a continuum. Together they represent a three dimensional space of possibilities. There is no “magic quadrant” here, each part of the space is a legitimate place to be, with its own solutions, risks and benefits.
Let’s go through them.
1. The range of possibilities looks something like this:
A: having built preliminary offline random forest models and created some prototypes, I want to extend these existing customer acquisition and retention models we have to our intentional markets, and operationalise them for real-time, event based activity, provided this is seem to yield further significant yield. We will need an industrial strength, scalable, and reliable tool, probably a commercial vendor tool, and possibly a Hadoop-based MapReduce solution
B. my CEO just attended a lavish conference where he saw a slide presentation mentioning the Davenport HBR article from 2006 and now he wants us to “get into analytics”.
Most people are somewhere in between. But you get the idea. And there are far too many initiatives that are precisely at B. the ideal vendor customer is precisely at A. Unfortunately, there are not enough A’s around (we call them “Eduacated Buyers”) so some vendors must sell to people who look more like B’s.
Naturally, Analyst First does not advise Bs to get into Big Data, buy expensive vendor tools, or ever believe anyone that there is such a thing as “a solution for getting started in Analytics” especially when said solution is no more than a bunch of software and maybe a few relatively junior technical consultants for a few months.
Indeed, we advise the Bs of this world to invest in learning, exploring and gaining experience, while managing their sponsors’ expectations and growing their personal investment and participation in the new Analytics enterprise (yep, it’s an Enterprise, with all the Lean Startup that entails), and eliciting from said sponsors their real, and realistically achievable needs.
This is a crucial time to invest in smarts, experience, talent, learning and plenty of Lean Startup.
If this approach is not feasible, I do not have high hopes for the future of the function, which will, at best become a showpiece trophy of high tech adding no value, and will more likely be shut down, “restructured” and restarted again, hopefully with a more sensible approach.
And what of the As ?
Speaking to an A recently, indeed one of the best As I know, he noted that his team had kicked some great business goals recently, having implemented a very necessary expensive vendor tool, after trying R and seeing that it was not up to the big data / big crunch job they had to do. He noted that this was necessary, even though he agreed with A1, and that this was not in line with A1′s preference for open source tools.
“not at all”, I replied, “This is exactly A1, you were the quintessential Educated Buyer! A1 is not against vendor tools. We are against people spending money on what they do not understand in the hope of a magic solution. You don’t fall into that category.”
Hopefully, the anonymous A in question will write a more detailed post on this blog, outlining his success story in more detail.
So, our advice to As is… You don’t really need our advice, until you want to do something new again. In which case, chances are you are following A1 principles already, explicitly or not – otherwise how did you get to A in the first place,anyway ?
Most people are somewhere in between, and usually closer to B than to A.
Answering the “what the heck are we going to do?” question involves exploration on a number of axes, including stakeholders needs, own capability, available resources (human and electronic), any impediments or constraints (Hello IT!) and data, the subject of questions 2 and 3. The actual hidden contents of the data, the “gold” of the data “mining” metaphor is a huge exploratory subject in its own right, and must be considered in the context of the others.
This is not a very easy target to hit, and needs defining before that can happen !
So, to all the Bs and almost-B’s out there : invest in learning : invest in your own and your sponsors’. Invest in getting your sponsor invested, supporting and covering you, letting you explore and grow. Invest, above all, in exploration and invest in managing expectations and delivering intermediate ressults to allow all this to happen. Buy your analytics function a chance to grow, learn, explore and breathe free of unreasonable pressures and constraints.
The other two questions will be covered in upcoming posts.
Pattern-driven Performance: Should You Start with Tools—or with Talent? That’s one of the questions addressed by Deloitte at Real Analytics:
Companies everywhere are catching onto the wisdom of mining information for patterns of performance. Using a combination of advanced statistical tools and good, old-fashioned experience, they’re discovering and dissecting hidden patterns that can help guide their choices in operations, talent, technology, financial strategy, you name it. For those looking to drive performance in this way, there are two paths forward: start by investing in tools or start by investing in talent.
In Analyst First terms it’s a contest between human and electronic infrastructures. Deloitte frames it as a debate, presenting a set of rhetorical, stylised point / counterpoints to which a panel of its Directors and Principals respond. The case for tools first cites scale, automation, efficiency, transparency, and some degree of insulation from undesirable human subjectivity. It also talks up the scarcity of good people and talks down the difficulty of analytics. The case for talent first argues for the importance of business knowledge, an appreciation of context and nuance, interpretative skill, big picture understanding, the ability to ask the right questions, and the soft skills required to build cross-functional communication, coordination, trust and support networks.
Three out of the four Deloitte contributors prioritise talent over tools; the fourth elects both. As Janet Foutty, National Managing Director, Technology, Deloitte Consulting LLP puts it:
[T]here’s a big problem with the “buy technology first” approach: What if you’re not asking the right questions. I know it might sound strange coming from a person who leads Deloitte’s IT services, but I’m “talent first” all the way.
One of Analyst First’s key principles is our advocation of investing in the human over the electronic infrastructure. Simply recommending “both” is appealing, but the reality is that investment decisions are always taken at some margin at which a trade-off is being made, so “both” is never a real choice. A decision to spend any amount of money on commercial software is always a decision to not spend that money on alternative uses—such as hiring more or superior analysts. In comparing the marginal utility of commercial tools with alternative investments, the following should be added to the case put by the Deloitte panel, which further strengthen it:
- Although some instantiations of analytics are process-based, analytics itself is not a process.
- Most of the analytical tools any organisation will require, especially at the outset of its exploration of analytics, are readily available. Much can be done—and is being done—with the commodity tools already available on analysts’ desktops (e.g. Excel and SQL) and with open source tools such as RapidMiner and R.
- Many different tools exist—commercial, commodity, and open source—and sensibly choosing between them means becoming an educated buyer. This entails leveraging experimentation and experience—either in-house in the form of trial and error, or that of outside help.
- Prominent expenditure on commerical tools—while it may perversely benefit individuals—erects a number of barriers to organisational success.
- Tools without sufficient expertise are not harmless. In fact, they may act as risk multipliers.
This does not mean that the marginal utility of commercial software is always less than the marginal utility of analysts. This is of course possible. However, it is empirically the case far less than outsiders to Business Analytics—and many insiders—intuitively expect.
Related Analyst First posts:
I presented on Business Analytics and Analyst First to 80+ people at the Corruption Prevention Network‘s 2011 Annual Full Day Forum in Sydney yesterday. The first half of the presentation covered the role of advanced analytic methods in audit and investigation contexts, illustrating with some anonymised examples of link analysis, digital analysis, clustering and predictive modelling. The second half addressed ‘how to get started’ with analytics the Analyst First way. The slides are available on our Resources page.
UPDATE: All papers from the forum (including previous years) are available here.
This year will see the cinema release of Moneyball, the story of the Oakland A’s baseball team’s successful use of analytics under general manager Billy Beane, based on Michael Lewis’ 2003 book of the same name:
Several themes Lewis explored in the book include: insiders vs. outsiders (established traditionalists vs. upstart proponents of Sabermetrics), the democratization of information causing a flattening of hierarchies, and the ruthless drive for efficiency that capitalism demands. The book also touches on Oakland’s underlying economic need to stay ahead of the curve; as other teams begin mirroring Beane’s strategies to evaluate offensive talent, diminishing the Athletics’ advantage, Oakland begins looking for other undervalued baseball skills such as defensive capabilities.
Professional sporting leagues typify arms race environments. It’s in this context that Daryl Morey, General Manager of the Houston Rockets basketball team and another sports analytics proponent, argues that ‘Success Comes From Better Data, Not Better Analysis’ at HBR:
I see a world teeming with really good analysts. Fresh analytical faces are minted each year and sports teams are hiring them in larger numbers. If talented analysts are becoming plentiful, however, then it follows that analysts cannot be the key to creating a consistent winner, as a sustainable competitive edge requires that you have something valuable AND irreplaceable… The answer is better data… Raw numbers, not the people and programs that attempt to make sense of them.
Data vs Analysts is a false dichotomy. Morey invokes it presumably to provoke. His description of Google makes this clearer:
Smart companies such as Google believe they need savants to crunch those numbers and find the connections that regular humans could not. But my experience, and what I’m hearing from more organizations (sports and non), shows that real advantage comes from unique data that no one else has.
Google’s belief in good analysts is not in dispute. It’s Chief Economist, Hal Varian, argues that competing organisations ideally want a ‘monopoly on left shoes when right shoes are free. That today, data is ubiquitous and cheap and analysis is the complementary scarce factor.’ But Google, of course, has masses of proprietary data that no one else has. I can’t get its dataset on me, for example. It doesn’t help its cause to talk this up in public given its dependence on its users’ trust and concerns about privacy. Doubtless Morey’s downplaying of analysts also serves PR and competitive ends.
It’s obvious that an analyst can’t do much without data, just as data alone is useless without an analyst. It’s also obvious that, given the same data, the better analyst will produce the competitive advantage. In an arms race environment, any such advantage will eventually become operationalised as common or ‘best practice’, and the search for a new competitive edge will shift focus. Competitive advantage comes from sustained uniqueness. It’s agnostic with respect to source.
Nor are analysts passive with respect to data. One of the key functions of an analyst is to determine what new data would be valuable, and to derive, source, or find other ways to create it. Morey:
For obvious reasons, I cannot reveal what data the Houston Rockets track but to track the significant data we gather we use a very large set of temporary labor that helps us develop these data sets that we hope will create an advantage over time.
It was no doubt the Rockets’ analysts who specified that new data, and it will be them, in the coming seasons, validating and testing it to assess its value. The analyst is always first.
Related Analyst First posts:
I’ve written before about vendor worldviews and their evolution as the competitive landscape changes. One of the interesting characteristics of vendor worldviews is their bias towards symmetric competition. Each vendor focuses most of its competitive attention on its nearest neighbours: those most closely matching its business model and product and service offerings. When I was working for a Comshare distributor a decade ago we worried most about Hyperion. When I was working for Cognos a few years later we worried most about Business Objects. In each case we accepted the paradigm we were placed in – by Gartner, for example – and focused our competitive energies on the minority of features which distinguished us from other occupants of our Magic Quadrant.
It’s easiest, and perhaps most comforting, to understand your competitors in terms of yourself. However, your most challenging competition is asymmetric. It typically comes from outside, it’s often unexpected, and it usually changes your paradigm. Australian newspapers fifteen years ago competed symmetrically with other newspapers for a slice of national, metropolitan or regional market share. Nowadays, as a result of the Internet, they must also compete asymmetrically: with global newspaper brands like The New York Times, and with alternative content generators (blogs, social media, Youtube) and delivery mechanisms (computers, smartphones, tablets).
Business Analytics today is a truly asymmetric marketplace. Megavendors compete with pure-plays. Commercial vendors compete with open source. Software competes with services. Inhouse functions compete with outside providers. The electronic infrastructure competes with the human infrastructure. Strategic focus competes with operational. Top-down competes with bottom-up. Bespoke competes with automated. The IT model competes with the intelligence model. The project-based approach competes with Lean Startup.
The Analyst First worldview recognises that each of these dimensions is its own continuum, that each matters, and that their interactions have substantive implications in terms of likelihood of success, cost and benefit trade-offs, and risk profile.
Related Analyst First posts:
- Same same but different
- Vendor worldviews
- Vendor worldviews evolve
- Measuring the Business Analytics software market
- Strategic First
- Solution buying
- Against best practices in Business Analytics
- Analytics is… Intelligence – The Podcast
- Forrester on the need for agility
- Analytics Is… A Lean Startup Enterprise
Tried and true best practices for enterprise software development and support just don’t work for business intelligence (BI). Earlier-generation BI support centers — organized along the same lines as support centers for all other enterprise software — fall short when it comes to taking BI’s peculiarities into account. These unique BI requirements include less reliance on the traditional software development life cycle (SDLC) and project planning and more emphasis on reacting to the constant change of business requirements.
That is from Forrester Research’s Agile Business Intelligence Solution Centers Are More Than Just Competency Centers report, just released. The full version of the report is paid (USD 499) but a free overview from its two lead authors, Boris Evelson and Rob Karel, is here. The case against the SDLC / project approach is summarised thus:
Earlier-generation BI support organizations are less than effective because they often:
- Put IT in charge
- Remain IT-centric
- Continue to be mostly project-based
- Focus too much on functional reporting capabilities but ignore the data
In response, Forrester advocates a a ‘flexible and agile’ approach to BI, and establishing “BI on BI” to explicitly learn from successes and failures.
This echoes much of what Analyst First advocates, namely that:
- Analytics is not IT
- IT risk management practices hamper Business Analytics initiatives
- It is prudent to assume bad data
- A Lean Startup approach makes more sense
- A good deal of Business Analytics is bespoke
Note that Forrester is making these recommendations at the Business Intelligence end of the Business Analytics spectrum. It’s arguing that, even where Business Analytics lives in an operational, repeatable, systematised, automated, decision automation context:
[No] repository can fully substitute for personal, qualitative knowledge; that’s often more art than science. Therefore, staff the BICC/COE [Business Intelligence Competency Centre / Centre Of Excellence] with individuals whose primary responsibility is to disseminate such knowledge above and beyond what’s available in the repository.
In other words, the human infrastructure is critical, and investments in electronic infrastructure which ignore it will be unsuccessful.
Related Analyst First posts:
At yesterday’s A1 meeting in Sydney we spent a lot of time debating the difference between the A1 view of the world, and the default, or vendor-driven view.
Putting aside questions of fault or intentionality, the question to be asked is: what is the simplest statement of the default view of Business Analytics, and how is the A1 position different, and more so how are the two views in opposition?
To me, the difference is as follows:
The Typical Vendor View
The (usually implicit) vendor promise is:
“Our software will make the typical knowledge worker much more productive”.
An even more implicit form of the promise is:
“Our software will make the typical knowledge worker more productive than commodity and open source tools would”.
The expectation is that knowledge workers are more or less the same given a specific job function, and that the right Analytics tools unlock tremendous produtivity in those workers.
Needless to say, the A1 view is somewhat different.
The Analyst First View
The A1 view is that while some software is required, there is actually minimal difference in productivity between different tools. While some tools may in fact be more extensive or user-friendly than others, Analytics is still largely a manual, expert task for highly skilled and gifted professionals.
Almost all difference in productivity is due to the qualitity of the analysts, and this is where spending should be concentrated, especially in the early stages. We thus focus spending on the human infrastructure, and explore the many commodity, open source and free tool options available.
Most high-end vendor tools claim to “add value” by providing user-friendly interfaces, and automating much of the statistical and computational operation of the tool, along “best practice” lines.
The A1 view is that this is if anything a risk multiplier. A mediocre analyst who does not have a strong grasp of what they a doing is much more likely to obtain what looks like a “result” given such a powerful tool. The result may however be dangerously incorrect.
A useful analogy here would be to that of a user-friendly plane, one that makes take-off relatively easy for the untrained pilot. Of course, a poorly trained pilot and an already flying plane may well be a recipe for disaster. As is a pretty-looking report supporting key decisions and actually containing garbage, generated by clever software run by poorly skilled staff. High end tools can hide catastrophic incompetence.
User-friendly tools in the hands of well-trained professionals are another matter. Yes, the best possible combination is indeed a powerful tool in the hands of someone who can drive it effectively. Once again, it pays to point out that the value add of the expertise is far more critical than that of the tool. Conversely, the risk due to lack of expertise is exacerbated by seemingly friendly tools.
So, in a nutshell:
Some vendors believe or imply – “our tools make everybody more productive”.
A1 says – “tools make far less positive difference than people. Tools can make a massive, negative difference by hiding incompetence. Focus on people, not tools”.
Good human infrastructure plus commodity and open source tools is a winning combination.
Corollary – Good people are critical.
So what is Analyst First all about?
In a nutshell, it is about making analytics cheaper, more relevant and appropriate to business (which can includes government, NGOs and any other folks actually using analytics to do something other than research for its own sake). It is also about presenting a radically different model of analytics to the one currently seen by most of the market.
Does this mean that it is not being done well already? Well… Let’s say that it could be done a whole lot better.
The biggest problem is: most people think that analytics is about software, when it is actually about people.
What does this mean? It means that buying very expensive software that the buyers do not understand and do not have the staff to select appropriately – let alone use – is a lousy way to get going with analytics.
On the other hand, investing in people might just be the right idea. Investing in people does mean getting skilled analysts before software. Hence “Analyst First”.
But this is only the beginning, getting us to the first key principle of Analyst First:
Invest in Smarts: Build The Human Infrastructure First
This means getting highly skilled experts in analytics to advise, demonstrate and trial a range of techniques, mentoring the new team.
It means carefully building an appropriate team of analysts, business experts, communicators and data manipulators (yes, they are different skill sets).
More importantly it means establishing the right channels, expectations and incentives to gently educate executives about what they can ask of analytics, and what they need to provide to make it happen.
This may sound hard enough, but what does the team work with if there is no software? But there is:
Use Free, Commodity and Open Source Tools First
Tools on the desktop, such as MS Excel and Access, are more than enough for most analytics tasks attempted by beginners, or business areas trying out analytics.
If serious power is required, tools like R, Rapidminer, Knime etc will probably do. These tools are free, and industrial-strength enough for most applications. Certainly worth trying first, and perhaps sticking with.
Here is some evidence that this is catching on.
In our experience, commodity and open source tools are good enough 95% of the time – 100% percent of the time for a new analytics unit. In the latter case, the unit is not entirely sure how analytics may be applied in their business, and their first job is to find out, capturing executive support in the process.
This is a big ask, made all the bigger if big $$$ have been spent on software, and a small number of mediocre staff are hired as an afterthought.
On the other hand, commodity and open source tools are a great alternative, allowing the money to be spent on human infrastructure.
Analyst First is not against buying expensive vendor tools, but it is against spending a cent on software until the buyer is an Educated Buyer, having used commodity and open source tools extensively, found their limitations, and seen a specific need for an expensive vendor solution.
Educated Buyers cut their teeth on readily available, inexpensive tools first, and invest their money in people: staff, skills, consultants, mentoring.
The Practice of Analytics: Exploration, Learning, and Making Mistakes
Analytics is not a linear process, like most engineering projects. Its end product is discovery: you cannot determine what will be discovered ahead of time. Thus the outcomes of analytics, and the decisions based on them, cannot be made before the analysis has been carried out.
Further, analytics is inherently exploratory in nature: data is a treacherous beast, and there may be many dead ends. Not all analytics exercises end in brilliant findings, accurate models or actionable insights. Nor should they. Mistakes are how we learn. The trick is to make them quickly, minimise their cost, learn from them and move on.
This organic, exploratory approach fits perfectly with a view that Analytics is an Intelligence activity.
Where it fits less well is with the view that analytics is IT…
Analytics is not IT
While accounting, graphic design, journalism and medicine are not part of the IT function, they all use a heck of a lot of IT.
Analytics done right is no different. While analysts require some very smart software to do their jobs, the software is not the star of the show. A strategic intelligence function does not belong within IT.
The fact that analytics relies on highly sophisticated software should not serve to downplay the far greater role of people.
The exploratory, organic growth, human centric model outlined above is in stark contrast with the IT view of the world, which is all about pre-defined systems, known outcomes and large, top-down specified projects. An intelligence-based, human-centric model has no chance to flourish within such an environment.
Finally, most of the issues concerning IT in the context of analytics apply in the operational deployment of analytics results (eg campaign models), but apply far less in the context of analytics proper: the exploration, description and modelling of data.
Issues around security, real-time effectiveness, transfer bottlenecks etc are not real issues in most analytics contexts, though they are thrown around by IT departments.
More later.
Meanwhile, here is a video of the first A1 presentation, with slides.
Follow Us!
