Technology Spectator recently published an article highlighting the need for big data speed …
This article highlights the communication challenge for accredited A1 professionals.
We all recognise Analytics is about using information better than competitors, so we are: 1. doing things better, and 2. doing better things than competitors/relevant comparators. But like so much of the coverage of our sector, the article focusses solely on Operational Analytics, not the latter area of Strategic Analytics.
Secondly, the article fails to recognise speed is only one part of the equation.
Taking the author’s example of the retail sector, sure real time analytics can detect an early decline in sales for a particular product, controlling for some extraneous factors. But a retailer’s promotional response (who they target and how) doesn’t necessarily require real time analytics (they can apply in real time outputs from models created last week, with little risk of degradation).
The most important questions for shareholders of the retailer require Strategic Analytic capability: how should pricing across the entire product portfolio be optimised?, what products should we be ordering now for next season (or the season after)?, how to optimise the physical network and supply chain? These strategic questions demand the right answer, not necessarily the fastest answer.
Any experienced industry professional gets that making sense of data is our primary role. But clearly interpreting data to the best of our ability flies in the face of throwing away information (e.g. because inconsistencies in the available data makes the task more cognitively complex). No one would advocate storing and processing data which possesses no incremental information value, but information value can be measured, so that shouldn’t be an issue.
Critically this article fails to recognise many of the barriers for Australian companies in effectively using their data relate to data quality, not their data storage and processing capacity.
Finally, there is no explicit recognition of the talent required to use data more effectively than your competitors.
From an A1 perspective we should welcome the growing focus on our sector, but we need to better articulate the more nuanced (and interesting) story of Analytics in an A1 Practice. It would be easy to criticise the journalist for being naive in swallowing the line of vendors and other vested interests, but the responsibility is ours to better explain the reality.
Eugene is totally right that we need to stand with a united voice. From today, NTF with publicly back A1 in all our proposals and marketing collateral. I regret not taking this action sooner.
If you enjoyed this article, please consider sharing it!
2 Responses to Time for a Big Data diet – really?
Leave a Reply
Follow Us!
Some thoughts about data ‘quality’
At a recent Analyst 1st meeting in Canberra, it was suggested that I offer a few thoughts about ‘data quality’ as a thought-piece. Here are some of the thoughts:
There is considerable discussion around ‘data quality’; and so it should be. Unfortunately, ‘quality’ is a qualitative and, at time, emotional, assessment of whether the data we have supports analytics. Some, after a quick look at available data, throw in the towel and abandon any attempts to make the data work.
I am inclined to abandon the word ‘quality’ as it tends to under-value the capabilities of skilled analysts and sophisticated software and leads managers to rash judgments about what is and what is not possible. It is far more constructive to consider whether the given data is able to support our processes with sufficient accuracy to be meaningful. In other words, is there enough value in the data to be fit for service?
The Situation: Data is rarely collected to support analytics
We tend to have data that is collected to support a business process. Even if analytics is considered at the initial design stages, by the time we need to get the data to provide answers, we would generally find that these new questions were probably not anticipated years ago when the project defined the current data structures and processes.
The challenge for analysts is, therefore twofold: firstly the analyst tries to build models that address the issues with the highest degree of accuracy from the available data; secondly the analysts compiles a ‘shopping list’ of data that would enhance the process if it were available. The ‘shopping list’ can be provided to management to ensure it is considered in future redesigns.
There will be times when even the smartest analyst cannot squeeze the answer out of existing data sets or sets outside the organisation that may be available.
While we wait to get more useful data analysts may be able to help the organisation build interim solutions based on what is available (eg intelligence reporting, business intuition and knowledge, etc).
Lessons Learnt
• Avoid the term ‘quality’. It assumes that there is a low or high quality data set. This is not helpful. We should probably focus on how well the data supports the analytics processes. As far as business operations are concerned, if the data processes support the core business functions then there is little wrong with its ‘quality’.
• Start analysis from the ‘data end’ rather than having in mind a business (intuition or anecdotally or experientially – derived) model. While these can be useful, measuring the data against such models and then making a call that the data is ‘not good enough’ in some way is not helpful. Analysts should be given the opportunity to test a range of methods to see what meaning can be drawn from the existing data first – in my experience, analysts have been known to produce pleasant surprises when given the chance and the right tools.
• Ensure that there is a business process in place (or build one) that can feed analysts’ suggestions back into future systems enhancements.
• Many vendors offer an end-to-end solution (everything from data capture to storage to analysis and reporting). Such systems would need to be sufficiently flexible to support changes over time to support changes in data structures, collection processes and tools available to support analytics processes.
• Educate the business areas to create an awareness of the value of data:
o enhanced data sets; and
o business processes that better support analytics (ie convince operational staff or clients entering data that there is value in timeliness, accuracy, completeness, etc).
Hi Klaus,
I am going to promote that one to a blog post.