So far from making us more profligate with information, perhaps the Goddess of ‘big data’ will spur us to be smarter in data selection, and ensure more intelligence is embedded within our data extraction, transformation and reporting processes.
Greg Taylor‘s comments on the ‘Knowing what you’re missing‘ post are spot on. One of the clear implications of the big data explosion—technical challenges aside—is that manual analysis methods simply can’t scale to the volume and velocity at which potentially relevant data is being generated. As such, analytics (particularly of the machine learning variety) is ever more vital. One of my rules of thumb in consulting is that any OLAP cube is a standing business case for predictive modelling. As I put it in the ‘Advanced analytics and OLAP‘ post:
OLAP makes multidimensional data exploration about as fast and intuitive as it can be when a human is doing the driving. This means being able to arrange on screen, in two dimensions (perhaps taking advantage of colour and shape to visualise a third and fourth), relatively small subsets and arithmetical summaries of data. Advanced analytics, however, automates exploration. Only data mining methods can look at all dimensions simultaneously, at all levels, in combination. And they can do this in unsupervised (looking for natural structure in the data) or supervised (inferring input-outcome relationships) modes.
Greg elaborates:
- So as Analysts search more broadly for relevant data to meet the decision making requirements of management, perhaps they need to increasingly ask themselves: how will this piece of information fit within the network of predictive functions which explains the business?
- How might Analysts apply Occam’s razor to ensure only information which contributes predictive understanding is included, given the exponential growth in the potential data sources that could be used? One logical approach is for Analysts to undertake more experimental testing of variables (and transformations) for their explanatory power with respect to business outcomes.
As the earlier post reported, status quo electronic infrastructures aren’t ready for big data, and new technologies and disciplines are evolving rapidly to close the gap. But even more substantive changes are required of organisations’ human infrastructures. The key transition that business users of data need to make in the big data context is from the default of consuming more data to the practice of consuming data of higher value. This means becoming analytically literate and learning how to trust and leverage analysts. The key role that analysts must play in supporting decision makers is to understand what constitutes higher value, and to seek it out and communicate it. The key role for IT functions and BI managers is to enable analysts to enable decision makers.
Related Analyst First posts:
If you enjoyed this article, please consider sharing it!
Follow Us!
