I’ve argued before that most businesses continue to struggle with query and reporting, and that the increasingly common invocation of big data and social media in business cases for Business Analytics is a way of leveraging hype in order to fund second and third chances at getting these basics right. Beth Stackpole, writing at SearchBusinessAnalytics.com, quotes Brian Hopkins of Forrester on a related aspect of this phenomenon:
“Over the last 20 to 25 years, companies have been focused on leveraging maybe up to 5% of the information available to them,” said Brian Hopkins, a principal analyst at Forrester Research Inc. in Cambridge, Mass. “Everything we didn’t know what to do with hit the floor and fell through the cracks. In order to compete well, companies are looking to dip into the rest of the 95% of the data swimming around them that can make them better than anyone else.”
Stackpole’s thrust is that big data presents fundamental technical challenges to established data management practices:
This whole notion of extreme data management has put a strain on traditional data warehouse and BI systems, which are not well-suited to handle the massive volume and velocity requirements of so-called big data applications, both economically and in terms of performance.
Sources of big data identified in the piece include:
[T]he constant stream of chatter on social media venues like Facebook and Twitter, daily Web log activity, Global Positioning System location data and machine-generated data produced by barcode readers, radio frequency identification scans and sensors.
Social media, the Web, and GPS are data sources we’re conscious of interacting with everyday. It’s transparent to us as users of social media sites and smartphones that we’re consuming and generating digital information. The social processes which generate big data are more obvious and more novel to us than the business processes which generate ‘small data’.
Part of the significance of social media from a Business Analytics perspective, then, is that it has made business consumers of information far more aware of how much data they’re not able to get their hands on. The 95% that Hopkins characterises as having fallen through the cracks in the past did so less visibly. In the age of the smartphone, however, it’s much clearer how little data makes it into warehouses relative to what’s out there.
Seen in this way, big data is a two-edged sword for BI managers. It increases awareness and demand for information, and can breath new life into data management initiatives, but it also increases the dissatisfaction of business consumers because they’re now more conscious, and more frequently reminded, of what they’re missing.
Related Analyst First posts:
About usAnalyst First is a new approach to analytics, where tools take a far less important place than the people who perform, manage, request and envision analytics, while analytics is seen as a non-repetitive, exploratory and creative process where the outcome is not known at the start, and only a fraction of efforts are expected to result in success. This is in contrast with a common perception of analytics as IT and process.
Tags in a CloudAIPIO analyst first Analyst First Chapters analytics analytics is not IT arms race environments big data business analytics business intelligence cargo cults collective forecasting commodity and open source tools complexity data decision automation decision support educated buyer EMC-greenplum forecasting HBR holy trinity human infrastructure incentives intelligence model of analytics investing in data lean startup literacy management culture MBAnalytics operational analytics organisational-political considerations Philip Russom Philip Tetlock prediction markets presales R Robin Hanson Strategic Analytics tacit data TDWI Tom Davenport uncertainty uneducated buyer vendors why analyst first