Currently viewing the tag: "OLAP"
We’re building new information delivery systems for a future that isn’t there. Our state-of-the-art environments are already becoming obsolete because our view is distorted by the lens of the past, showing us the future as it was years ago. That world of scarce computing resources and limited data is gone.
That’s Mark Madsen at TDWI, arguing that many of the key assumptions driving our construction of analytic systems—decision support systems, data warehouses, and business intelligence—are wrong. The first wrong assumption is of scarcity. Processor cycles, memory, and storage used to be expensive. They aren’t any longer, but we’re still batch processing our ETL, prematurely archiving, summarising and normalising our data, and limiting our storage of derived information.
His second target is the tabula rasa impulse:
Most data warehouse and BI methodologies assume that you start with no analysis systems in place. The methodologies were created at a time when information delivery meant reports from OLTP applications.
The reality today is that analytics projects don’t start with a clean slate. Reporting and BI applications are common in different parts of the organization.
Third is the assumption of stability. Build-from-scratch methodologies made sense the first time around, but:
By not focusing on evolution, the methodologies miss a key element about analytics: they often focus on decisions that change business processes. Process change means the business works differently and new data will be needed. When someone solves a problem, they move on to a new problem. The work is never done because an organization is constantly adapting to changing market conditions.
One of Analyst First’s key principles is that:
Analytics is not a linear process, like most engineering projects. Its end product is discovery: you cannot determine what will be discovered ahead of time. Thus the outcomes of analytics, and the decisions based on them, cannot be made before the analysis has been carried out.
Consequently we advocate the primacy and ongoing centrality of strategic analytics over operational analytics. The more analytics is conceived of as a set of activities which only adds value at existing operational margins, the more it is unnecessarily constrained and the less it is able to change the game. Echoing yesterday’s Analyst First post on analysis as a read-write activity, Madsen continues that:
Business intelligence methods and architecture assume that what’s being built is a single system to meet all data needs. We still think of analytics as giving reports to users. This ignores what they really want: information in the context of their work process and in support of their goals. Sometimes reports are sufficient; sometimes more is needed.
He goes on to confirm that big data is both challenging status quo electronic infrastructures and driving demand for higher value advanced analytics:
The interaction model for BI delivery is that a user asks a question and gets an answer. This only works if they know what they are looking for. Higher data volumes, more sophisticated business needs, and high-performance platforms require that BI be extended to include advanced analytics. These answer “why” questions that can’t be answered by the simple sums and sorts of BI.
As I’ve argued before, the assumption-heaviness and manual intensiveness of standard BI technologies such as OLAP can’t compete, at scale, with the automated exploration that machine learning methods make possible. Madsen concludes that the data warehouse should be conceived of as a platform rather than an application. His closing four paragraphs are worth quoting in full:
The data warehouse has evolved to the point where it needs to provide data infrastructure, and needs to support information delivery by other applications rather than trying to do both. Data infrastructure requires a focus on longer planning horizons, stability where it matters, and standardized services. Information delivery requires meeting specific needs and use cases.
Design methods today seldom address the need to separate data infrastructure from delivery applications. Designs focus on data management and fitting the database to the delivery tools. This leads to IT efforts to standardize on one set of user tools for everything, much like Henry Ford tried to limit the color of his cars to black.
The new needs and analysis concepts go against the idea that a data warehouse is a read-only repository with one point of entry. They do not fit with established ideas, tools, and methodologies.
Today, the tight coupling of data, models and tools via a single SQL-based access layer prevent us from delivering what both business users and application developers need. The data warehouse must be split into data management infrastructure that can meet high-performance storage, processing, and retrieval needs, and an application layer that is decoupled from this infrastructure. This separation of storage and retrieval from delivery and use is a key concept required by data warehouse architectures as business and technology move forward.
Related Analyst First posts:
The previous post noted that vendor worldviews change and can do so with more speed than product and service offerings. In doing so it made the distinction between worldview and product. Products solve problems. Problem solving requires problem diagnosis, which requires a worldview. Products require disciplined and painstaking development, which takes time. Worldviews, on the other hand, can be far more adaptive.
I’ll illustrate this through a quick look back at Cognos’ evolving worldview over the past five years. By way of personal disclosure I worked at Cognos from mid 2003 to the end of 2005.
In late 2006 Cognos remained positioned as a leading business intelligence provider. As this December 2006 press release illustrates the term “analytics” was starting to attract mainstream attention (unsurprisingly: 2006 was the year of Tom Davenport’s seminal Competing on Analytics HBR paper). Clearly, however, Cognos is still presenting itself as squarely in the business intelligence camp.
Cognos announced its acquisition of Applix a year later, at which point its positioning began to shift. Applix’s flagship product, TM1, begins to be presented as an enhancement of Cognos’ “analytics capabilities”. See, for example, this press release from December 2007. TM1′s strength and reputation, then as now, was in planning, budgeting and reporting. Cognos already had Cognos Planning for planning and budgeting and Cognos 8 for reporting. To this point Cognos had used “analysis” as a synonym for OLAP (also a Cognos 8 capability, previously PowerPlay). The term “analytics” is new and becomes duly associated with the new acquisition, TM1. Note that the functional capabilities have not changed substantively – the acquisition has doubled them up – but the worldview has expanded in response to the increasing buzz around “analytics”.
The “analytics” talk continued following IBM’s acquisition of Cognos another year later. This IBM press release from October 2008 associates the Cognos product stack, now including TM1, with both “analytics” and “advanced analytics”. It also associates enterprise search technology with “text analytics”.
IBM’s subsequent purchase of SPSS – in mid 2009 – incorporated a set of products more conventionally recognised as “analytics”. The acquisition press release positions these as “advanced analytic” and “predictive analytics” capabilities.
There are two trajectories here: language and product. The change in language (the integration of various flavours of “analytics” into the worldview) occurs well ahead of any change in product capability (in this case via the TM1, Cognos, and SPSS acquisitions rather than through in-house development). Being largely a linguistic phenomenon, positioning is inherently more flexible than product.
The lesson here for organisations investing in Business Analytics is that the languages through which vendors communicate their worldviews are more like different dialects than a common language. Not only do you have to understand precisely what “analytics” means for your organisation, you also have to understand whether it means, say, OLAP or Predictive Modelling for each of your prospective service providers.
Follow Us!
