Traits of Next-Generation BI

Dave Kellogg writes an excellent blog where he covers next-generation database management, search, and content management technologies along with commentary on Silicon Valley, venture capital, and the business of software. Dave's blog is worth reading. This month, he wrote a great post, entitled Traits of Next-Generation BI (Business Intelligence), where he commented on his thoughts of the future of BI. Below are the Dave's Traits of next-generation BI:

In memory, columnar, and compressed. Most solutions rely on the fact that the source data for most problems can now fit in memory, typically using a columnar and compressed format. Some solutions are even able to perform work on the data without first decompressing it.

Fast. The dream of BI — particularly for interactive analysis tools – has always been “speed of thought” analysis. Thanks to the above point and thanks to additional performance optimizations (e.g., to expoit CPU cache locality), this dream is becoming a reality.

Directly connected. Next-generation BI tools generally connect directly to the underlying source databases (and/or the Internet) to capture data. This means they must also have basic data integration capabilities both so they properly align data from different systems and dynamically refresh it.

Schema-free. In order to accomodate semi-structured information and to be able integrate information from different sytems, next-generation BI does not require the up-front definition of a schema. Instead, relationships among data (e.g., hierarchy) are discovered dynamically.

Beautiful. While this is best exemplified by Tableau (where visualization is the principal focus) next-generation BI tools generally provide beautiful visualizations that are more powerful than the basic report and bar chart. (Note that I named a name here because I consider Tableau mid-stage, not early-stage.)

Mobile. Next-generation BI tools typically assume a brower-based client and often the need to create device-specific clients (e.g., a native iPad app) to supplement it. Some companies focus exclusively on mobile BI.

Neutral. Next-generation BI tools exploit the fact that a multi-billion dollar vacuum was created in the market when the BI leaders were consolidated and became units of IBM (e.g., Cognos) or SAP (e.g., BusinessObjects).

About the future of data warehouse, he wrote: if you can fit your entire data set in memory and dynamically calculate the answer to any question at high speed, then why do you need a data warehouse full of precalculated aggregates again? I thus see a “middle squeeze” on the data warehouse market in the future.

- For most applications of normal size and analytic complexity, people will use next-generation BI on top of raw data sources, unless they have very messy data or a need for extensive history.

- For large applications (i.e., big data) and/or high analytic complexity, people will use advanced analytic platforms (e.g, Aster Data). This, of course, begs the question whether anyone is working on BI tools that exploit and optimize the new, high-end analytic engines and the answer to that question is happily “yes” as well.