How a Business Intelligence Center Can Help With Fast Data

business intelligence center

A business intelligence center of excellence can help with various real-time data challenges, including business value, data management, and governance.

How do you resolve the split between business and IT departments when dealing with data?

If you come from the world of traditional business intelligence (BI) and analytics, your first thought may be of the BI Competency Center. Or let’s call it by its more expansive name “Business Intelligence Center of Excellence” — BI CoE — because of its key role as the bridge between business and IT in the realm of data analytics.

The BI CoE has been around a long time. A 2003 Gartner report describes its role as “to champion the BI technologies and define standards, as well as the business-alignment, project prioritization, management and skills issues associated with significant BI projects”. This description, like all others I know, stresses that implementing BI is an ongoing process, managed as a program of projects. Undoubtedly, this basic premise and the resulting roles and activities apply equally to the BI CoE when fast data is prevalent. But, what are the new considerations?

Internal Static Data vs. External Fast Data

fast dataTraditional BI obtains the vast majority of its data from internal sources, which are typically well defined and understood, relatively clean, and slowly changing in structure and content. Real-time data comes primarily from external sources, the characteristics of which are diametrically opposite to those just listed. With external, fast data, the BI CoE must monitor and, to some extent, control the acquisition process to ensure quality and consistency, knowledge sharing across departments, and contextual integration. I know of one case, for example, where three different departments separately and individually purchased access to the same data set within months of one another. Perhaps surprisingly, the vendor failed to inform the company of the error of their ways! Clearly, a good BI CoE could have, at the very least, saved this company some real expense. In general, there are significant savings to be made here if the BI CoE focuses on this aspect.

From Data Lakes to Fast Data Pipelines

Second, the BI CoE must take a much stronger role than previously in the promotion of real-time analytics discoveries to production. This need arises because of the deepening chasm that data lakes tend to create in data management. In traditional BI, operational and informational environments, while separate, are still closely linked through shared data, common sourcing design and development, and often overlapping teams of users and/or developers. With real-time data projects, we are dealing with new data sources, separate sourcing approaches, and organizational models which often over-emphasize the independence of data scientists. As a result, the path from experimental sandbox to everyday production (wherein lies the real value) is longer and more arduous than for traditional BI. The BI CoE can be instrumental in driving real revenue from real-time data in this area by building better pipes between the data lakes and the operational systems and getting data and value flowing more rapidly.

Business Intelligence Center: Education and Training

A third and, perhaps, shorter term focus for the business intelligence center is on the education and training of data scientists and other denizens of the data lake. Many of these people come from backgrounds where data governance and management principles carry less weight. With a focus on experimentation, statistics and data wrangling, data scientists may fail to appreciate the longer term, expensive remedial work that may be needed to clean up data lakes into which every imaginable data set has been poured and forgotten. The BI CoE can take on a powerful role in education and, perhaps, enforcement to ensure that the good data management principles—that we learned in BI and data warehousing over 30 years—are applied.


Fast data and the new biz-tech ecosystem

How automated decisions will change BI roles

Want more? Check out our most-read content:

White Paper: How to ‘Future-Proof’ a Streaming Analytics Platform
Research from Gartner: Real-Time Analytics with the Internet of Things
E-Book: How to Move to a Fast Data Architecture
The Value of Bringing Analytics to the Edge
Preventing Downtime With Predictive Analytics
IoT Hacking: Three Ways Data and Devices Are Vulnerable

Liked this article? Share it with your colleagues!

Dr. Barry Devlin

About Dr. Barry Devlin

Dr. Barry Devlin is founder and principal of 9sight Consulting. He is among the foremost authorities on business insight and one of the founders of data warehousing, having published the first architectural paper in 1988. With over 40 years of IT experience, including 20 years with IBM as a Distinguished Engineer, he is a widely respected industry analyst, consultant, speaker and author of the seminal book, “Data Warehouse—from Architecture to Implementation” and numerous White Papers. His 2013 book, “Business unIntelligence—Insight and Innovation beyond Analytics and Big Data '' is available in both hardcopy and e-book formats. In his role at 9sight Consulting, Barry provides strategic consulting and thought leadership to buyers and vendors of BI solutions. He is continuously developing new architectural models for all aspects of decision-making and action-taking support.

Leave a Reply

Your email address will not be published. Required fields are marked *