An Enterprise Framework for Business Intelligence
Colin White BI Research
Ma y 2009
Sponsored by Oracle Corporation
Copyright 2009 BI Research, All Rights Reserved.
T ABLE OF C ONTENTS
AN ENTERPRISE FRAMEWORK FOR BUSINESS INTELLIGENCE 1
THE BIPROCESSING NEEDS OF THE BUSINESS 1
TYPES OF INFORMATION WORKERS 3
BITECHNOLOGY REQUIREMENTS 4
PUTTING THE PIECES TOGETHER:AN ENTERPRISE BIFRAMEWORK 7 AN EVOLUTIONARY APPROACH TO DEPLOYING A BISYSTEM 8
Brand and product names mentioned in this paper may be the trademarks or registered trademarks of their respective owners.
A N E NTERPRISE F RAMEWORK FOR B USINESS I NTELLIGENCE
The business intelligence (BI) and data warehousing industry is changing rapidly as organizations strive to exploit business intelligence to improve business performance, while also at the same time trying to manage increasing data volumes and the need to provide business users with faster and easier access to information.
To support constantly changing BI technologies and information needs, organizations need to design a flexible enterprise BI framework that incorporates the different kinds of BI processing, information workers and BI technologies that organizations must support if they are to fully realize the business benefits that business
intelligence offers. This paper presents a BI framework that can be used to develop and deploy a new BI system, or to enhance an existing one. This flexible and adaptable framework enables BI applications and the underlying data warehousing environment to evolve and adapt to the constantly changing technology and product landscape.
The paper includes the results of a survey that asked BeyeNETWORK site visitors about the different approaches and technologies their organizations use to plan, build and grow their BI systems to exploit the business value business intelligence and data warehousing offers. There were 217 responses to this survey.
T HE BI P ROCESSING N EEDS OF THE B USINESS
Three key elements need to be considered when designing and developing a BI system: BI processing needs, the information worker audience and the technologies that need to be provided. This first section discusses the BI processing needs of organizations.
There are three main modes of BI processing: strategic BI, tactical BI and operational BI. One of the key distinguishing features between these three processing modes is the time period over which each of them analyzes and reports on information. This time period can best be explained in terms of the workflow shown in Figure 1, which illustrates how business intelligence aids decision making.
In Figure 1, data flows from operational applications that run day-to-day business processes to BI applications that monitor and analyze the data to provide insight about the actual business performance of those processes. Business users then employ collaborative applications to share and evaluate the results produced by the BI applications. This evaluation may result in users making changes to a business process (e.g., a modified marketing campaign) or a business plan (e.g., an updated sales forecast). The positive or negative effects of those changes can then be measured by the BI applications to create a closed-loop decision workflow.
Organizations need to build BI systems that can handle technology and product changes
Closed-loop processing enables business users to see the effects of business decisions
Copyright 2009 BI Research, All Rights Reserved. 2 The speed at which business users need to do this closed-loop processing determines over what period of time data is analyzed and how responsive the BI environment needs to be. This speed can be expressed in terms of the elapsed or action time between a business event occurring that requires action and the user taking that action. In the case of fraud detection, for example, the required action time maybe a matter of seconds, whereas in a marketing campaign, the action time could be a few hours, days or weeks.
Figure 2 outlines the action times supported by the three modes of BI processing.
Other variables, such as the types of users supported and the types of data required, are also shown in the figure. Figure 2 is intended to illustrate general differences, rather than absolute boundaries, between the three modes of processing.
Strategic and tactical BI processing represent the types of solutions that organizations have been developing for many years. In this environment, data flowing from
operational processes is captured into a data warehouse for use by BI analytical applications. Users run these analytical applications at periodic intervals to collect data analytics for the specific data points or key performance indicators (KPIs) they are interested in – sales in a particular region, for a specific product, during a specific time period would be an example. One industry trend here is to supplement these data analytics with content analytics created by analyzing unstructured data.
Operational BI processing focuses on the performance of a specific operational business process, rather than on business unit or enterprise-wide KPI data points.
Operational BI is one of the fastest growing areas of business intelligence.
As action time requirements approach near real time, operational BI processing becomes driven by business events occurring in operational systems, rather than by users requesting information at periodic intervals. Examples of events that may trigger operational BI processing include credit card transactions (to detect potential fraud) and customer calls to a call center (to monitor call center call backlog and response times).
Figure 1:
Closed-loop decision workflow
BI agility can vary from near real time to weeks and months depending on business need
Operational BI is the fastest growing BI segment
To process business events and support so-called event analytics, the BI environment must either enable the loading of low-latency event data into a data warehouse or the embedding of BI processing into the operational processes that handle events.
Strategic BI Tactical BI Operational BI Action
time
Weeks to months Days to weeks Intra-day to daily
Types of users
Executives &
business analysts
Business analysts &
line-of-business managers
Line-of-business managers &
operational users Focus of
analysis
Enterprise Line of business Business process
Type of data
Historical Historical Real time, low
latency, historical Mode of
operation
User driven Data centric
User driven Data centric
User & event driven Process centric Business
objective
Achieve long-term business goals
Manage tactical initiatives
Manage business processes
T YPES OF I NFORMATION W ORKERS
Business intelligence specialists, analysts and vendors often use the term information worker to describe a business user that makes use of the information produced by BI applications. There are two main types of information worker: information
consumers and information producers.
Information consumers range from senior and middle managers to call center staff. A common characteristic of these information workers is that they do not use corporate data and IT information-handling tools throughout their working day. These types of users represent the bulk of the information workers in an organization.
Information consumers need to discover information that enables them to do their jobs more efficiently and make more informed business decisions. Information discovery involves locating and retrieving information of interest, and then filtering and organizing it so that it is easy to navigate and explore. Some information consumers, such as business managers, may do some limited analysis of the discovered information, while others, such as call-center staff, just use the information as is.
Information producers create the information used by information consumers. In addition to using information creation and discovery tools, information producers also use analysis techniques to enhance, aggregate and report on information.
Figure 2:
The three modes of BI processing
Two main types of information worker – consumers and producers
Information consumers are a large user audience who need easy to use tools
Copyright 2009 BI Research, All Rights Reserved. 4 Information analysis extends information discovery by enabling information producers to apply their knowledge and expertise to discovered information and enhance its value by putting it into a business context. Enhancing information in this way makes it easier and faster to use by information consumers in their jobs.
Information consumers may do additional analysis of the information, but this is likely to be limited in scope. Examples of information producers include BI specialists, and data and business analysts.
An important consideration is that there are different types of information consumer.
The skills and requirements of a call center representative are different from those of a senior manager (a VP of Sales, for example). The former is a pure information consumer, whereas the latter may also have some information producer skills. It is important therefore to map job roles to both skill and technology requirements.
As organizations deploy business intelligence to a wider audience of information consumer they will need to focus on making the BI system easier to use, and the information produced easier to consume by putting it into a business context.
BI T ECHNOLOGY R EQUIREMENTS
The next task in the design of a BI system concerns the BI technologies that it needs to provide. These technologies can be categorized according to how they support each of the activities of the decision workflow shown in Figure 3. This figure is a more detailed version of the workflow of Figure 1.
The six main activities of a decision workflow are:
• Discover the operational data and other business content that can aid business decision making.
• Access the operational data and business content required to make business decisions.
• Integrate the retrieved operational data and business content into a shared data store, such as a data warehouse, as required.
Information
producers are more knowledgeable about the BI technologies and corporate data
Information needs to be made easier to consume
Figure 3:
Main components of a decision workflow
• Analyze operational data, business content and data warehouse information to produce analytics, alerts and recommendations that aid decision making.
• Deliver the results of analytical processing to business users and applications.
• Share the results with other users and collaborate to determine what actions need to be taken to resolve any business issues identified by the results.
A wide range of technologies supports these six activities. These technologies can be broken down into two broad groups (see Figures 4a and 4b): data management technologies (discover, access and integrate activities) and BI application processing technologies (analyze, deliver and share and collaborate activities).
Activity Technology
Discover Faceted data search
Business glossary Data analysis & modeling
Data profiling
Access EII data federation
Integrate Data quality management & services ETL data consolidation Changed data capture & propagation
Data replication
Metadata management & reporting
Activity Technology
Analysis Production reporting
Ad hoc analysis
Advanced analytics (statistics, time series, etc.) Data & text mining
Predictive modeling
Performance management scorecards Operational BI (alerts, event analytics,
embedded BI services, etc.) Content analytics
Metadata management & reporting
Deliver Dashboards
Portal integration
Web interface & rich Internet application features Advanced visualization
Office product integration Mobile computing support
Share & collaborate Collaborative & social computing integration Figure 4a:
Data management technology requirements
Figure 4b:
BI application processing technology requirements
Copyright 2009 BI Research, All Rights Reserved. 6 The BeyeNETWORK survey asked respondents about the importance of the
technologies listed in Figure 4 to their organizations. The results are shown in
Figures 5a and 5b. Figure 5a shows the results for data management technologies and Figure 5b the results for BI application processing technologies.
The results for the data management part of the survey show the interest and importance of data quality management technology. Respondents to this part of the survey were also asked about the types of data they needed to integrate into a data warehouse. Some 51% needed to integrate unstructured data, 59% application package data, and 69% application event and web data. The respondents were also asked about the amount of data their data warehouse environments needed to support.
Some 54% of respondents indicated a size in excess of 10 terabytes of data.
Figure 5a:
Percentage of survey respondents rating a data management technology as very important or
somewhat important
Figure 5b:
Percentage of survey respondents rating a BI application processing technology as very important or
somewhat important
54% of survey respondents need to support data
warehouses in excess of 10 terabytes of data
Source: BeyeNETWORK Source: BeyeNETWORK
The results for the BI application processing part of the survey indicate the drive by organizations toward making BI technology more usable. Scores for web-based interfaces, executive dashboards and office product integration demonstrate this trend. As discussed earlier, the results show operational BI to be a key industry direction.
Survey respondents were also asked about several other BI application processing requirements. Items that received high scores here were spreadsheets (73% of respondents), ease of use for less experienced users (78%) and low-cost licensing for a large number of users (79%).
P UTTING THE P IECES T OGETHER : A N E NTERPRISE BI F RAMEWORK
Now that we reviewed the three modes of BI processing (strategic, tactical,
operational), types of information workers (information producers and information consumers), and the data management and BI application processing technologies that support these BI processing modes and information workers, we are now in a position to discuss an enterprise BI framework that can be used to design and build a flexible BI system. An example of such a framework is shown in Figure 6.
The framework in Figure 6 is a more detailed version of the decision workflows illustrated in Figures 1 and 3. The framework shows how data flows between
operational, BI analytical and collaborative applications. At the center of the diagram are the traditional BI applications that produce tactical and strategic data analytics.
The left-hand side of the figure shows how embedded BI services receive operational events and produce event analytics. It also illustrates how embedded BI services interact with operational applications to process service requests. As an example, a 78% of respondents
require ease of use features for less experienced users
Figure 6:
An enterprise BI framework
The BI industry is evolving to support embedded BI and event analytics
Copyright 2009 BI Research, All Rights Reserved. 8 service request could ask a BI service to access customer lifetime value scores from a data warehouse for use in a call center application.
The right-hand side of Figure 6 depicts how analytics are produced from unstructured business content. In some cases, content analytics may be produced directly by content analysis applications, while in other situations the unstructured content may be converted into a structured or semi-structured format and stored in a data
warehouse for use by tactical and strategic BI applications.
At the bottom of Figure 6 are the data management services that handle the data discovery, access and integration activities of a decision workflow. At the top of the figure are the collaborative applications that receive data, event and content analytics, which enable business users to share information and expertise and determine what actions are required to handle different business situations. These collaborative applications also interact with office and social computing applications as well as business portals and other types of web interfaces.
A N E VOLUTIONARY A PPROACH TO D EPLOYING A BI S YSTEM
The enterprise BI framework shown in Figure 6 can be used to construct a BI system that can evolve with the constantly changing BI technology and product landscape.
There are many approaches to constructing such a BI system.
Some organizations, for example, may select and integrate best of breed products, while others may opt for using a single platform from a single supplier. Some
companies may focus on open source solutions in order to reduce software costs. It is also likely, given the current recession, that IT budgets will become restricted, and as a result some business units may deploy their own short-term BI solutions. Given this dynamic environment, it is important that the BI system remain flexible to be able to support and integrate these various approaches.
The BeyeNETWORK survey showed that 52% of respondents are building their BI systems using best of breed products, while 25% said they use a single suite of products from a single supplier. Most of the remainder (21%) also use a single suite of products, but employ best of breed products to fill in any missing capabilities.
Another approach to filling in these gaps is with open source software. About 50% of respondents were using, or planning to use, an open source data management or BI product.
The direction of the BI vendors over the past few years has been to create a single suite of BI products through both product development and product acquisition. As a result, the level of integration in these product suites varies depending on how recent the products were acquired and on how much effort is required to incorporate the acquired products into the BI suite.
When asked if they thought the BI industry benefited from the large number of acquisitions over the past few years, 9% of respondents strongly agreed, 44% partly agreed, but nearly half were neutral or felt is was not beneficial. These results suggest that customers are still unsure about this industry trend and are waiting to see how successful vendors are at integrating acquired products. Regardless, 43% of respondents said they intend to reduce and consolidate the number of BI-related Support for content
analytics is also an important trend
A BI system must be able to interact with collaborative and office applications
Regardless of the approach for
selecting products, a BI system must be able to support different approaches and handle
technology changes
The level of product integration is a key distinguishing feature between vendors
products they use, and 22% were considering it. The level of integration in a product suite and its ability to incorporate other best of breed solutions are key distinguishing features between different vendor solutions.
One approach to building and evolving a BI system using an enterprise BI framework is to create a list of recommended products that can be updated as the market
changes. This approach helps control product proliferation and enables the organization to efficiently manage the move toward supporting the complete enterprise BI framework shown in Figure 6. Among our survey respondents, 66%
have a recommended product list. Not all companies enforce such a list, but two- thirds of those that do maintain a list enforce it and have a policy that deviations have to be justified.
The results of the BeyeNETWORK study clearly demonstrate that organizations see the importance of bringing new and evolving business intelligence and data
warehousing technologies into their organizations. In the data management and data warehousing area companies were especially focused on data quality management, supporting large data warehouses, and on expanding their data warehousing environments to support web and unstructured data. For business intelligence, the areas of focus were on operational BI, and on making business intelligence more usable through web-based interfaces, executive dashboards and integration with office products. Improved usability enables business intelligence to be deployed to a wider user audience, which leverages the investment made in an organization’s BI and data warehousing system.
To exploit new BI and data warehousing developments, organizations need a flexible BI system that can absorb new technology developments and support new types of BI processing without a major impact to existing users. This paper has presented an enterprise BI framework that includes the key elements of such a system. The framework can support the three main modes of BI processing, different kinds of information workers, and the main data management and BI application processing technologies that exist today. It will take time for organizations to build a BI system that supports this framework. The objective of the framework is not to document a short-term objective, but to provide a long-term goal for a BI system that
organizations can gradually move toward and can adjust as products and technologies change.
About BI Research
BI Research is a research and consulting company whose goal is to help companies understand and exploit new developments in business intelligence and data integration. When combined, business intelligence and a sound data integration infrastructure enable an organization to become a smart and agile business.
About the BeyeNETWORK
The BeyeNETWORK focuses on business intelligence, performance management, data warehousing, data integration and data quality, serving these communities with unparalleled industry coverage and resources.
Organizations are trying to reduce the number of BI products they use
Organizations are embracing new BI and data
warehousing technologies
It is important to be able to evolve a BI system to handle different types of BI processing,
information workers and technologies
