Finding your Big Data Way
A multiple case study on the implementation of Big Data
Date: July 2015 Author: Amani Michael
Introduction
New Information Technologies and new possibilities for their application are abound in the current information age. The materializing of Moore’s Law led to the creation of a digital infrastructure of computers, mobile devices, network connections and advanced application platforms. This digital infrastructure accelerated the emergence of new technologies such as Big Data. Big Data is a new IT innovation and technique.
Organizations using Big Data are said to be able to enhance the predictions of customer behavior, find new business models, optimize pricing, improve operation efficiency, reduce labor costs, optimize logistics and improve services etcetera. Although there is suggested that Big Data can bring value for organizations, there is no path described how to implement Big Data. Therefore this paper explores how organizations implement Big Data, leading to the research questions: How do organizations try to implement Big Data?
This research question is answered by conducting a scientific research; a qualitative case study in multiple profit and non-profit organizations. The participating organizations expressed an information need on how other organizations use Big Data and challenges organizations face when implementing Big Data. This paper gives insight in those questions, presenting valuable insights for organizations trying to implement Big Data or trying to obtain value out of it.
Reading guide
I start with a literature chapter, explaining what Big Data is and introducing relevant literature. Next I explain how the research is conducted and which organizations participated in the research. I end this paper with presenting the results and recommendations.
4
1 Literature
1.1 Disambiguation of Big Data
Background
The origins of Big Data are founded by the possibility to store and retrieve data. With the development of social media, internet of things and multimedia, organizations collect increasingly more information, which leads to an exponential growth of data. Currently the daily amount of data exceeds 2.5 exabytes (McAfee and Brynjolfsson) and the globally stored information is increasing with more than 20% each year. The increase of data and turnover in the Big Data Segment makes Big Data relevant for academic and practical initiatives.
Defining Big Data
Big Data is an ambiguous concept and there are different definitions. Gartner defines Big Data as ‘high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization’. This model contains volume, velocity and variety, which is called the 3Vs model. In another definition of Big Data, a fourth ‘V’ is added, referring to value, ‘Big Data technologies describe a new generation of technologies and architectures, designed to economically extract value from very large volumes of a wide variety of data, by enabling the high-velocity capture, discovery, and/or analysis’.
There are thus four Vs:
Volume: The ‘big’ in Big Data’ refers to volume, the amount of data. Each day more and more data is created data sets grow in size because they are gathered by ubiquitous information-sensing devices such as mobile-devises, software logs, networks etc. Variety: There are various types of Data; unstructured, semi-structured and structured
data, but also internal and external data. Previously it was not possible to analyze certain types of data, but technical developments enabled analysis of different types of data. Velocity: Velocity means that Big Data collection and analysis must be rapidly and timely
executed to utilize the commercial value of Big Dat. The speed of the data creation can be more important than the volume, because real-time or near real-time makes organizations more agile than their competitors.
Value: Big Data can create significant value as it enhances the productivity and competitiveness for the public and private sector. Organizations can enhance the predictions of customer behavior, optimize pricing, identify and develop new services and products and identify new markets and customers.
5 Big Data Challenges
Organizations face IT and organizational issues when implementing Big Data:
Technology: In order to capture value from Big Data new technologies and techniques need to be deployed. Issues are data access and incompatible standards. Further formats and legacy systems can prevent integration of data and sophisticated analytics. Data analysis in itself includes other challenges, such as data inconsistency, incompleteness, scalability, timeliness and security. Lack of data quality can lead to false interpretations and conclusions. Data policy issues include privacy, with a trade-off between privacy and utility, security, how to protect competitively sensitive data, intellectual property, with legal issues and liability.
Organizational change: when implementing Big Data, a first task is to create a data minded culture, in which decisions aimed at improving performance are based on data. Employees: Recruiting and retaining analytical and managerial talent is a critical issue.
Data scientists and computer scientists will become more important, as they are able to work with large datasets and communicate with the business. These scientists are hard to find and are in great demand.
1.2 Sense making
The implementation and application of innovations such as Big Data is shaped by how people and organizations make sense of it. In this sense making process individuals develop particular assumptions, expectations and knowledge of the technology, which can influence actions towards that technology.
This sense making process is influenced by Technological Frames people and organizations have. Frames refer to definitions of organizational reality that serve as vehicles for understanding and action. Technological Frames are a subset of those organizational frames and include the nature and role of the technology, but also the conditions, applications and consequences of the technology in a specific context. Technological Frames influence the way innovations are perceived and affect choices made in the design and use of a technology.
6
2 Research design and data collection
Big Data as a phenomenon is relatively new and this connection is not proposed before. Therefore qualitative research (based on interviews) is a logical research approach, as it is relevant when prior insights about a phenomenon are modest. For this research a multiple case study approach is used, as I intend to describe a case within its natural context.
Six cases were chosen for this research from the public and private sector and operating in various industries, with the primary criteria that an organization was already implementing Big Data. In each organization I conducted three to five semi-structured interviews with Managers, Team Leaders, Project Leaders, Business Analysts and / or Business Experts connected with Big Data implementation or application. I conducted 22 interviews which where all recorded and transcribed later, see table 1 on the next page for a short case description.
Table 1
Case description Main tasks/
challenges
B2B / B2C
Governmental organization A is a nationwide organization performing a public task from
the central administration. Last years the organization faced cutbacks, which mainly resulted in less IT investments and downsizing the human capital, meaning they have to do the same job with less employees.
Efficient working B2B & B2C
Governmental Organization B is a regional organization, executing a public task. The
organization started formally on 1 January 2014, therefore organization and formalization is not all in place. The revenues of their services should cover their costs, so being efficient is important.
Efficient working B2B
Broadcasting Company C is a not-for profit broadcasting company and is the result of
a merger between two broadcasting organizations. Due to cutbacks they receive less funding, which is the reason the organizations merged.
Marketing B2C
Insurance Company D is a large insurance company, operating in The Netherlands and
in a dozen other, mainly European, countries. They offer financial products for insurance, pensions, bank savings, savings, mortgages, investments and loans. In recent years the organization implemented budget cuts and reduced the number of employees to increase profitability and keep up with competition.
Pricing B2C
Insurance Company E is a Dutch Insurance Company. Competition is fierce within their
industry. Marketing B2C
Chemical Production Company F is a large international Chemical Production
organization. They have a devisionalized organization. I talked with employees in the Netherlands. Due to the size of the organization it is difficult to generalize over geographical borders and product differences.
pricing, product differentiation B2B
7
3 Results
In this chapter the results are presented on how organizations make sense of and implement Big Data
3.1 Big Data definition
In most cases the Big Data definition was directed to the four V’s. However, the definition of Big Data does not fully define how the respondents made sense of Big Data. This is more defined by the reason to implement Big Data and the value organizations expect.
3.2 Value, Reason to implement and Big Data application
In most organizations the expected value, the reason to implement and the application of Big Data are directed towards the same goal. It is remarkable that the expected value resembles the specific tasks or challenges the organizations face. Organizations see an innovation through their organizational lens expecting that this innovation can resolve or improve existing organizational tasks, challenges or issues. The reason to implement Big Data was mainly directed to enhancing the execution of their tasks or resolving challenges and issues. Further, the expected value and the reason to implement Big Data are related with how Big Data is applied. Most organizations expect that Big Data brings a certain value, which resembles the reason to implement Big Data and how Big Data is used, or how the organization aims to use it.
In Governmental Organizations the expected value, the reason to implement and the application of Big Data are mainly directed to efficiency, which can be explained by the cutbacks they face. In commercial organizations the Big Data initiatives are mainly directed towards competition and marketing, as they experience competition and need to keep up with that competition in order to survive.
3.3 Manner of implementation
In those organizations who use Big Data for marketing purposes there is a connection between the reasons to implement Big Data, the value and how Big Data is organized (which is a part of the manner of implementation). The connection that is found within these organizations might be explained by the clear pre-defined goal. Having clear goals might enable organizations to pursue an implementation strategy, in which the value, organization and application of Big Data are described and aligned. In order to implement Big Data organizations form a team, start an experiment or pilot or try to create support bottom-up. There are no clear differences in the processes between different organizations.
8
3.4 IT issues and Organizational issues
Found IT issues are related to data access, data quality, legacy systems, need for new systems and a shortage of IT capacity. Found organizational issues are related to personnel issues, such as a need for a Big Data Analyst, training and developing / shortage of skills, support, meaning management and operational support or a lack of support, and if the organization is data driven. The IT and Organizational issues affect the application of Big Data, as issues can obstruct an analysis or lower the quality of an analysis. Based on this research the issues do not affect the manner of implementation.
4 Conclusion
This paper explores the question: How do organizations try to implement Big Data?
Organizations implement Big Data by making sense of it; they address the definition, the value and why they would like to implement Big Data. Their views on value of Big Data and reason to implement Big Data are further influenced by their organizational tasks and challenges. Governmental Organizations facing cutbacks see value in more efficiency, allowing them to do the same with less money. Commercial organizations see value in improved and personalized marketing, allowing them to face competition. Organizations thus adopt Big Data because they see a certain value, which is related to the organizational tasks and challenges.
The sense making and the manner of implementation influences how Big Data is applied; organizations choose an application that matches with the expected value and the reason to implement Big Data. The application of Big Data is also influenced by in which team the Big Data analyzing takes place, as that influences the use Big Data (in relation to the task of that team).
Another finding is that organizations with a clear goal towards Big Data have a better alignment between the value and reason to implement Big Data and the organization and application of Big Data. This alignment can contribute to a successful implementation, as the implementation matches with the expectations.
9
Big Data
application
Organizational
task / challenge
Manner of
implementation
Sensemaking
Big Data
definition
Big Data value
Reason to
implement
10
5 Recommendations
This research can bring guidance to managers and project leaders implementing Big Data. Based on the results I advise organizations to do research and find what value Big Data has in relation to your organizational tasks and challenges; what are the planned or expected results or outcomes? Why do you want to implement Big Data? In short: make sense of it before you start implementing it. Having a clear goal namely leads to better alignment between the expected value and the application of Big Data, which can help with a more successful implementation.
Another practical advice is to learn from other similar organizations. Organizations face different IT and organizational issues. And different organizations use different approaches; form a team, start a pilot or try to create support bottom-up. Most organizations participating in this research were curious about the results and experiences of the implementation of Big Data of their competitors. This research shows that companies in the same industry experiences similar issues and goals. Therefore sharing experiences or working together on Big Data can contribute to the implementation and implementation success. Thus; know why your organizations is implementing Big Data, and build on previous experiences of others.
6 Acknowledgements
This research is enabled by the contribution and input of many people. First l like to thank Invenna, for granting me access to their network and supporting my research, in particular Arjan de Koning, René Hopmans and Mischa Westerman. My supervisor from the Vrije Universiteit, Stella Pachidi, who coached me and challenged me to during the thesis project. The research is read carefully by my fellow students Karen Stegers and Laura van Tol which encouraged me reconsider my actions and writings. At last but not least all the interviewees who made time to speak with me, and entrusted me with sometimes delicate information.
11
7 Recommendations for further reading
Brown, B., Chui, M., & Manyika, J. (2011). Are you ready for the era of ‘big data’?. McKinsey Global Institute, 1-12.
Chen, M., Mao, S., & Liu, Y., (2014). Big Data: A Survey. Mobile Networks and Applications, 19-2, 171-209.
Chen, H., Chiang, R.H.L., & Storey, V.C. (2012). Business Intelligence and Analytics: From Big Data to Big Impact. MIS Quarterly, 36-4, 1165-1188.
Gantz, J., & Reinsel, D., (2011). IDC IVIEW Extracting Value from Chaos. EMC Corporation, 1-12.
LaValle, S., Hopkins, M., Lesser, E., Shockley, R., & Kruschwitz, N. (2010). Analytics: The new path to value. IBM Institute for Business Value, 1-21.
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H., (2011). Big data: the next frontier for innovation, competition, and productivity, McKinsey Global Institute, 1-143.
McAfee, A., & Brynjolfsson, E. (2012). Big Data: The Management Revolution. Harvard Business Review, 90, 1-9.
Savitz, E. (2012). Gartner: 10 Critical Tech Trends for the Next Five Years, October 2012. Gartner.
Schlegel, K., (2014). Hype Cycle for Business Intelligence and Analytics, 2014. Gartner, 1-64.
On request: Michael, A.L., (2015). Finding your Big Data way. A multiple case study on the implementation of Big Data. Thesis, Vrije Universiteit Amsterdam.
12
Appendix A: Total table cross case analysis
Co d es G o v ern men t al Organ i z at i o n A G o v ernmental Organization B Bro ad c as t i n g Co mp an y C In s u ran c e Co mp an y D In s urance Company E Ch emi c al P ro d u c t i o n Co mp an y F Industry Governmental Governmental Broadcasting Insurance Insurance Chemical production
Shares listed No No No Yes No Yes
Main tasks Efficient working Efficient working Marketing Pricing Marketing Efficient working Organizational
structure Machine bureaucracy
between machine bureaucracy and simple
Professional
Bureaucracy Divisional structure
Professional
Bureaucracy Divisional structure Geographical
orientation National Regional National International National International
Big Data definition Big Data is not new, Big
Data variety Volume, variety
Volume, variety and analysis, marketing
Variety, velocity, analyzing
Variety, velocity and analysis
variety, volume, velocity analysis Reason to
implement
Get closer to society,
efficiency Efficiency, knowledge marketing
Competition, Insights for business, develop
knowledge
Marketing, customer
profile Heard about it
Value Value is in efficiency, and
generate knowledge Value is in efficiency
Customer profile, marketing Competition, better pricing, customer orientation, process improvements Competitive advantage, customer profile, marketing
efficiency, increase sales
Data application Operational application, new insights/knowledge
New insights, what work to do
Insights past, operational questions,
marketing
Insights past, interest
in forecasting Marketing Operational questions
Implementation Experiment, pilot, engage, agile development
Big Data team, experiment, promotion
Top down and bottom up
Agile development, experiment, form
team, engage operation
Create pull, part of project
Bottom up, pilot, online forum
Organization
Central analytics team, combining data with
expert knowledge
Central analytics team No central department no central team,
cooperation Form new team
Online forum, multiple teams
Data access Availability, control access, external data
No access, access to external data, fear to share
data
No internal access, not share data, fear to
share data
Internal data, external data, share
data
No access fear to share data, not know were data is
Data quality Data preparation, variable quality
Input defines quality, quality
not good quality is good, varies
Invested in data quality, not good due
to systems
Improved quality,
quality varies Quality varies
General IT issues Legacy, new systems needed New IT needed, priority No central data Legacy, limited capacity regional systems Legacy systems,
Complex IT architecture complicates implementation
and combinations of data
Support Support by management, no support by operation Support by management, no support by operation, support by operation Management support Management support, organizational support Management support, operational support varies Management support, operation critical Data driven Management awareness , behaviour to data, judgement on data
Not data driven
Old working methods, analysis mainly
operational
Cultural change, attention
Not used to work
data driven Not discussed HR issues /
development
Training, not right skills,
external knowledge Skills, capacity Civil servants
Capacity, skills, need data Analyst
Adapt skills, need Big Data Analyst
Internal knowledge fragmented General
organizational issues
Older employees, old working methods, limited
view, juridical issues
No realistic expectations Organizational issues
No realistic expectations, old working methods No general issues Older management, conservative organization, division structure to share
