Irish Data Analytics
Landscape Survey
2014-‐2015
Analysis
April, 2015Analytics
The
Store
Table of Contents
1 INTRODUCTION 2
2 SURVEY METHOD 2
3 ANALYSIS OF SURVEY RESPONSES 2
3.1 CHARACTERISTICS OF SURVEY PARTICIPANTS 3
3.2 ORGANISING ANALYTICS 5
3.3 ANALYTICS APPLICATIONS 7
4 SUMMARY 11
1 Introduction
How big is the average Irish analytics team? What industries in Ireland are using analytics? Are companies in Ireland planning to expand their analytics teams in 2015? While there have been a number of European and international surveys of the analytics industry1,2,3 there has not been a specifically Irish survey taking the
pulse of the analytics industry in Ireland. For this reason The Analytics Store launched the Irish Data Analytics Landscape Survey 2014-‐2015 in December 2014. This document describes an analysis of the results of this survey.
2 Survey Method
The survey was conducted online in December 2014 through the SurveyMonkey platform, and promoted primarily using social media. Figure 1 shows a screenshot of the survey interface. The survey contained 16 questions, only a small number of which were mandatory. Depending on the answers that were given to certain questions participants were guided through different routes through the survey. Overall there were 94 survey responses, 75 of which completed the survey. We omit incomplete responses and so include 75 responses in our analysis.
Figure 1: A screenshot of the survey interface.
3 Analysis of Survey Responses
This section analyses the responses of participants to the survey. The survey was broken down into thematic sections and responses within each section are analysed separately.
1 PwC’s Global Data & Analytics Survey 2014: Big Decisions
https://www.pwc.com/gx/en/issues/data-‐and-‐analytics/big-‐decisions-‐survey/index.jhtml
2 NewVantage Partners’ 2014 Big Data Executive Survey
http://newvantage.com/thought-‐leadership/publications-‐and-‐executive-‐surveys/
3 BARC’s Big Data Analytics 2014 Survey
3.1 Characteristics of Survey Participants
The first four mandatory questions in the survey were designed to characterise the survey participants, and the companies in which they worked. Figure 2 and Figure 3 characterise the companies in which participants worked. Two interesting things stand out. First, the survey responses came primarily from people in large companies. Second, the majority of participants (over 75%) came from the banking and finance, IT, and services sectors. This is mostly due to the networks within which the survey was promoted, but is also indicative of the fact that analytics is primarily practiced within larger organisations and in particular industries. The other industries represented in the survey responses were
hospitality and events, media, and gaming.
Figure 2: Question 1 -‐ What size is your organisation? We defined small companies as having less than 20 employees, medium companies as having between 20 and 249 employees and large companies as having 250 or more employees. (Responses: 75)
Figure 3: Question 2 -‐ In which sector does your organisation operate? (Responses: 75)
29.3% 14.7% 56.0% 0% 10% 20% 30% 40% 50% 60%
Small Medium Large
What size is your organisation? 2.7% 4.1% 4.1% 6.8% 6.8% 23.0% 24.3% 28.4% 0% 5% 10% 15% 20% 25% 30% Manufacturing Telecommunications Retail & Wholesale Public Sector Other Services IT Banking & Finance
In which sector does your organisation operate?
The third question in the survey asked participants to characterise their own roles at the company in which they worked. Figure 4 summarises participants’ responses. Half of the participants were in roles that mixed technology and business, with the remainder split between solely business roles, solely technology roles and other roles (which included finance, communications, and
education). We believe that this means that the survey respondents were in a
position to offer an interesting mix of views from both business-‐focused and technology-‐focused points of view.
Figure 4: Question 3 -‐ What best describes your role at your organisation? (Responses: 75) The final question in the first part of the survey asked users to characterise the use of data analytics at their companies. Figure 5 summarises participants’ responses. Only 7 of the survey participants (9.3%) worked in companies where data analytics was not used at all. These participants were redirected to the end of the survey and did not answer any of the other questions. The remainder of the survey participants were using analytics, at least to some extent, which validated their participation in the survey.
Figure 5: Question 4 -‐ Does your organisation use data analytics techniques to drive decision making? (Responses: 75) 50.7% 20.0% 9.3% 20.0% 0% 10% 20% 30% 40% 50% 60% Business &
Technology Business Technology Other
What best describes your role at your organisation? 9.3% 24.0% 41.3% 25.3% 0% 10% 20% 30% 40% 50%
Never Rarely Frequently Almost always
Does your organisation use data analytics techniques to drive decision making?
3.2 Organising Analytics
The next section of the survey asked participants about how analytics was used at their organisations. This began by asking what the major drivers for using analytics were. Figure 6 summarises responses to this question (participants were allowed to select multiple answers). Most participants were interested in improving decision making, either selecting the Ensuring greater accuracy in
decision making option or the Removing "gut instinct" from decision making
option4. This is in line with international survey responses.
Figure 6: Question 7 -‐ What are the major drivers of the use of data analytics for decision making in your organisation? (Responses: 66)
Implementing successful data analytics projects is not without its challenges and Question 8 in the survey addressed this. Figure 7 summarises the responses to this question. It is interesting that the most common challenge faced is the difficulty in hiring suitably qualified staff. Although there has been a growth in third level analytics courses (for example at UCD5 and DIT6) qualified,
experienced analytics practitioners are still thin on the ground. It is likely that this will continue for a number of years until the current pool of graduates gain relevant experience. It is also interesting that, in spite of all we hear about the deluge of data facing us, 35% of participants saw insufficient relevant data as a challenge to their analytics projects. Relevant is the key word here. It is very often the case that although masses of data are available in organisations, there is a dearth of clean, recent, appropriate data for analytics projects. Finally, it is worth noting that only 7 participants (11%) mentioned data protection issues. We expect this to become a bigger challenge in the future as awareness of data protection and privacy issues grows in the public consciousness.
4 Of the 28 participants (42%) who selected the Removing "gut instinct" from decision making option 17 also
selected the Ensuring greater accuracy in decision making option and 11 did not. Taking these 11 participants together with the 45 participants (68%) who selected Ensuring greater accuracy in decision
making this means that 56 participants (85%) were interested in improving decisions making.
5 www.ucd.ie/mathsciences/graduatestudents/onlinecoursesindataanalytics/ 6 www.dit.ie/postgrad/programmes/dt228adt228bmscincomputingdataanalytics/ 20% 23% 38% 42% 55% 68% 0% 20% 40% 60%
Demonstrating our capacity for innovation Regulatory compliance Competitive advantage Removing "gut instinct" from decision making Exploiting our data resources Ensuring greater accuracy in decision making
What are the major drivers of the use of data analytics for decision making in your organisation?
Figure 7: Question 8 -‐ What challenges did you face in undertaking data analytics projects this year? (Responses: 65)
Questions 9 and 10 in the survey asked participants about the size of the analytics teams in which they worked and whether they intended to grow these teams in the coming year. Figure 8 summarises the responses to these questions. It is interesting that in the survey participant pool 26 participants (38%) work in organisations with large analytics teams (more than 10 team members). Having unpacked this responses a little, most of these respondents work either in analytic services companies or at large companies in the banking and finance sector. This is not unexpected as most banking and finance organisations have large risk teams, which do a lot of analytics work.
It is promising for the Irish analytics industry that over half of the people surveyed work in companies that plan to hire new analytics staff in the coming year. A word of caution is required, however, given the fact that access to qualified staff is the number one challenge faced by practitioners.
(a) (b)
Figure 8: (a) Question 8 -‐ What is the size of the data analytics team at your organisation? (Responses: 68) (b) Question 9 -‐ Do you plan to hire additional data analytics staff in 2015? (Responses 67) 5% 11% 22% 23% 25% 35% 45% 0% 10% 20% 30% 40% Lack of compelling business case
Data protection issues Lack of corporate sponsorship Difhiculty accessing suitable tools Cost Insufhicient relevant data Difhiculty hiring suitable staff
What challenges did you face in undertaking data analytics projects this year? 13% 19% 15% 38% 15% 0% 10% 20% 30% 40% 1 2 -‐ 5 6 -‐ 10 > 10 None
What is the size of the data analytics team at your organisation?
43% 37% 12% 7% 0% 10% 20% 30% 40% No 1 -‐ 5 5 -‐ 10 > 10
Do you plan to hire additional data analytics staff in 2015?
Where to house an analytics team is a perennial question at analytics conferences, and Question 11 in the survey explored this. Figure 9 summarises the results. 34% of participants worked in organisations where analytics
belonged to a line of business. In fact most of the other responses actually
referred to a line of business so this number is really closer to 45%. This is the classic siloed approach that tends to emerge as a company begins to use analytics, and is also common internationally. Analytics also remains something associated with IT, and 21% of respondents work in companies in which this is the case. It is promising, however, that in 10% of organisations analytics belonged to a specific analytics team, which suggests that these companies are moving towards a scenario in which analytics is recognised as a function in its own right.
Figure 9: Question 11 -‐ Which department at your organisation controls data analytics? (Responses: 68)
3.3 Analytics Applications
The next section of the survey asked participants about the ways in which they were using analytics at their companies. The first question in this section, Question 12, asked about the departments that utilised the outputs of analytics efforts. Figure 10 summarises these results. Sales and marketing dominates the use of analytics, followed by finance and operations. The prevalence of sales and marketing departments as consumers of analytics is consistent with international surveys. It is interesting to see that 9 of the survey participants (15%) worked in organisations at which analytics was being used in human resources. This is a growing trend that is also seen internationally.
4% 10% 15% 16% 21% 34% 0% 10% 20% 30%
Data Architecture Team Specihic Analytics Team Other BI Team IT Line of Business
Figure 10: Question 12 -‐ Which departments at your organisation utilise the outputs of data analytics projects? (Responses: 60)
It is also interesting to consider the types of analytics that people are doing and the types of data they are working with. Questions 13 and 14 dealt with this, and the results are summarised in Figure 11 and Figure 12. Almost all participants used reporting, which is not surprising, and over two thirds performed statistical and exploratory analysis. It is encouraging to note that over half of the participants reported using some aspects of advanced analytics (for example predictive modelling or forecasting). This speaks to the maturity of the practice of analytics in Irish companies.
Figure 11: Question 13 -‐ Which of the following data analytics techniques has your organisation used in projects this year? (Responses: 60)
15% 22% 23% 27% 30% 62% 62% 75% 0% 10% 20% 30% 40% 50% 60% 70% 80% Human Resources Other Production IT Research & Development Operations Finance Sales & Marketing
Which departments at your organisation utilise the outputs of data analytics projects? 3% 23% 23% 55% 57% 58% 65% 68% 92% 0% 20% 40% 60% 80% 100% Other Association analysis Text analytics Segmentation Forecasting Predictive modelling Exploratory data analysis Statistical analysis Reporting
Which of the following data analytics techniques has your organisation used in projects this year?
Participants primarily reported using transactional data in their analytics work. Again this is in line with international results. The use of unstructured data (for example text, audio, images or video) is growing in the international analytics community and was relatively well represented here with 12 respondents (20%) reporting its use.
Figure 12: Question 14 -‐ What types of data did you use in data analytics projects this year? (Responses: 60)
Question 15 in the survey asked respondents which tools they were using for analytics projects -‐ their responses are summarised in Figure 13. SQL and Excel were the most commonly used tools, and remain workhorses on most analytics projects. After this there were a wide spread of tools, with the open source programming language R being the next most commonly used tool. For advanced analytics tools there seems to be an even balance between GUI-‐based tools (for example IBM SPSS, SAS Enterprise Miner and or RapidMiner), and programming languages (for examples Base SAS, R and Python). The tools mentioned more than once under the other category were Qlikview and Teradata.
7% 15% 20% 28% 32% 85% 0% 20% 40% 60% 80% 100% Other Sensor data Unstructured data Social media data Log data Transactional data
Figure 13: Question 15 -‐ Which of the following data analytics tools or languages did your organisation use in projects this year? (Responses: 62)
The final question in the survey asked participants whether they had used so-‐ called big data tools in an analytics projects this year (a list of the following tools was included: BigML, Cassandra, Giraph, Hadoop, HBase, Hive, Mahout, Pig, and Spark). Figure 14 summarises the responses. 21 respondents (28%) had used one or more of these tools. This suggests that people are starting to work with datasets outside of the typical small transactional data that characterises early use of analytics. The most commonly used big data tools were Hadoop (15 respondents), Hive (5 respondents), and Spark (3 respondents). It will be interesting to see if the use of these types of tools increases in future iterations of this survey. 3% 3% 6% 6% 6% 6% 6% 15% 18% 18% 26% 29% 32% 34% 37% 58% 74% 0% 20% 40% 60% 80% Weka KNIME SAP Rapidminer SPSS Matlab Oracle Data Miner IBM SPSS Other Python Tableau Microsoft SQL Server SAS (Enterprise Miner) SAS (Base) R SQL Excel
Which of the following data analytics techniques has your organisation used in projects this year?
Figure 14: Question 16 -‐ Were "big data" specific tools or languages used by your organisation in projects this year? (Respondents: 75)
4 Summary
The purpose of this survey was to capture the data analytics landscape in Ireland in 2014-‐2015. The number of survey respondents was not high enough to say that the results are fully representative of the entire analytics industry, but they are sufficient for interesting analysis and present a snapshot of the current state of the industry, and a baseline for comparison of future studies.
The survey results paint a picture of a reasonably mature analytics industry, with many participants reporting analytics applications moving beyond descriptive analytics to the application of advanced analytics techniques (such as predictive modelling), which in some cases use interesting unstructured data sources and high-‐end big data toolsets. There is also some evidence for the emergence of strong, centralised analytics teams -‐ although in most cases analytics work is siloed in a line of business.
The main challenge facing analytics practitioners is a lack of experienced, qualified candidates to fill analytics roles. It is expected that this will continue to be the case for the immediate future. In-‐house training of existing staff is likely to be a solution to this issue for many companies. It is somewhat surprising that, in spite of the data deluge we constantly hear about, a lack of relevant data continues to be an issue for some practitioners. This is likely to continue to be the case and highlights the importance of strong data management and governance.
As we repeat this survey in the coming years it will be interesting to benchmark against these results to see how the Irish data analytics landscape is changing.
Yes 28%
No 72%
Were "big data" speci[ic tools or languages used by your organisation in projects this year?
