• No results found

THE STATE OF THE DATA WAREHOUSE

N/A
N/A
Protected

Academic year: 2021

Share "THE STATE OF THE DATA WAREHOUSE"

Copied!
10
0
0

Loading.... (view fulltext now)

Full text

(1)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

March 2015

(2)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

Introduction

As the volume and types of business data have increased at a phenomenal pace, and the cost to store that data has plummeted, businesses have looked to data analytics to gain new insights into their customers and operations. The buzz about Big Data, including the use of technologies like Hadoop, has dominated the news. But what does that mean to the future of data warehousing? Are companies abandoning their data warehouses in favor of new technologies? And what is the impact of new paradigms created by cloud computing?

The following report, sponsored by Snowflake Computing, is based on a survey of 319 individuals with responsibility for data initiatives. The goal of the survey was to understand experiences, challenges and trends in data warehousing and data analytics.

Key Findings

• Data warehouses still critically important, but challenges remain

- 99% say their data warehouse is important to their business operations - 70% are increasing their investment in data warehousing

- 97% face challenges with their current data warehousing solutions - Scalability and cost are top challenges

• Plenty of interest in Big Data, but so far investment is limited

- 91% have considered a Big Data investment

- Only 11% have a pilot in place, and only 5% have fully deployed their Big Data initiative - 91% have concerns about Hadoop

- 96% say Hadoop will not replace their existing data warehouse

- Only 12% have easy access to Hadoop expertise; in contrast, 93% have easy access to SQL expertise

• Cloud-based data warehousing desirable, especially for data in the cloud

- 93% see value in the potential benefits of cloud data warehousing - Scalability and reduced overhead most valuable

- 32% have a cloud-based data warehouse today; 16% use only a cloud-based data warehouse - 49% of those with data already in the cloud have a cloud data warehouse

- 79% of those with data in the cloud bring it into the data warehouse, although only 9% bring all data in

(3)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

www.dimensionalresearch.com © 2015 Dimensional Research.All Rights Reserved. Page 3

Detailed Findings

Data warehouses still critically important, but challenges remain

When asked about the importance of their data warehouse to business operations, almost all of those responsible for data initiatives (99%) indicated that it was important, including almost three-quarters (72%) who characterized it as very important. Only a very few (1%) described their data warehouse as not important.

Even with the significant amount of discussion of alternative Big Data platforms, data warehousing investment

continues to grow. A large majority of companies (70%) are increasing their data warehousing investment, while only a very small number of companies (4%) are reducing their investment in data warehousing.

n = Accountants

72%   27%   1%  

0%   10%   20%   30%   40%   50%   60%   70%   80%   90%   100%   How  important  is  your  data  warehouse  to  business  opera3ons?  

Very  important   Important   Not  important   Investment  is   increasing   70%   Investment  is   decreasing   4%   Not  changing   26%  

How  is  the  level  of  investment  in  your  company’s   data  warehouse  changing?    

(4)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

That said, data professionals do see room for improvement in their existing data warehouse. The vast majority of data professionals surveyed (97%) face challenges with their current solutions for data warehousing. Top challenges cited included scalability (43%), cost to maintain (38%), cost to purchase (37%), and the difficult learning curve for business users (37%).

Plenty of interest in Big Data, but so far little investment

To understand the impact of Big Data initiatives, we first examined the state of those initiatives. We discovered that while most data stakeholders are interested in Big Data – only 9% have not considered a Big Data initiative – most are still investigating or planning to investigate.

Just 16% have made an investment in Big Data, including 5% who are fully deployed and a further 11% that are in a pilot. Most companies, 70%, are interested in Big Data but haven’t yet made any investment. That does include 29% who are currently investigating their options, but the largest number is those who are intrigued, but haven’t yet done any work to find out more (41%).

3%   19%   30%   32%   33%   37%   37%   38%   43%   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%   50%   We  have  no  challenges  with  data  warehousing  

Not  able  to  handle  all  of  the  different  types  of  data  we  have   Takes  too  long  to  deploy  new  projects   Difficult  to  find  experJse  to  manage   Users  complain  it  is  too  slow   Challenging  for  business  users  to  learn   Very  expensive  upfront  purchase  costs   Very  expensive  to  maintain  and  operate   Concerned  about  ability  to  scale  to  handle  projected  data  and  user  growth  

What  challenges  do  you  face  with  your  current  data  warehousing  solu4ons?  

5%   11%   29%   41%   5%   9%   0%   5%   10%   15%   20%   25%   30%   35%   40%   45%  

Fully  deployed   Have  a  pilot  in  place   Currently  inves<ga<ng   We  are  interested,  but  

haven’t  inves<gated  yet   Have  inves<gated  and  decided  not  to  pursue   Not  being  considered   What  investment  is  your  company  making  in  Big  Data?    

(5)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

www.dimensionalresearch.com © 2015 Dimensional Research.All Rights Reserved. Page 5

Hadoop not a practical solution for most enterprises

Big Data is rarely discussed without talking about Hadoop, the most common foundational technology for these initiatives. It appears that concerns about Hadoop may be an important factor in the slow pace of investment in Big Data. The vast majority of data stakeholders (91%) indicated that they have concerns about using Hadoop in their organization, with the biggest concern by far being lack of expertise (71%).

It is not surprising that lack of expertise is a challenge for any new technology like Hadoop. In contrast, SQL expertise is common. The vast majority of data stakeholders surveyed can easily access SQL expertise (93%), while only a few (12%) have easy access to Hadoop expertise.

9%   4%   18%   23%   24%   71%   0%   10%   20%   30%   40%   50%   60%   70%   80%   I  have  no  concerns  

Other   Hadoop  is  too  complex   No  compelling  business  need   Don’t  have  enough  data  to  bother   Lack  of  experIse  with  Hadoop  

Do  you  have  any  concerns  about  using  Hadoop  in  your  organiza6on?    

Yes   93%   No   7%  

 Do  you  have  easy  access  to  SQL  exper3se?  

Yes   12%  

No   88%  

(6)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

This difference in access to Hadoop expertise is even more pronounced for smaller companies, whereas SQL expertise is accessible to almost everyone. Among the companies in our study with fewer than 1,000 employees, only 5% have access to Hadoop skills, while most (92%) have easy access to SQL expertise.

When asked directly about opinions on Hadoop as a possible replacement for the data warehouse, only a very small number (4%) reported that they thought Hadoop could eventually replace their existing data warehouses. Most data stakeholders (64%) believed Hadoop would not impact their data warehouse, and any investment would be fully complementary. About a third (32%) did think that Hadoop could replace some but not all of what was done today using a data warehouse.

95%   93%   92%   22%   13%   5%   0%   20%   40%   60%   80%   100%   More  than  5,000   1,000  -­‐  5,000   100-­‐1,000  

Access  to  SQL  or  Hadoop  exper1se  by  company  size  

Have  easy  access  to  SQL   exper@se  

Have  easy  access  to  Hadoop   exper@se  

No,  Hadoop  will  be   complementary  to  our  data  

warehouses   64%   Hadoop  will  replace  some  of  

what  we  do  with  our  data   warehouse,  but  not  all  

32%  

Yes,  Hadoop  will  completely   replace  our  need  for  data  

warehouses   4%  

(7)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

www.dimensionalresearch.com © 2015 Dimensional Research.All Rights Reserved. Page 7

Cloud-based data warehouse seen as valuable

Data stakeholders see potential value in having a data warehouse that is in the cloud. The majority (93%) see benefits in cloud data warehousing with the right capabilities. A cloud-based data warehouse with the ability to scale on demand (66%) and deliver lower administrative overhead (59%) would deliver the most benefit.

Because of this potential value, cloud data warehousing is no longer something that only early adopters are using. Almost a third of companies (32%) are already using cloud-based data warehousing, including 14% who have exclusively adopted cloud and do not support an on-premises data warehouse.

7%   27%   47%   47%   53%   59%   66%   0%   10%   20%   30%   40%   50%   60%   70%   None  of  these  would  be  valuable  

Op=mized  for  structured  and  semi-­‐structured  data   Simplify  access  for  business  users   Ability  to  easily  share  access  to  data   Reduced  cost   Less  administra=ve  overhead   Ability  to  scale  on  demand  

Which  of  these  poten-al  benefits  of  a  cloud-­‐based  data  warehouse   would  be  valuable  to  your  organiza-on?    

32%  

86%  

0%   10%   20%   30%   40%   50%   60%   70%   80%   90%   100%   Cloud-­‐based  data  warehouse  

On-­‐premises  data  warehouse  

(8)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

What concerns have kept the rest from using a cloud data warehouse? When we asked those data stakeholders who did not have a cloud data warehouse, the top reason was that they had concerns about security (64%).

It is worth noting that some of these who do not have a cloud-based data warehouse today do plan to adopt the technology in the near future. This includes 16% who indicated that they want a cloud-based data warehouse but haven’t gotten around to it yet, as well as multiple participants who used the “other” option to indicate they were actively in process with a cloud-based data warehouse initiative. who wrote in as an “other” option saying that they were actively in process with a cloud-based data warehouse initiative.

Cloud data warehouse a natural fit for cloud data

Cloud services have traditionally caused challenges for data and business intelligence professionals because cloud data needed to be brought back into the datacenter to be available for general use. However, this has clearly changed. Almost two-thirds of participants in this survey (62%) have data in the cloud – either in SaaS applications like Salesforce.com or in public infrastructure clouds like Amazon AWS. These participants were asked how that data fit into their data warehouse initiative.

Most companies with data in the cloud are bringing at least some of their cloud data into a data warehouse (79%). Only a small portion of those (9%) are bringing all cloud data into the data warehouse. Most (55%) have less than half of their cloud data in their data warehouse.

10%   11%   16%   19%   26%   64%   0%   10%   20%   30%   40%   50%   60%   70%   Other  

Have  never  thought  about  it   We’d  like  to  but  haven’t  go?en  around  to  it   Not  saBsfied  with  current  cloud-­‐based  data  warehouse  offerings   Technically  complicated  to  move  our  data   Concerned  about  security  of  data  in  the  cloud  

Why  doesn’t  your  company  use  a  cloud-­‐based  data  warehouse?    

n = do not have a cloud data warehouse

n = have data in the cloud

21%   33%   22%   15%   9%   0%   5%   10%   15%   20%   25%   30%   35%  

No  data   Less  than  25%   25-­‐50%   More  than  half   All  data   Percentage  of  cloud  data  brought  into  a  data  warehouse  

(9)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

www.dimensionalresearch.com © 2015 Dimensional Research.All Rights Reserved. Page 9

Interestingly, there is a clear correlation between companies who have data in the cloud, and those who have a cloud-based data warehouse. When we consider only companies who have data in the cloud, almost half have a cloud data warehouse (49%), while very few companies who don’t use data in SaaS applications or public infrastructure clouds have adopted cloud for data warehousing (4%)

Survey Methodology and Participant Demographics

In February 2015, technology and analytics stakeholders with responsibility for corporate data initiatives were invited to participate in an online survey on the topic of data analytics and warehousing. Participants were asked a series of questions about their role in data initiatives as well as specific questions on experiences, challenges, and trends with data warehousing, Big Data, Hadoop, and cloud.

A total of 319 individuals completed the survey. All had professional responsibility for data initiatives. Participants represented a wide range of geographies, company sizes, role, and vertical industries. All participants worked at companies that used data warehousing.

8%   11%   27%   46%   51%   0%   10%   20%   30%   40%   50%   60%   BI  Business  Owner  

BI  or  Analy;cs  Technology  Owner     IT  execu;ve  with  data  ini;a;ves  in  my  porFolio   IT  staff  responsible  for  implemen;ng  and  opera;ng  

data  infrastructure       IT  manager  responsible  for  delivering  data  ini;a;ves  

Role  in  data  ini+a+ves  

100  –  1,000   42%   1,000  –   5,000   37%   More  than   5,000   21%   Company  Size   5%   1%   1%   1%   2%   2%   2%   2%   4%   4%   4%   8%   9%   12%   13%   14%   15%   0%   2%   4%   6%   8%   10%   12%   14%   16%   Other   Pharmaceu6cal   Media  and  Adver6sing   Transporta6on   Telecommunica6ons   Hospitality  and  Entertainment   Food  and  Beverage   Retail   Energy  and  U6li6es   Services   Non-­‐Profit   Government   Educa6on   Healthcare   Manufacturing   Financial  Services   Technology   Ver$cal  Industry   Yes   62%   No   38%  

Have  business  data  in  the  cloud  

4%   49%   0%   20%   40%   60%  

No  cloud   Cloud  data  

(10)

THE STATE OF THE DATA WAREHOUSE

A SURVEY OF DATA PROFESSIONALS

Dimensional Research | March 2015

About Dimensional Research

Dimensional Research® provides practical market research to help technology companies make their customers more

successful. Our researchers are experts in the people, processes, and technology of corporate IT and understand how corporate IT organizations operate. We partner with our clients to deliver actionable information that reduces risks, increases customer satisfaction, and grows the business. For more information visit www.dimensionalresearch.com.

About Snowflake Computing

Snowflake Computing, the cloud data warehousing company, was founded in 2012 by a team of experts who hold over 120 patents in the fields of database architecture, data warehouses, query optimization and parallelization. Snowflake was created with the vision to reinvent the data warehouse, bringing together all users, all data and all workloads in a single repository. The company is backed by leading investors including Redpoint Ventures, Sutter Hill Ventures and Wing Ventures. Snowflake is headquartered in Silicon Valley and can be found online at www.snowflake.net.

References

Related documents

Players can create characters and participate in any adventure allowed as a part of the D&amp;D Adventurers League.. As they adventure, players track their characters’

En efecto, así como los libertarianos ven en cual- quier forma de intervención del Estado una fuente inevitable de interferencias arbitrarias –con la excepción de aquella acción

PowerScore test takers who analyze the answer choices will realize that they can skip the first three answer choices because all three expressions are squared, meaning that all

[87] demonstrated the use of time-resolved fluorescence measurements to study the enhanced FRET efficiency and increased fluorescent lifetime of immobi- lized quantum dots on a

The variable angle valley gutter system provides secure and watertight installations for both glass and polycarbonate glazing and is able to accommodate a wide range of

Expanding Window Random Linear Codes (EW RLCs) are a flexible unequal error protection fountain coding scheme which can provide prioritized data transmission.. In this paper, we

Although the effects of career-related social support have been demonstrated for a general population, the question remains how social support at school or at work

Advances in medical treatment, management, diagnosis and surgical palliation have improved the quality and longevity of children born with Congenital Heart Disease. As