IBM Analytics The fluid data layer: The future of data management

(1)

(2)

Resources

1 2 3 4 5 6

The new world vision for data architects

Why the fluid data layer matters

Must-have capabilities

The fluid data layer in the real world

(3)

3 1 The new world vision

for data architects 2 Why the fluid data layer matters 3 Must-have capabilities 4 The fluid data layer in the real world 5 Benefits for the hybrid future 6 Resources

The new world vision for data architects

The world of the data architect is getting more complex. New technologies—such as clustered databases, cloud platforms and mobile data synchronization—have become today’s reality. Now more than ever, data architects must change the way they look at the data under their purview.

First, there must be freedom of movement. Data must be able to move

where users need it, when they need it, even in real time.

There must be freedom of location.

Data should live where its management, storage and analytics are most economical and logical. This could apply to any number of places, including data centers, private clouds, public clouds, on-premises storage or user devices.

Finally, there must be freedom of expression and assembly. Data must

be available and accessible so it can be combined with other sources for more detailed insights and analysis.

(4)

But for this vision to come to fruition, the traditional boundaries and data management structures have to change. Data type and location no longer matter; what is important is data access, responsiveness, accuracy and analytic power.

This e-book explores a new

approach to making data available and accessible when and where it’s needed most: the fluid data layer.

Creating a fluid data layer enables a business to move with unprecedented flexibility. By enhancing a hybrid cloud environment with data refinement,

integration, management and more, data architects can free data from location restrictions and capitalize on its full value.

(5)

The fluid data layer offers numerous advantages for data architects:

• Being able to offload processing quickly

to a hybrid cloud improves uptime

• Organizations can adjust seasonal

or daily spikes without over- provisioning equipment

• Understanding the shortest path of

movement between data source and user ensures lower latency and faster access

2 Why the fluid data layer matters

• Teams can try short-term projects that

may have been expensive to attempt previously, because data architects can quickly stitch together systems without worrying about intensive data movement efforts

• If business users have an idea for a new

application, data architects can get prototypes up and running quickly to prove or disprove concepts

Why the fluid data layer matters

What’s a data layer?

(6)

On an organizational level, the fluid data layer helps data architects keep the data warehouse infrastructure out of the users’ way, while still providing tremendous business benefits (see Figure 1). Data architects can more easily move not just data, but databases themselves. For example, a music-sharing service with a data center on the eastern seaboard of the US suddenly discovers it’s a big hit in Asia. Its data architects can set up a cloud-based data center in Asia that improves performance and reduces latency. If either data center goes down, the other will act as a backup.

2 Why the fluid data layer matters

Figure 1.A fluid data layer distributes data management across data sources and allows simple data access and movement.

Analytics Applications Mobile Users

Automatic

synchronization Alwaysavailable Scalable

Location-independent access

Private

cloud centerData Publiccloud data centerCloud Devices Offices Off-site/remotelocations DBaaS IBM DataWorks Acquisition Refinement Cleansing Security IBM dashDB™ portable

(7)

for data architects 2 Why the fluid 2 Why the fluid data layer mattersdata layer matters 3 Must-have capabilities 4 The fluid data layer in the real world 5 Benefits for the hybrid future 6 Resources

That fluidity extends further. One of

enterprises’ big concerns with as-a-service offerings is lock-in and inflexibility. A fluid data layer eliminates that concern, letting data architects shift data from one cloud-based system to another as necessary, changing the architecture without recoding or interrupting data accessibility.

(8)

To create a fluid data layer, architects need a full portfolio of management capabilities that can be used together in various combinations to meet business needs, and easily integrated and reused as those needs change.

That means being able to:

• Move between internal data centers and

cloud-based systems

• Handle structured and unstructured data

3 Must-have capabilities

Must-have capabilities

• Scale up and down based on traffic, time

of day and other parameters

• Integrate existing tools, systems and data

(9)

Data management

At the heart of the data layer is a database management system and the supporting distributed infrastructure. Data architects rely on this common foundation for data sourcing and targets, using their experience to build on other technologies in the portfolio. But a data management strategy should also let data architects create a database in the cloud that can be quickly populated for a short-term application, whether for a pilot project or a seasonal need.

Using a cloud database helps reduce costs because developers can provision environments quickly, but also provides agility though rapid project testing. Data architects should also have a way to build “portable analytics” that flow from where data is collected to where it’s analyzed. Taking advantage of both in-memory and columnar technology enables them to deliver insights faster. Features that allow integration between the data layer and current tools and systems increase reuse opportunities and reduce the steep learning curves on new techniques.

Data refinement

Data architects need to deal with extract, transform, load (ETL) quickly, to identify the source and the target of data and automate the exchange of data between the two. That means helping data architects classify, profile, cleanse and qualify data to ensure consistent accessibility and usability across and beyond the enterprise. Plus, while enterprises are becoming more confident about the cloud’s ability to secure data when it’s in transit, protecting sensitive data remains critical. The fluid data layer must automatically encrypt data both at rest and in transit.

(10)

Data analytics

Data architects must be able to analyze streaming data—sensor data, telematics, GPS information, regional sales data and more—with as little delay as possible. Ideally, they should be able to design systems so that analytics can be performed in near-real time as data enters the system, regardless of the number of sources. Processing data from multiple sources offers the ability to add context to insights for additional understanding.

The same need for speed applies to

unstructured data. With the fluid data layer, data architects can assemble output from unstructured databases ranging from Apache Hadoop systems to NoSQL and JSON data flowing into an in-memory database. They don’t have to force data into a schema that’s not appropriate, which makes development more agile. There should also be no restrictions on tools so data architects can use SQL to craft queries of unstructured databases or take advantage of high-level statistical analysis.

(11)

for data architects 2 Why the fluid data layer matters 3 Must-have capabilities 4 The fluid data layer in the real world 5 Benefits for the hybrid future 6 Resources IBM cloud data services run on the IBM

SoftLayer® bare-metal cloud platform, which enables reliability, performance and a flexible global footprint.

IBM gives you the flexibility to deploy your cloud data on public, private or hybrid cloud platforms to maximize cost efficiency, control and performance.

IBM is renowned for its data management intellectual property, now available through the cloud.

IBM integrates its cloud data services to make them easier to use, avoiding the integrate-and-provision-it-yourself burden imposed by other standalone services.

Why IBM for cloud data management?

3 Must-have capabilities

1

2

3

4

The IBM advantage: Integrated data services

(12)

for data architects 2 Why the fluid data layer matters 3 Must-have capabilities 4 The fluid data layer 4 The fluid data layer in the real worldin the real world 5 Benefits for the hybrid future 6 Resources

The fluid data layer in the real world

Any industry that relies on data for insights and efficiency will benefit from the fluid data layer. Even more important, this architecture can break down barriers for industries that traditionally store lots of data but have been unable to share it or analyze all of it. What could you do if data location was irrelevant?

IBM customer Physion has created a fluid data layer for its cloud-based Ovation application,1_{which makes it}

easier for scientists to store, organize, manage and share experimental research data. By breaking down barriers to scientific collaboration, Ovation maximizes the value of data otherwise trapped on individual researcher desktops or within organizational silos.

Data architects for a retailer could craft a cloud-based data warehouse that

combines structured data such as inventory with unstructured data from social media or weather services to anticipate upcoming adverse weather conditions. The result: shipping tools or food to stores whose shelves need replenishment fast.

In healthcare, the ability to incorporate data from multiple systems can

help providers and public health

organizations share data about illness outbreaks. Analyzing social media

(13)

Or consider the potential benefit to a ridesharing service, aggregating data

from a variety of different data sources to improve the customer experience and ensure its drivers are in the right place at the right time. Data architects could take historical data showing when its service is in highest demand, and aggregate it with real-time information about current

demands. They could also load additional weather and traffic data to provide more complete insight. With this information delivered in near-real time, the ridesharing company can inform its drivers of the best places for pickups and provide customers with more accurate travel times. Customers have shorter wait times, and drivers get more business.

dashDB

Data warehouse services

Pre-route drivers to heavy use locations at peak request times

Advise drivers of routes around construction, flooding or accidents

Proactively send more cars to areas with historically high volumes and long wait times Weather data

Traffic patterns Road construction Local event schedules

DataWorks Location Time of request Wait times Mobile rideshare app

running on IBM Cloudant

Refined, cleansed, secure data

Analytics

Figure 2.Combining multiple data sources and types, analyzing all of the data and delivering it in near-real time enables a ridesharing service to reduce customer wait times and keep drivers busy.

(14)

for data architects 2 Why the fluid data layer matters 3 Must-have capabilities 4 The fluid data layer in the real world 5 Benefits for the 5 Benefits for the hybrid futurehybrid future 6 Resources

Benefits for the hybrid future

The fluid data layer provides a foundation for accessibility, agility and adaptability.

Data architects can now enable self-service access to trusted data. They can give developers the ability to embed data service into new applications and give business analysts the ability to find, use and contribute data for analysis. Data redundancy across data sources—mobile, on-premises, in the cloud or elsewhere— offers non-stop data access for all users. By expanding data layers on demand, they can hold more information for data scientists to analyze. And—perhaps most important— they can use the inherent scalability to expand and relocate the data layer as necessary, allowing it to adapt to future needs without a massive replacement or upgrade effort.

(15)

To learn more about IBM services that can help you build and enhance the fluid data layer, visit these resources:

•Bluemix:ibm.com/software/bluemix

•IBM Cloud Marketplace:ibm.com/marketplace/cloud

•IBM cloud computing:ibm.com/cloud-computing

•IBM Cloudant: https://cloudant.com

•IBM DataWorks:ibm.com/software/data/make-data-work

• IBM dashDB:ibm.com/software/data/dashdb

6 Resources

(16)

A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/

copytrade.shtml

SoftLayer is a trademark or registered trademark of SoftLayer, Inc., an IBM Company.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates.

The client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided.

1_{Physion life science app: IBM Cloudant case study.} https://cloudant.com/resources/case-studies