Big Data Ppt

BIG DATA

BIG DATA

Prepared By

Prepared By

Vinay Krishna Medishetty

Vinay Krishna Medishetty

Under the guidance of

Under the guidance of

Richey

Introduction

Introduction

 Big Data may well  Big Data may well e the !e"t Big #hing in the $#e the !e"t Big #hing in the $# world%

world%



Big data urst upon the scene in the &rst decade ofBig data urst upon the scene in the &rst decade of the '(

the '(st cest centuryntury%%



 _{#he &rst organi)ations to emrace it were online and #he &rst organi)ations to emrace it were online and}

startup &rms% F

startup &rms% Firms like *oogle+ eBay+ irms like *oogle+ eBay+ ,inked$n+ and,inked$n+ and Faceook were uilt around ig data from the

Faceook were uilt around ig data from the eginning%

eginning%



,ike many new information technologies+ ig data can,ike many new information technologies+ ig data can ring aout dramatic cost reductions+ sustantial

ring aout dramatic cost reductions+ sustantial impro-ements in the time re.uired to perform a impro-ements in the time re.uired to perform a

computing task+ or new product and ser-ice o/erings% computing task+ or new product and ser-ice o/erings%

What is BIG DATA?



_{‘Big Data’ is similar to ‘small data’, but bigger in}

size



but having data bigger it requires different

approaches:



_{Techniques, tools and architecture}



an aim to solve new problems or old problems in a

better way



Big Data generates value from the storage and

processing of very large quantities of digital

information that cannot be analyzed with

traditional computing techniques

What is BIG DATA



_{0al1Mart handles more than ( million customer}

transactions e-ery hour%

2 Faceook handles 34 illion photos from its user

ase%

2 Decoding the human genome originally took (4years

to process5 now it can e achie-ed in one week%

Three Characteristics of Big Data V3s

   V

  o

 l

  u

   m

  e

   D

  a

  t

  a

  .

  u

  a

  n

  t

 i

  t

  y

   V

  e

 l

  o

  c

 i

  t

  y

   D

  a

  t

  a

  6

  p

  e

  e

  d

   V

  a

  r

 i

  e

  t

  y

   D

  a

  t

  a

  #

  y

  p

  e

  s

1

 Character of Big Data

Volume

•

7 typical P8 might ha-e had (4 gigaytes of storage

in '444%

•

 #oday+ Faceook ingests 944 teraytes of new data

e-ery day%

•

Boeing :;: will generate '34 teraytes of <ight data

during a single <ight across the U6%

•

 #he smart phones+ the data they create and

consume5 sensors emedded into e-eryday o=ects will

soon result in illions of new+ constantly1updated data

feeds containing en-ironmental+ location+ and other

2nd Character of Big Data

Velocity



 8lick streams and ad impressions capture user

eha-ior at millions of e-ents per second



 high1fre.uency stock trading algorithms re<ect

market changes within microseconds



 machine to machine processes e"change data

etween illions of de-ices



 infrastructure and sensors generate massi-e log data

in real1time



 on1line gaming systems support millions of concurrent

users+ each producing multiple inputs per second%

3rd Character of Big Data

Variety



_{Big Data isn>t =ust numers+ dates+ and strings%}

Big Data is also geospatial data+ ;D data+

audio and -ideo+ and unstructured te"t+

including log &les and social media%



 _{#raditional dataase systems were designed to}

address smaller -olumes of structured data+

fewer updates or a predictale+ consistent data

structure%



_{Big Data analysis includes di/erent types of}

toring Big Data



_{Analy!ing your data characteristics}



_{6electing data sources for analysis}

_{?liminating redundant data}



_{?stalishing the role of !o6@,}



_{"#er#ie$ of Big Data stores}



_{Data modelsA key -alue+ graph+ document+}

column1family



_{adoop Distriuted File 6ystem}



_Base

%rocessing Big Data



_{Integrating dis&arate data stores}



_{Mapping data to the programming framework}

_{8onnecting and e"tracting data from storage}

 _{#ransforming data for processing}



_{6udi-iding data in preparation for adoop Map}

Reduce



_{'m&loying (adoo& )a& *educe}



_{8reating the components of adoop Map Reduce}

 =os



_{Distriuting data processing across ser-er farms}

_{?"ecuting adoop Map Reduce =os}

Data



_6tructured

•

Most traditional

data sources



_{6emi1structured}

•

Many sources of ig

data



_Unstructured

•

Video data+ audio

Why Big Data

•

!rowth of Big Data is needed

 –

"ncrease of storage capacities

"ncrease of processing power

 –

#vailability of data$different data types%

 –

&very day we create '( quintillion bytes of data) *+

of the data in the world today has been created in

the last two years alone

Why Big Data

B generates (4#B daily

itter generates :#B of data

aily

M claims C4 of todayEs

tored data was generated

Big Data Analytics

&-amining large amount of data

#ppropriate information

"dentification of hidden patterns, un.nown correlations

/ompetitive advantage

Better business decisions: strategic and operational

&ffective mar.eting, customer satisfaction, increased revenue

Ty&es of tools used in

Big+Data



_{0here processing is}

_hosted



_{Distriuted 6er-ers G 8loud He%g% 7ma)on ?8'I}



_{0here data is}

_stored



_{Distriuted 6torage He%g% 7ma)on 6;I}



_{0hat is the}

_{&rogramming model}



_{Distriuted Processing He%g% MapReduceI}



_{ow data is}

_{stored , inde-ed}



_{igh1performance schema1free dataases He%g%}

MongoDBI



_{0hat operations are performed on data}

analytics

*is0s of Big Data

•

0ill e so o-erwhelmed

•

!eed the right people and sol-e the right prolems

•

8osts escalate too fast

•

$snEt necessary to capture (44

•

Many sources of ig data

is pri-acy

•

self1regulation

,egal regulation

(o$ Big data im&acts

on IT

•

Big data is a troulesome force presenting

opportunities with challenges to $#

organi)ations%



_{By '4(9 3%3 million $# =os in Big Data 5 (%C}

million is in U6 itself 



_{$ndia will re.uire a minimum of ( lakh data}

scientists in the ne"t couple of years in

addition to data analysts and data managers

to support the Big Data space%

Benets of Big Data

•

Real1time ig data isnEt =ust a process for

storing peta ytes or e"a ytes of data in a

data warehouse+ $tEs aout the aility to make

etter decisions and take meaningful actions at

the right time%

•

Fast forward to the present and technologies

like adoop gi-e you the scale and <e"iility to

store data efore you know how you are going

to process it%

•

 #echnologies such as Map Reduce+i-e and

Benets of Big Data

_{Jur newest research &nds that organi)ations are using}

ig data to target customer1centric outcomes+ tap into internal data and uild a etter information ecosystem%

_{Big Data is already an important part of the L3 illion}

dataase and data analytics market

_{$t o/ers commercial opportunities of a comparale}

scale to enterprise software in the late (C4s

_{7nd the $nternet oom of the (CC4s+ and the social media}

uture of Big Data



_{(9 illion on software &rms only}

speciali)ing in data management and

analytics%



 _{#his industry on its own is worth more}

than (44 illion and growing at almost

(4 a year which is roughly twice as fast

as the software usiness as a whole%



_{$n Feruary '4('+ the open source analyst}

&rm 0ikion released the &rst market

forecast for Big Data + listing 9%(B

re-enue in '4(' with growth to 9;%3B in

'4(: