Age of Big data
Presented by:
Mohammad Iqbal
Agenda
What is a Big Data ?
Big Data Attributes
Big data Sources
Getting Value from Big data
New Tools for Big Data
Hadoops' Architecture
Hadoop evolution from Google
The future is here!
What is a Big Data ?
Name Symbol Value
Kilobyte KB 10^3
Megabyte MB 10^6
Gigabyte GB 10^9
Terabyte TB 10^12
Petabyte PB 10^15
Exabyte EB 10^18
Zettabyte ZB 10^21
Yottabyte YB 10^24
BIG DATA
So large data that it becomes
difficult to process it using the
traditional system
Getting New Tools Hadoop
Difficult to process by Traditional
System
100 MB
document
100 TB document
100 GB
document
Unable to
send
Unable
to Edit
Unable
to View
Depends on
capability of
system
Organization/Context Specific
500 TB Text,Audio,Video
data per day
Company A
Company B
Big
Date
NOT
a Big
data
Depends on
capabilities
of the
organization
Getting New Tools Hadoop
Areas of Challenges
Capture
Curation
Storage
Anlaysis Visualization
Transfer
Sharing
search
Big Data Attributes
Big
Data • Large & growing files
• At High speed
• In various Format
VELOCITY VOLUME VARIETY
Data
comes at
high speed
This files
comes in
various
formats
Data
result in
large file
V^3
Structured / Unstructured
Unstructured Data
90%
Structured
Data 10%
Challenge
/Opportunity Mostly
wasted
Used in
decision
making
To analyze & extract
meaningful information
Big data Sources
Users
Applications
Systems
Sensors
Large & growing
files
(Big data files)
Data Generation point Examples
Mobile devices
Microphones
Readers/Scanners
Software/program
Social Media
cameras
Machine Sensors
Science facilities
Sample Events generating Data
• Every day, we create 2.5 Exabytes of data i.e 2.5 billion GB, so much that
90% of the data in the world today has been created in the last few years
alone.
• CERN Atomic facility generates 40 TB data per second.
• Twitter generates 12 TB of data every day.
• Airbus A380 generates 10 TB every 30 minutes of flight. About 650TB
generated in one flight.
• In 2009 total data in world was estimated to be 1 ZB. By 2020 estimated to
be 35 ZB .
(Source :IBM.com)
Getting Value from Big data
Collect Analyze Understand
Big data Applications
• Companies gaining edge by collecting
,analyzing and understanding information.
• Government forecasting events and taking
proactive actions.
New Tools for Big Data
Traditional
Systems
(e.g RDBMS
,SQL)
Big data tool
(e.g Hadoop
NoSQL)
Time
Not able to
handle Big
data
Created to
handle big data
Traditional Enterprise Approach
Big data Processing Limit Computer Powerful
Only So much
data could be
processed
Modern Hadoop’s approach
Big data
Combined result Computation
Computation
Computation
Computation
Hadoops’s Architecture
Source :hortonworks/hadoop/hdfs/.com/
