# Not a part of 1Z0-061 or 1Z0-144 Certification test, but very important technology in BIG DATA Analysis

(1)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

1

Section 9 : Case Study ^#

Objectives of this Session

The Motivation For Hadoop

What problems exist with traditional large-scale computing systems

What requirements an alternative approach should have

How Hadoop addresses those requirements

Hadoop: Basic Concepts

What Is Hadoop?

The Hadoop Distributed File System (HDFS)

How Google MapReduce Algorithm works

Anatomy of a Hadoop Cluster

Who uses Hadoop ?

db.suven.net

# Not a part of 1Z0-061 or 1Z0-144 Certification test , but very important technology in BIG DATA Analysis

(2)

• Hadoop Solutions

– The most common problems Hadoop can solve

– The types of analytics often performed with Hadoop

– Where the data comes from ?

– The benefits of analyzing data with Hadoop

– How some real-world companies use Hadoop

• Hadoop Ecosystem

• Cloudera Software (All Open-Source)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

²

Objectives of this Session … contd…

(3)

The Motivation For Hadoop

compiled by Rocky Jagtiani Tech Head for 3 SCTPL , 9892544177

(4)

*

MPI: Message Passing Interface

PVM: Parallel Virtual Machine

(5)

Major

Problem

(6)

1 GB = 1000 MB , 1 TB = 1000 GB , 1 PT = 1000 TB , 1 Exabyte = 1000 PT

PT => petabyte , TB => teraByte

(7)

(8)

The Motivation For Hadoop

(9)

1.

2.

(10)

3.

4.

5.

10 compiled by Rocky Jagtiani Tech Head for

SCTPL , 9892544177

(11)

Hadoop History

(12)

Core Hadoop Concepts

(13)

Hadoop Components

(14)

HDFS

(15)

HDFS

Concepts

(16)

HDFS : How Files Are Stored ?

(17)

How Files Are Stored: Example

(18)

IMP :

How MapReduce Work ?

(19)

MapReduce: The Mapper

19 compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

(20)

Example :

(21)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177 21

(22)

(23)

(24)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

²⁴

(25)

compiled by Rocky Jagtiani Tech Head for

SCTPL , 9892544177 25

Anatomy of a Hadoop Cluster :

(26)

(27)

(28)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

²⁸

Who uses Hadoop ?

(29)

SCTPL , 9892544177 29

Hadoop Solutions

(30)

A

(31)

SCTPL , 9892544177 31

B What is Problem if the data is coming ?

(32)

SCTPL , 9892544177 32

C

(33)

The most common problems Hadoop can solve :

We understand how each problem is solved using Hadoop in brief

D

(34)

(35)

SCTPL , 9892544177 35

(36)

(37)

(38)

(39)

(40)

SCTPL , 9892544177 40

(41)

SCTPL , 9892544177 41

(42)

SCTPL , 9892544177 42

How some real-world companies use Hadoop

E

(43)

SCTPL , 9892544177 43

Hadoop Ecosystem

(44)

Cloudera Software (All Open-Source)

(45)

SCTPL , 9892544177 45

*enterprise data

warehouse (EDW)

Conclusion :

(46)

1) Input to mapper is

"Google is one of the richest companies "

"one who works with the Google is technical expert "

what will be the out put after reducing ?

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

46 Questions

(47)

2) Input to mapper is

"Cat is eating milk"

"Cat is very sweet and she likes milk"

"milk is in bottle"

what will be the out put after reducing ?

47

(48)

3) Input to mapper is

"Dollar is national currency for USA"

"Rupee is national currency for India"

"Dollar is ahead of Rupee in economy"

"India is developing country"

what will be the out put after Mapping ?

48

(49)

# Not a part of 1Z0-061 or 1Z0-144 Certification test, but very important technology in BIG DATA Analysis

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Section 9 : Case Study #

Objectives of this Session

The Motivation For Hadoop

What problems exist with traditional large-scale computing systems

What requirements an alternative approach should have

How Hadoop addresses those requirements

Hadoop: Basic Concepts

What Is Hadoop?

The Hadoop Distributed File System (HDFS)

How Google MapReduce Algorithm works

Anatomy of a Hadoop Cluster

Who uses Hadoop ?

db.suven.net

# Not a part of 1Z0-061 or 1Z0-144 Certification test , but very important technology in BIG DATA Analysis

• Hadoop Solutions

– The most common problems Hadoop can solve

– The types of analytics often performed with Hadoop

– Where the data comes from ?

– The benefits of analyzing data with Hadoop

– How some real-world companies use Hadoop

• Hadoop Ecosystem

• Cloudera Software (All Open-Source)

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Objectives of this Session … contd…

The Motivation For Hadoop

*

MPI: Message Passing Interface

PVM: Parallel Virtual Machine

Major

Problem

1 GB = 1000 MB , 1 TB = 1000 GB , 1 PT = 1000 TB , 1 Exabyte = 1000 PT

PT => petabyte , TB => teraByte

The Motivation For Hadoop

1.

2.

3.

4.

5.

Hadoop History

Core Hadoop Concepts

Hadoop Components

HDFS

HDFS

Concepts

HDFS : How Files Are Stored ?

How Files Are Stored: Example

IMP :

How MapReduce Work ?

MapReduce: The Mapper

Example :

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Anatomy of a Hadoop Cluster :

compiled by Rocky Jagtiani Tech Head for SCTPL , 9892544177

Who uses Hadoop ?

Hadoop Solutions

A

B What is Problem if the data is coming ?

C

The most common problems Hadoop can solve :

We understand how each problem is solved using Hadoop in brief

D

How some real-world companies use Hadoop

E

Hadoop Ecosystem

Cloudera Software (All Open-Source)

*enterprise data

warehouse (EDW)

Conclusion :

1) Input to mapper is

"Google is one of the richest companies "

"one who works with the Google is technical expert "

what will be the out put after reducing ?

46

Questions

2) Input to mapper is

"Cat is eating milk"

"Cat is very sweet and she likes milk"

"milk is in bottle"

Section 9 : Case Study ^#