DEPLOYING AND MONITORING HADOOP MAP-REDUCE ANALYTICS ON SINGLE-CHIP CLOUD COMPUTER

(1)

DEPLOYING AND MONITORING

HADOOP MAP-REDUCE ANALYTICS

ON SINGLE-CHIP CLOUD COMPUTER

ANDREAS-LAZAROS GEORGIADIS, SOTIRIOS XYDIS, DIMITRIOS SOUDRIS

MICROPROCESSOR AND MICROSYSTEMS LABORATORY ELECTRICAL AND

COMPUTER ENGINEERING DEPARTMENT

NATIONAL TECHNICAL UNIVERSITY OF ATHENS

(2)

TIME IS … BYTES

(3)

NEW OPEN QUESTIONS FOR

COMPUTER ARCHITECTS

300 Exabyte @ 2000 8500 Exabytes @ 2015 Single server @ 2000 Data-centers @ 2015 Single-core Pentium-Pro@ 2003

50-core Intel Phi @ 2015

DATA SCALE UP

WORKLOADS SCALE OUT

TECHNOLOGY SCALE DOWN

How Big Data and

Scale-Out Workloads

Performs on Manycores

?

(4)

SCOPE OF THIS PAPER

•

HADOOP ANALYTICS ON CHIP

: HOW TO DEPLOY & MONITOR HADOOP MAPREDUCE CLUSTERS

INTEL SINGLE-CLOUD-CHIP (SCC) MANYCORE

•

WORKLOAD CHARACTERIZATION

: ANALYSIS OF HADOOP ANALYTIC WORKLOADS ON

REAL-SILICON INTEL-SCC MANYCORE

•

PERFORMANCE-POWER TUNING

: HADOOP CONFIGURATIONS FOR EFFICIENT

PERFORMANCE-POWER TRADE-OFFS W.R.T. CLUSTER TOPOLOGIES AND FREQUENCY SETTINGS

(5)

INTEL SCC: ARCHITECTURAL SPECIFICATION

•

RESEARCH CHIP BUILT IN INTEL LABS. 48 P54C IA CORES ORGANIZED

IN 24 TILES. CORE FREQUENCY FROM 100 MHZ TO 800 MHZ.

•

ON-DIE 2D MESH NETWORK. 24 PACKET-SWITCHED ROUTERS. MESH

NETWORK FREQUENCY 800 MHZ OR 1.6 GHZ.

•

32 GB OF DRAM THROUGH 4 DDR3 MEMORY CONTROLLERS.

MEMORY CONTROLLER FREQUENCY 800 MHZ OR 1066 MHZ.

•

16 KB OF FAST LOCAL SRAM ON EACH TILE, CALLED THE MESSAGE

PASSING BUFFER (MPB).

•

BOARD MANAGEMENT MICROCONTROLLER (BMC). INITIALIZES AND

SHUTS DOWN CRITICAL SYSTEM FUNCTIONS.

•

CONNECTED TO A MANAGEMENT CONTROL PC (MCPC) BY A

PCI-EXPRESS CABLE.

Intel SCC Power Management

Andreas - Lazaros Georgiadis (NT UA) Diploma T hesis April 27, 2015 17 / 56 PARMA-DITAM 2016

(6)

HADOOP CLUSTER: HDFS + MAPREDUCE

•

HDFS

:

• DISTRIBUTED FILE SYSTEM [NAMENODE, DATANODES]  EFFICIENT AND RELIABLE ACCESS TO DATA

• NAMENODE: MANAGES FILE SYSTEM NAMESPACE AND REGULATES ACCESS TO FILES BY CLIENTS.

• DATANODE: MANAGES STORAGE ATTACHED TO THE NODES THAT THEY RUN ON. BLOCK CREATION, DELETION, AND REPLICATION UPON INSTRUCTION FROM THE NAMENODE.

•

MAPREDUCE

:

• SCALABLE PARALLLEL PROGRAMMING MODEL

• MAP TASKS PROCESS INDEPENDENT SPITS OF INPUT DATA AS <KEY,VALUE> PAIRS TO GENERATE A SET OF INTERMEDIATE <KEY, VALUE> PAIRS.

• REDUCE TASKS MERGE ALL INTERMEDIATE VALUES ASSOCIATED WITH THE SAME INTERMEDIATE KEY, SO AS TO PRODUCE THE FINAL OUTPUT <KEY, VALUE> PAIRS.

• INPUT AND THE OUTPUT FILES ARE STORED IN HDFS. TASK SCHEDULING WHERE THE DATA IS ALREADY PRESENT, VERY HIGH AGGREGATE BANDWIDTH ACROSS THE CLUSTER.

(7)

LIMITATIONS

•

RESTRICTED APPLICATION DEVELOPMENT

API OF INTEL SCC LINUX

•

ONLY 640 MB OF MAIN MEMORY FOR

EACH CORE.

•

LIMITED TCP/IP STACK TO SUPPORT

CLUSTER SW

SOLUTION

•

GENTOO IMAGE FOR THE INTEL SCC ON

EACH INTEL SCC CORE.

•

JAVA HEAP SPACE OF 128 MB FOR HADOOP

DAEMONS AND THE CHILD JVM

•

NAT ROUTING WITH MODIFIED ROUTING

TABLES

•

INTERNET ACCESS FOR INTEL SCC CORES

•

DIRECT ACCESS TO INTERNAL VIRTUAL

NETWORK INTERFACES OF THE INTEL SCC

CORES (MB0)

HADOOP DEPLOYMENT OF INTEL SCC (1/2)

(8)

LIMITATIONS

•

HIGH I/O LOAD AND VERY LOW FREE

MAIN MEMORY SPACE CAUSES CORES TO

FREEZE AND BECOME UNREACHABLE

FREQUENTLY.

•

HADOOP CONSIDERS CONSIDERS RACK

PROXIMITY

•

NO EFFICIENT FRAMEWORK RUNTIME

MONITORING

SOLUTION

•

NODE-FAILOVER WATCHDOG. PINGS INTEL

SCC CORES PERIODICALLY. IF CORE

UNREACHABLE, INTEL SCC LINUX IS

BOOTED AND CORRESPONDING HADOOP

DAEMON IS STARTED

•

ON-DIE CLUSTER EXPLICITLY DIVIDED TO

HADOOP RACKS.

•

ADAPT GANGLIA MONITORING ON INTEL

SCC

HADOOP DEPLOYMENT OF INTEL SCC (2/2)

(9)

HADOOP CLUSTER TOPOLOGY EXPLORATION (1/4)

16-node

Hadoop cluster

(10)

24-node

Hadoop cluster

(11)

32-node

Hadoop cluster

(12)

48-node

Hadoop cluster

(13)

EXPERIMENTAL PROCESS

•

THE PERFORMANCE OF FOUR MAPREDUCE APPLICATIONS (WORDCOUNT, BAYES

CLASSIFICATION [COUDSUITE], K-MEANS CLUSTERING AND FREQUENT PATTERN

GROWTH[DATACENTERBENCH]) IS INVESTIGATED WHEN THEY ARE EXECUTED ON THE INTEL

SCC

•

EXPERIMENTAL ANALYSIS EXPLORES SCALABILITY, PERFORMANCE AND POWER CONSUMPTION

TRADEOFFS FOR DIFFERENT CLUSTER TOPOLOGY ORGANIZATIONS AND FREQUENCY

CONFIGURATIONS.

(14)

WORKLOAD CHARACTERIZATION:

BAYES CLASSIFIER (1/3)

•

DCSDCSC

Bayes Classiﬁcation: Application Execution Overview (1/ 3)

Andreas - Lazaros Georgiadis (NT UA) Diploma T hesis April 27, 2015 41 / 56

(15)

•

DCSDCSC

Bayes Classiﬁcation: Application Execution Overview (2/ 3)

(16)

•

DCSDCSC

Bayes Classiﬁcation: Application Execution Overview (3/ 3)

(17)

IMPACT OF RESOURCE ALLOCATION ON

PERFORMANCE - ENERGY

Wordcount: Experimental Analysis (1/ 2)

Andreas - Lazaros Georgiadis ( NT UA) Diploma T hesis April 27, 2015 38 / 56

(a) WordCount

Bayes Classiﬁcation: Experimental Analysis (1/ 2)

Andreas - Lazaros Georgiadis ( NT UA) Diploma T hesis April 27, 2015 44 / 56

(b) BayesClassiﬁer

K-Means Clustering: Experimental Analysis (1/ 2) [More...]

More...

(c) K-Means

Frequent Pattern Growth: Experimental Analysis (1/ 2)

[More...]

(d) Frequent-Pattern Groth

Figur e 3: Per for mance-Ener gy char acter ization of M apReduce wor kloads on I ntel SCC Hadoop cluster topologies. (ISSCC), 2010 IEEE International, pages108–109, Feb

2010.

[3] Jeffrey Dean and Sanjay Ghemawat. Mapreduce: Simpliﬁed data processing on large clusters. Commun. ACM,

51(1):107–113, January 2008.

[4] Jens Dittrich and Jorge-Arnulfo Quiané-Ruiz. Efﬁcient big data processing in hadoop mapreduce. Proc. VLDB Endow., 5(12):2014–2015, August 2012.

[5] Jacob Leverich and Christos Kozyrakis. On the energy (in)efﬁciency of hadoop clusters. SIGOPS Oper. Syst. Rev., 44(1):61–65, March 2010.

[6] Bingsheng He, Wenbin Fang, Qiong Luo, Naga K. Govindaraju, and Tuyong Wang. Mars: A mapreduce framework on graphics processors. InProceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques, PACT ’08, pages 260–269, New York, NY, USA, 2008. ACM.

[7] Miao Xin and Hao Li. An implementation of gpu accelerated mapreduce: Using hadoop with opencl for data- and

compute-intensive jobs. InService Sciences (IJCSS), 2012 International Joint Conference on, pages6–11, May 2012. [8] Nikos Hardavellas, Ippokratis Pandis, Ryan Johnson, Naju

Mancheril, Anastassia Ailamaki, and Babak Falsaﬁ.

Database servers on chip multiprocessors: Limitations and opportunities. InCIDR 2007, Third Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA,

January 7-10, 2007, Online Proceedings, pages79–87, 2007. [9] B. Wheeler. Tilera sees opening in clouds. Microprocessor

Report,25(7):13–16, March 2011.

[10] Kevin Lim, Parthasarathy Ranganathan, Jichuan Chang, Chandrakant Patel, Trevor Mudge, and Steven Reinhardt. Understanding and designing new server architectures for emerging warehouse-computing environments. In

Proceedings of the 35th Annual International Symposium on Computer Architecture, ISCA ’08, pages 315–326,

Washington, DC, USA, 2008. IEEE Computer Society.

[11] Philipp Gschwandtner, Thomas Fahringer, and Radu Prodan. Performance analysis and benchmarking of the intel scc. In Proceedings of the 2011 IEEE International Conference on Cluster Computing, CLUSTER’11, pages 139–149,

Washington, DC, USA, 2011.

[12] John-Nicholas Furst and Ayse K. Coskun. Performance and power analysis of rcce message passing on the intel single chip cloud computer. InProceedings of the 4th Many-core Applications Research Community (MARC) Symposium, MARC’11, 2011.

[13] Andrea Bartolini, MohammadSadegh Sadri, John-Nicholas Furst, Ayse Kivilcim Coskun, and Luca Benini. Quantifying the impact of frequency scaling on the energy efﬁciency of the single-chip cloud computer. InProceedings of the Conference on Design, Automation and Test in Europe, DATE’12, pages 181–186, San Jose, CA, USA, 2012. EDA Consortium.

[14] Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. The hadoop distributedﬁle system. In Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), MSST ’10, pages 1–10, Washington, DC, USA, 2010. IEEE Computer Society. [15] Hadoop: Open source implementation of mapreduce.

http://hadoop.apache.org/.

[16] Peter Troger Jan-Arne Sobania. Gentoo linux on intel scc. operating systems and middleware group, it systems

engineering, university potsdam, 2010

https://www.dcl.hpi.uni-potsdam.de/research/scc/gentoo.htm.

[17] Ganglia monitoring system: http://ganglia.sourceforge.net. [18] Cloudsuite, a benchmark suite for scale-out applications.

http://parsa.epﬂ.ch/cloudsuite/cloudsuite.html.

[19] Dcbench, a benchmark suite for data center workloads. ict, chinese academy of sciences. http://prof.ict.ac.cn/dcbench/.

(18)

CORE-FREQUENCY ASSIGNMENT IN HADOOP CLUSTER

•

HIGH PERFORMANCE REGION:

•

TASKTRACKER: 800 MHZ

•

DATANODE: 200 – 800 MHZ

•

LOW ENERGY REGION:

•

TASKTRACKER: 800MHZ

•

DATANODE: 200 MHZ

(19)

CONCLUSIONS

•

HADOOP MAPREDUCE WORKLOADS DEPLOYED AND MONITORED ON THE INTEL SCC CORES.

•

PLATFORM LIMITATIONS AND SOLUTIONS

•

EXTENSIVE EXPERIMENTATION REGARDING TO THE ANALYSIS OF HADOOP MAPREDUCE

WORKLOADS OVER DIVERSE CLUSTER TOPOLOGIES

•

INTERESTING PERFORMANCE-ENERGY TRADE-OFFS IN RESPECT TO CORE- FREQUENCY

ALLOCATION STRATEGY OF THE DATA-NODES VS. TASKTRACKERS

(20)

•

THANK YOU

•

QUESTIONS

?

PARMA-DITAM 2016