Massive scale analytics with Stratosphere using R

(1)

Database Systems and Information Management Technische Universit¨at Berlin

Supervised by Volker Markl Advised by Marcus Leich, Kostas Tzoumas

(2)

Introduction

(3)

Data analysis to the masses

I Deep analytics1: sophisticated statistical methods like linear models, clustering or classification that frequently are used to extract knowledge from the data.

Data warehousing and BI can’t answer all the questions.

The ever-growing number of new data sources and tools make it worse.

I There is demand for this questions.

I In small scale: data pipelining tools (RapidMiner) and numerical computing environments (R, Matlab or SPSS). I Big data brings new opportunities to the market but also

presents unfamiliar challenges.

(4)

Options

I R:

R is a numerical computing environment and DSL for stats.

Not a query language unlike SQL.

Succesful for small scale (in combination with CRAN packages).

I MapReduce/Hadoop:

Highly parallel programs but lack of expressivity.

HDFS: a de-facto standard to store big amounts of data. I Stratosphere:

Platform for massively parallel computing / big data analytics.

PACT: MapReduce + New operators + Iterations.

(5)

Basic terms and definitions

I KDD is compound of nine steps: understanding the domain and the goals, creating the target source, cleaning and

processing the source, data reduction and projection, choosing a data mining method, choosing the data mining algorithm, mining the data, interpretation of the patterns.

(6)

Motivation

(7)

(8)

Classification

(9)

(10)

Writing massively parallel programs

I It is a cumbersome and onerous process. I We need of single tools.

I We need tools that can process from a small amount of data up to very large volumes.

I The majority of data researchers are strongly skilled in R and statistics and poorly skills in Big Data systems and

implementation of machine learning algorithm.3 4 I Although Stratosphere offers a more expressive interface,

writing a parallel program is still not a trivial job.

3

Harlan Harris, Sean Murphy, and Marck Vaisman.Analyzing the Analyzers: An Introspective Survey of Data Scientists and Their Work. O’Reilly Media, Inc., 2013

4_{Sean Kandel, Andreas Paepcke, Joseph M Hellerstein, and Jeffrey Heer.}

Enterprise data analysis and visualization: An interview study.Visualization and Computer Graphics, IEEE Transactions on, 18(12):2917–2926, 2012

(11)

Relation with the KDD process

I Data extraction is covered by other solutions. I Pre-processing and transformation seem difficult. I Data mining: where we have a competitive advantage. I Data visualization is a different problem.

(12)

Design goals

I Easiness: ready-to-use algorithms. I Design a library.

I Facilitate working with data. I Easy to distribute.

I Focus on algorithms that scale.

(13)

(14)

Architecture

(15)

(16)

Library: Goals

I Classification, clustering and regression.

I No Free Lunch Theorem: more than one algorithm. I Presence in other ML libraries.

I Large-scale.

I Ensemble scenarios.

(17)

(18)

R package

I Easy to distribute.

I Organized in namespaces. I Submitting jobs to the cluster. I Working with files.

I Mining. I Configuration.

(19)

(20)

(21)

(22)

(23)

Performance

I Competitive and even faster than native R programs thanks to the pipelining for every parallelizable programs in the same (small) file size range.

I Competitive with R for data mining tasks with a lot of iterations in the same file size range.

I Able to process files of a volume that is inaccessible for R. I Able to scale to gigabyte level without significant loss.

(24)

Performance: Frequent Terms example

(25)

(26)

Performance: Breakdown example

Figure: Clustering example nonparallel breakdown (Time in seconds)

(27)

(28)

Related work

(29)

Data mining libraries

I Don’t scale: Weka and sci-kit. I Large-scale:.

Mahout: limited set of problems.

MLlib: also facilitates implementation of new algorithms.

Oryx.

(30)

Data intensive computation with R

I External memory.

Don’t scale-out: biglm, bigmemory, ff, foreach.

RevoScaleR: xdf files and Hadoop.

I Divide and recombine: it’s necessary to use the MR model. I Query languages:

Limited expressivity.

Good for the first step of the KDD process. I Distributed collection manipulation:

_{Limited set of operators.} Presto andSparkR.

(31)

(32)

Conclusion

I Contributions:. Library definition.

File manipulation and cluster interaction.

Scenarios that proof the concept. I Code very similar to the original one. I Promising performance evaluation.

(33)

Future work

I Improvements in the library. I Hybrid approaches.

I Distributed evaluation.

(34)

I Sudipto Das, Yannis Sismanis, Kevin S Beyer, Rainer Gemulla, Peter J Haas, and John McPherson. Ricardo: integrating r and hadoop. InProceedings of the 2010 ACM SIGMOD International Conference on Management of data, pages 987–998. ACM, 2010 I Hadley Wickham.Advanced R Programming. CRC Press, 2014. To

appear

I Alexander Alexandrov, Rico Bergmann, Stephan Ewen,

Johann-Christoph Freytag, Fabian Hueske, Arvid Heise, Odej Kao, Marcus Leich, Ulf Leser, Volker Markl, Felix Naumann, Mathias Peters, Astrid Rheinlnder, MatthiasJ. Sax, Sebastian Schelter, Mareike Hger, Kostas Tzoumas, and Daniel Warneke. The stratosphere platform for big data analytics.The VLDB Journal, pages 1–26, 2014

I Hai Qian. Pivotalr: A package for machine learning on big data.The R Journal, 6(1):57–67, June 2014

I Usama Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth. The kdd process for extracting useful knowledge from volumes of data.Commun. ACM, 39(11):27–34, November 1996

(35)

1 Introduction

Data analysis to the masses

Options

Basic terms and definitions

2 Motivation

Motivating problems Writing massively parallel programs

Relation with the KDD process Design goals 3 Our approach Architecture Library R package Example Performance 4 Related work

Data mining libraries Data intensive computation with R

5 Conclusions and Future Work Conclusion

Future work