Information Technology Reference
In-Depth Information
9
RPig: Concise Programming Framework by
Integrating R with Pig for Big Data Analytics
MingXue Wang and Sidath B. Handurukande
CONTENTS
Summary .............................................................................................................. 194
9.1 Introduction ................................................................................................ 194
9.2 Motivating Scenarios ................................................................................. 196
9.2.1 Intensive Scenario with Both Input/Output and
Central Processing Unit with Exponential Moving Average .... 196
9.2.2 A CPU-Intensive Scenario with SVM ......................................... 196
9.3 Background ................................................................................................. 197
9.3.1 R and R Packages ........................................................................... 197
9.3.2 Hadoop and MapReduce .............................................................. 198
9.3.3 Pig and Pig Latin ............................................................................ 198
9.4 The Framework .......................................................................................... 199
9.4.1 The R Script Engine Extension..................................................... 200
9.4.2 Data-Type Conversions ................................................................. 201
9.4.2.1 From Pig to R ................................................................... 201
9.4.2.2 From R to Pig ................................................................... 201
9.4.3 Execution and Monitor .................................................................. 202
9.4.4 Implementation .............................................................................. 203
9.5 Use Case and Experiment......................................................................... 203
9.5.1 Summary Statistics with Quantiles ............................................ 204
9.5.1.1 Design and Implementation .......................................... 204
9.5.1.2 Result and Discussion .................................................... 205
9.5.2 Forecasting with EMA .................................................................. 206
9.5.2.1 Design and Implementation .......................................... 206
9.5.2.2 Result and Discussion .................................................... 208
9.5.3 Prediction with SVM ..................................................................... 209
9.5.3.1 Design and Implementation .......................................... 209
9.5.3.2 Result and Discussion .................................................... 212
9.6 Related Work .............................................................................................. 213
9.6.1 Related to R ..................................................................................... 213
9.6.2 Other Related Solutions ................................................................ 214
9.7 Conclusion .................................................................................................. 215
References ............................................................................................................. 215
193
Search WWH ::




Custom Search