Large-Scale Network Traffic Analysis for Estimating the Size of IP Addresses and Detecting Traffic Anomalies - Large Scale and Big Data: Processing and Management - page 434

14 Large-Scale Network

Traffic Analysis for
Estimating the Size of IP
Addresses and Detecting
Traffic Anomalies

Ahmed Metwally, Fabio Soldo,
Matt Paduano, and Meenal Chhabra

CONTENTS
14.1 Introduction .................................................................................................. 436
14.2 IP Size: Challenges and Approach ............................................................... 438
14.2.1 Estimation Challenges and Methodology ......................................... 438
14.2.2 The Size Estimation Cycle ................................................................ 439
14.3 IP Size Estimation ........................................................................................ 4 41
14.3.1 The Learning Models ....................................................................... 443
14.3.2 Gauging Estimation Accuracy .......................................................... 443
14.4 IP Size Prediction ......................................................................................... 445
14.4.1 The Size Prediction Alternative Approaches ................................... 445
14.4.2 Considering Multiple Size Periodicities ...........................................446
14.4.3 Iterative Variance Reduction ............................................................446
14.4.4 The PredictSizes Algorithm ............................................................. 447
14.4.5 Evaluating Predictions ...................................................................... 447
14.4.5.1 Prediction Accuracy ........................................................... 447
14.4.5.2 Predictions Coverage .........................................................448
14.5 Detecting Machine-Generated Attacks ........................................................ 449
14.5.1 Observed IP Size Distributions ........................................................ 449
14.5.2 Machine-Generated Attacks and IP Size Distributions ................... 450
14.5.3 The Data Set ..................................................................................... 450
14.5.3.1 Assessing the Quality of Traffic ........................................ 450
14.5.4 Click Filtering ................................................................................... 452
14.5.4.1 IP Size Histogram Filter Overview .................................... 452
14.5.4.2 Grouping Publishers .......................................................... 452
14.5.4.3 Threshold Model for Legitimate Click Traffic .................. 452

435