Databases Reference
In-Depth Information
Often, a data analysts use industry accepted analytics products like SAS, R, SPSS, SQL
Server Analytics Services, Oracle Data Miner, Microsoft Excel, or other analytics tools.
Design for Big Data Scale
Once the prototype results are evaluated, found to be of high quality, and meet the
business expectations, the next step is to build an analytics system that will handle
the scale of big data challenges with desired performance levels. Not every analytical
method that works with small amounts of data will scale to big data problems. Therefore,
the analyst must choose a method that will scale to big data production size. There are
several considerations in designing an analytics system for production readiness:
•
Complexity:
How complex are the data sets, and how complex
are the analytics algorithms? Do they require special provisioning
and skills to manage?
•
Efficiency:
How efficient is the analytical model? Does it require
specialized s/w and h/w configurations?
•
Performance:
How capable is the analytics algorithm of
running across big data dimensions? Does it require in-database
processing? Does it require in-memory processing? Does it have
the ability to run massively parallel processing across huge data
sets and grid architectures?
•
Reliability and Accuracy:
How calibrated is the analytics output?
What are the confidence intervals and error measures? Does it
throw false positives and false negatives within an acceptable
range?
•
Coverage and Reach:
Does the system have the ability to cover
all data types and to do depth-search as well as breadth-search
across several data dimensions?
•
Flexibility:
How flexible is the analytics system in adopting new
algorithms?
