dashboards with key performance indicators from these repositories
to assist with business decisions.
More recently, the concept of BI is being expanded to include data
mining techniques to extract knowledge and generate predictions for
business problems, thereby enabling companies to make better use of
a costly corporate asset, their data. With the advent of fast and inex-
pensive hardware, the ability to mine large volumes of data is not
only possible, but the building of models and the scoring of data can
often be performed in real time. Advances in data mining techniques
and Moore's Law [Webopedia 2006] help to ensure that businesses
and researchers will be able to keep pace with data repositories dou-
bling at about the same rate as hardware performance.
On the other side of the spectrum are the marketers, financial
analysts, and even call center representatives—generally application
users—who leverage data mining increasingly through intelligent
applications. These users either know nothing about the techniques
of data mining or do not need to know anything about data mining
to reap its benefits. The algorithms and the data mining process are
concealed by business-centric user interfaces, which present the
results of data mining, not its mechanics. For many, this is the only
way to take advantage of the benefits of data mining—through
verticalized solution applications.
Data mining is also relevant today as the technology becomes more
accessible to a broader audience. Traditionally, data mining has been
the realm of statisticians, data analysts, and scientists with Ph.D.'s in
machine learning. These users wrote their own algorithms and graph-
ical tools, leveraged complex commercial graphical user interfaces and
application programming interfaces. Their process for mining data
was often ad hoc, stitching together analytic workflows using Perl
[Perl 2006], AWK, Python, Tcl/Tk, among others [SoftPanorama 2006].
With modern advances in data mining automation technology
and the introduction of standard interfaces and processes, data
mining is being made accessible to a new class of users: application
architects and designers, and mainstream developers. Although the
role of the data analyst will likely always be in fashion for superior
results, technology is at the point where nonexperts can get good
results. Here, good is defined as results that could be achieved by a
junior statistician and sometimes much better. This empowers archi-
tects, designers, and developers to experiment with mining the data
available to their applications and to enhance those applications with
predictive analysis, presenting new insights to application users.