INTRODUCTION
Since the computer age dawned on mankind, one of the most important areas in information technology has been that of “decision support.” Today, this area is more important than ever. Working in dynamic and ever-changing environments, modern-day managers are responsible for an assortment of far reaching decisions: Should the company increase or decrease its workforce? Enter new markets? Develop new products? Invest in research and development?The list goes on. But despite the inherent complexity of these issues and the ever-increasing load of information that business managers must deal with, all these decisions boil down to two fundamental questions:
• What is likely to happen in the future?
• What is the best decision right now?
Whether we realize it or not, these two questions pervade our everyday lives — both on a personal and professional level. When driving to work, for instance, we have to make a traffic prediction before we can choose the quickest driving route. At work, we need to predict the demand for our product before we can decide how much to produce. And before investing in a foreign market, we need to predict future exchange rates and economic variables. It seems that regardless of the decision being made or its complexity, we first need to make a prediction of what is likely to happen in the future, and then make the best decision based on that prediction. This fundamental process underpins the basic premise of Adaptive Business Intelligence.
BACKGROUND
Simply put, Adaptive Business Intelligence is the discipline of combining prediction, optimization, and adaptability into a system capable of answering these two fundamental questions: What is likely to happen in the future? and What is the best decision right now?
(Michalewicz et al. 2007). To build such a system, we first need to understand the methods and techniques that enable prediction, optimization, and adaptability (Dhar and Stein, 1997). At first blush, this subject matter is nothing new, as hundreds of topics and articles have already been written on business intelligence (Vitt et al., 2002; Loshin, 2003), data mining and prediction methods (Weiss and Indurkhya, 1998; Witten and Frank, 2005), forecasting methods (Makridakis et al., 1988), optimization techniques (Deb 2001; Coello et al. 2002; Michalewicz and Fogel, 2004), and so forth. However, none of these has explained how to combine these various technologies into a software system that is capable of predicting, optimizing, and adapting. Adaptive Business Intelligence addresses this very issue.
Clearly, the future of the business intelligence industry lies in systems that can make decisions, rather than tools that produce detailed reports (Loshin 2003). As most business managers now realize, there is a world of difference between having good knowledge and detailed reports, and making smart decisions. Michael Kahn, a technology reporter for Reuters in San Francisco, makes a valid point in the January 16, 2006 story entitled “Business intelligence software looks to future”:
“But analysts say applications that actually answer questions rather than just present mounds of data is the key driver of a market set to grow 10 per cent in 2006 or about twice the rate of the business software industry in general.
‘Increasingly you are seeing applications being developed that will result in some sort of action, ‘said Brendan Barnacle, an analyst at Pacific Crest Equities. ‘It is a relatively small part now,, but it is clearly where the future is. That is the next stage of business intelligence.’”
MAIN FOCUS OF THE CHAPTER
“The answer to myproblem is hidden in my data… but I cannot dig it up!” This popular statement has been around for years as business managers gathered and stored massive amounts of data in the belief that they contain some valuable insight. But business managers eventually discovered that raw data are rarely of any benefit, and that their real value depends on an organization’s ability to analyze them. Hence, the need emerged for software systems capable of retrieving, summarizing, and interpreting data for end-users (Moss and Atre, 2003).
This need fueled the emergence of hundreds of business intelligence companies that specialized in providing software systems and services for extracting knowledge from raw data. These software systems would analyze a company’s operational data and provide knowledge in the form of tables, graphs, pies, charts, and other statistics. For example, a business intelligence report may state that 57% of customers are between the ages of 40 and 50, or that product X sells much better in Florida than in Georgia.1
Consequently, the general goal of most business intelligence systems was to: (1) access data from a variety of different sources; (2) transform these data into information, and then into knowledge; and (3) provide an easy-to-use graphical interface to display this knowledge. In other words, a business intelligence system was responsible for collecting and digesting data, and presenting knowledge in a friendly way (thus enhancing the end-user’s ability to make good decisions). The diagram in Figure 1 illustrates the processes that underpin a traditional business intelligence system.
Although different texts have illustrated the relationship between data and knowledge in different ways (e.g., Davenport and Prusak, 2006; Prusak, 1997; Shortliffe and Cimino, 2006), the commonly accepted distinction between data, information, and knowledge is:
• Data are collected on a daily basis in the form of bits, numbers, symbols, and “objects.”
• Information is “organized data,” which are pre-processed, cleaned, arranged into structures, and stripped of redundancy.
• Knowledge is “integrated information,” which includes facts and relationships that have been perceived, discovered, or learned.
Because knowledge is such an essential component of any decision-making process (as the old saying goes, “Knowledge is power!”), many businesses have viewed knowledge as the final objective. But it seems that knowledge is no longer enough. A business may “know” a lot about its customers — it may have hundreds of charts and graphs that organize its customers by age, preferences, geographical location, and sales history — but management may still be unsure of what decision to make! And here lies the difference between “decision support” and “decision making”: all the knowledge in the world will not guarantee the right or best decision.
Moreover, recent research in psychology indicates that widely held beliefs can actually hamper the decision-making process. For example, common beliefs like “the more knowledge we have, the better our decisions will be,” or “we can distinguish between useful and irrelevant knowledge,” are not supported by empirical evidence. Having more knowledge merely increases our confidence, but it does not improve the accuracy of our decisions. Similarly, people supplied with “good” and “bad” knowledge often have trouble distinguishing between the two, proving that irrelevant knowledge decreases our decision-making effectiveness.
Figure 1. The processes that underpin a traditional business intelligence system
Today, most business managers realize that a gap exists between having the right knowledge and making the right decision. Because this gap affects management’s ability to answer fundamental business questions (such as “What should be done to increase profits? Reduce costs? Or increase market share?”), the future of business intelligence lies in systems that can provide answers and recommendations, rather than mounds of knowledge in the form of reports. The future of business intelligence lies in systems that can make decisions! As a result, there is a new trend emerging in the marketplace called Adaptive Business Intelligence. In addition to performing the role of traditional business intelligence (transforming data into knowledge), Adaptive Business Intelligence also includes the decision-making process, which is based on prediction and optimization as shown in Figure 2.
While business intelligence is often defined as “a broad category of application programs and technologies for gathering, storing, analyzing, and providing access to data,” the term Adaptive Business Intelligence can be defined as “the discipline of using prediction and optimization techniques to build self-learning ‘ decision-ing’ systems” (as the above diagram shows). Adaptive Business Intelligence systems include elements of data mining, predictive modeling, forecasting, optimization, and adaptability, and are used by business managers to make better decisions.
This relatively new approach to business intelligence is capable of recommending the best course of action (based on past data), but it does so in a very special way: AnAdaptive Business Intelligence system incorporates prediction and optimization modules to recommend near-optimal decisions, and an “adaptability module” for improving future recommendations. Such systems can help business managers make decisions that increase efficiency, productivity, and competitiveness. Furthermore, the importance of adaptability cannot be overemphasized. After all, what is the point of using a software system that produces sub par schedules, inaccurate demand forecasts, and inferior logistic plans, time after time? Would it not be wonderful to use a software system that could adapt to changes in the marketplace? A software system that could improve with time?
FUTURE TRENDS
The concept of adaptability is certainly gaining popularity, and not just in the software sector. Adaptability has already been introduced in everything from automatic car transmissions (which adapt their gear-change patterns to a driver’s driving style), to running shoes (which adapt their cushioning level to a runner’s size and stride), to Internet search engines (which adapt their search results to a user’s preferences and prior search history). These products are very appealing for individual consumers, because, despite their mass production, they are capable of adapting to the preferences of each unique owner after some period of time.
The growing popularity of adaptability is also underscored by a recent publication of the US De partment of Defense. This lists 19 important research topics for the next decade and many of them include the term “adaptive”: Adaptive Coordinated Control in the Multi-agent 3D Dynamic Battlefield, Control for Adaptive and Cooperative Systems, Adaptive System Interoperability, Adaptive Materials for Energy-Absorbing Structures, and Complex Adaptive Networks for Cooperative Control.
Figure 2. Adaptive business intelligence system
For sure, adaptability was recognized as important component of intelligence quite some time ago: Alfred Binet (born 1857), French psychologist and inventor of the first usable intelligence test, defined intelligence as “… judgment, otherwise called good sense, practical sense, initiative, the faculty of adapting one’s self to circumstances.” Adaptability is a vital component of any intelligent system, as it is hard to argue that a system is “intelligent” if it does not have the capacity to adapt. For humans, the importance of adaptability is obvious: our ability to adapt was a key element in the evolutionary process. In psychology, a behavior or trait is adaptive when it helps an individual adjust and function well within a changing social environment. In the case of artificial intelligence, consider a chess program capable of beating the world chess master: Should we call this program intelligent? Probably not. We can attribute the program’s performance to its ability to evaluate the current board situation against a multitude of possible “future boards” before selecting the best move. However, because the program cannot learn or adapt to new rules, the program will lose its effectiveness if the rules of the game are changed or modified. Consequently, because the program is incapable of learning or adapting to new rules, the program is not intelligent.
The same holds true for any expert system. No one questions the usefulness of expert systems in some environments (which are usually well defined and static), but expert systems that are incapable of learning and adapting should not be called “intelligent.” Some expert knowledge was programmed in, that is all.
So, what are the future trends for Adaptive Business Intelligence? In words of Jim Goodnight, the CEO of SAS Institute (Collins et al. 2007):
“Until recently, business intelligence was limited to basic query and reporting, and it never really provided that much intelligence …”
However, this is about to change. Keith Collins, the Chief Technology Officer of SAS Institute (Collins et al. 2007) believes that:
“A new platform definition is emerging for business intelligence, where BI is no longer defined as simple query and reporting. [...] In the next five years, we’ll also see a shift in performance management to what we’re calling predictive performance management, where analytics play a huge role in moving us beyond just simple metrics to more powerful measures.”
Further, Jim Davis, the VP Marketing of SAS Institute (Collins et al. 2007) stated:
“In the next three to five years, we’ll reach a tipping point where more organizations will be using BI to focus on how to optimize processes and influence the bottom line .. ”
Finally, it would be important to incorporate adaptability in prediction and optimization components of the future Adaptive Business Intelligence systems.
There are some recent, successful implementations of Adaptive Business Intelligence systems reported (e.g., Michalewicz et al. 2005), which provide daily decision support for large corporations and result in multi-million dollars return on investment.However, further research effort is required. For example, most of the research in machine learning has focused on using historical data to build prediction models. Once the model is built and evaluated, the goal is accomplished. However, because new data arrive at regular intervals, building and evaluating a model isjust the first step inAdaptive Business Intelligence. Because these models need to be updated regularly (something that the adaptability module is responsible for), we expect to see more emphasis on this updating process in machine learning research. Also, the frequency of updating the prediction module, which can vary from seconds (e.g., in real-time currency trading systems), to weeks and months (e.g., in fraud detection systems) may require different techniques and methodologies. In general, Adaptive Business Intelligence systems would include the research results from control theory, statistics, operations research, machine learning, and modern heuristic methods, to name a few. We also expect that major advances will continue to be made in modern optimization techniques. In the years to come, more and more research papers will be published on constrained and multi-objective optimization problems, and on optimization problems set in dynamic environments. This is essential, as most real-world business problems are constrained, multi-objective, and set in a time-changing environment.
CONCLUSION
It is not surprising that the fundamental components of Adaptive Business Intelligence are already emerging in other areas of business. For example, the Six Sigma methodology is a great example of a well-structured, data-driven methodology for eliminating defects, waste, and quality-control problems in many industries. This methodology recommends the sequence of steps shown in Figure 3.
Note that the above sequence is very close “in spirit” to part of the previous diagram, as it describes (in more detail) the adaptability control loop. Clearly, we have to “measure,” “analyze,” and “improve,” as we operate in a dynamic environment, so the process of improvement is continuous. The SAS Institute proposes another methodology, which is more oriented towards data mining activities. Their methodology recommends the sequence of steps shown in Figure 4.
Again, note that the above sequence is very close to another part of our diagram, as it describes (in more detail) the transformation from data to knowledge. It is not surprising that businesses are placing considerable emphasis on these areas, because better decisions usually translate into better financial performance. And better financial performance is what Adaptive Business
Intelligence is all about. Systems based on Adaptive Business Intelligence aim at solving real-world business problems that have complex constraints, are set in time-changing environments, have several (possibly conflicting) objectives, and where the number of possible solutions is too large to enumerate. Solving these problems requires a system that incorporates modules for prediction, optimization, and adaptability.
Figure 3. Six Sigma methodology sequence
Figure 4. SAS Institute recommended methodolgy sequence
TERMS AND DEFINITIONS
Adaptive Business Intelligence: The discipline of using prediction and optimization techniques to build self-learning ‘decisioning’ systems”.
Business Intelligence: A collection of tools, methods, technologies, and processes needed to transform data into actionable knowledge.
Data: Pieces collected on a daily basis in the form of bits, numbers, symbols, and “objects.”
Data Mining: The application of analytical methods and tools to data for the purpose of identifying patterns, relationships, or obtaining systems that perform useful tasks such as classification, prediction, estimation, or affinity grouping.
Information: “Organized data,” which are prepro-cessed, cleaned, arranged into structures, and stripped of redundancy.
Knowledge: “Integrated information,” which includes facts and relationships that have been perceived, discovered, or learned.
Optimization: Process of finding the solution that is the best fit to the available resources.
Prediction: A statement or claim that a particular event will occur in the future.
