INTRODUCTION
After more than 30 years of research on how the work of managers can be supported by computers, the observation that developing computer systems that are truly useful for top management is a highly complex and uncertain task is still as valid as ever. Information systems for executives raise specific problems, which have primarily to do with the nature of managerial work itself (Mintzberg, 1973), as they are intended to tackle the needs of users whose most important role is “to create a vision of the future of the company and to lead the company towards it” (King, 1985, p. xi).
BACKGROUND
The major difficulty in supporting managers with computer systems comes from the very nature of management work (Mintzberg, 1973, 1975, 1976), which is concerned with communication, coordination, and people’s management for more than 80%. At the time of his research, Mintzberg (1973) had noted how little time is left for reflection and for “playing” with computer systems. This has been a significant difficulty from the origins of MIS systems because their primarily “operational” focus was not central to executives’ concerns (Ackoff, 1967; Keen & Scott Morton, 1978). Twenty years later, this difficulty has also been largely responsible for the shift from decision support systems (DSSs) to executive information systems (EISs). EISs were intended to be very easy to use and to help users manipulate required data without the need for much training, which would be very attractive to top executives who want to have, at a glance, a very comprehensive view of their business. Specific descriptions of the differences between DSSs, EISs, and cooperative decision systems can be found in Pomerol and Brezillon (1998). Naturally, computer literacy among executives has increased to a great extent, notably thanks to the development of electronic mail and the World Wide Web. However, whatever designs were put forward over the years, it has remained true that managers are not inclined to spend countless hours browsing computer data, such is the time pressure under which they operate.
Beyond the time pressures under which executives must operate, there are issues of trust and of credibility of the information that can be found in a computer system, which mitigate against intensive executive reliance on information systems, especially in a long-term perspective. First of all, the lack of confidence of executives in their models has been noted by many researchers (e.g., Wallenius, 1975; Cats-Baril & Huber, 1987; Abualsamh, Carlin & McDaniel, 1990). The idea that decision makers need sophisticated models may actually be wrong. People in charge of the preparation of decisions would probably be able to understand and use smart models, but the high-level executives who most commonly make the final decisions are far too busy to train with and use involved systems. On the contrary, they appear to prefer simple systems that they trust and understand, and that display very timely simple information. More often, the data required to make the best decisions will already reside in some form or another in the database of the organization or can be captured with an online feed into a computer system, and what is really needed is a device to filter and display and to warn executives about the most important variances (Simon, 1977). As noted by Kleinmutz (1985): “the ability to select relevant variables seems to be more important than procedural sophistication in the processing of that information” (p. 696).
In EIS, the underlying models built into the system are normally very simple and easily understandable, which is a great help in increasing the acceptability of a computer system.
To conclude, the specificities of managerial decision making can be synthesized as follows:
• Most decisions are made very quickly under considerable time pressure (except some strategic decisions).
• Strategic decision making is often the result of collaborative processes.
• Most decisions are linked to individuals who have specific intentions and commitments to personal principles and ideas.
It is therefore very difficult to support managers, and despite many years of research, little is known about the way information systems could support such unstructured tasks.
A VEHICLE FOR INFORMATION REQUIREMENTS ANALYSIS: CRITICAL SUCCESS FACTORS
In pre-EIS days, Rockart (1979) put forward a methodology called critical success factors or CSF to guide information systems planning. The method had its advantages, though it failed to make a general impact on the planning process of organizations. Its potential in other areas, notably the development of information systems, has been explored by a number of researchers. It is argued in this article that it can be very useful as a guide for the development of executive systems, as both from an information content perspective as for the design of the interface of these systems.
CSF assumes that the performance of organizations can be improved by focusing on “the few key areas where things must go right for the business to flourish” (Rockart, 1979). In simple terms, the method seeks to isolate, using the expertise and gut feeling of managers, the factors which may make the difference between success and failure for the firm.
A number of key points about CSF make it a very attractive technique. First of all, while CSF is essentially a generic framework, it recognizes that all firms are different and operate in different markets. Thus, CSFs are different for different organizations. Secondly, the CSF theory takes into account that the needs of managers within the same organizations are also different based on their hierarchical level, but more importantly, based on their style and their specific areas of responsibility. In general, there are only a limited number of factors that each manager should monitor closely, and this guarantees that managers can concentrate their limited attention to factors that really matter and that are within their control. The attractive thing about this breakdown of responsibility is that the CSF sets controlled by the different managers add up to a complete organizational set that covers all the key areas of the business.
Van Bullen and Rockart (1986) identified a number of primary categories of CSF that are useful in guiding the analysis of the organizational CSF set. These generic sources of CSFs are: (1) the industry where the organization operates (these CSFs are shared by mainstream organizations in this industry), (2) the competitive position and strategy pursued by the organization (which are unique to its set of circumstances and objectives set by its top managers), (3) the environmental factors surrounding the organization (which it has no control over, but which it must monitor closely to compete), (4) temporal factors (which relate to specific events or change programs currently facing the organization, and require the temporary monitoring of additional factors), and finally, (5) CSFs that are specific to each manager and their role in the company. Other authors have added other potential sources such as CSFs related to the analysis of main competitors (especially industry leaders) and the evolution of their business (Leidecker & Bruno, 1984). These sources add up to a wealth of potential factors and measurements that are sufficient for effective monitoring of the business of most organizations.
Dashboards and Control Rooms
In the next stage of the development of executive systems, designers must create an interface for displaying the CSFs. The design of this interface is nearly as important as the selection of the indicators in shaping the perception of managers of the usefulness of their information systems and keeping their interest in the long run. One technique that has worked well in selecting and presenting indicators is the application of the dashboard concept to the management of organizations.
Fundamentally, the concept of dashboard reflects the application of the concept of control room to the management of the firm and echoes the call for a warning or exception reporting functionality in EIS-type systems. In engineering, the control room is a specially designed physical area of a plant where the proper operation of key equipment can be monitored. Control rooms have developed because ofthe need to monitor increasingly complex processes, such as petrol refining or the operation of nuclear power plants. The control room allows operators to control a process without looking at it with their own eyes, and with a degree of accuracy and completeness that could not be achieved with human perception alone.
This suggests that dashboards may be developed that considerably help managers in their day-to-day search for problems and matching solutions. Naturally, the nature of management itself is highly dynamic and diverse and involves consideration of infinite number of parameters in a way that is fundamentally different from the monitoring of a manufacturing process. Thus, management has a significant “human interaction” component that cannot easily be supported by computer systems. Simon (1977), Gorry and Scott Morton (1971), and others have commented comprehensively on the degree to which managerial decisions are programmable or not, however it remains that the implementation of many of the objectives of the firm, however elusive, can be monitored using a dashboard-type interface. Further, the CSF method can be a powerful vehicle in selecting the indicators to be shown on each manager’s dashboard.
The difficulty with CSF-based dashboards resides in the operationalization of managers’ key concerns and the identification of specific targets for CSF monitoring, the design of measurement logic for each indicator, and in the development of the interfaces that can be used by managers to easily and effectively review the performance of the firm in relation to each of the indicators.
TOWARDS A METHODOLOGY FOR DASHBOARD DEVELOPMENT
At the height of the EIS movement, King (1985) remarked:
“It is so easy to lose sight of reality—to believe that the computer model’s numerical forecasts are real and that they describe future outcomes that will, in fact, come to pass…The computer model’s forecasts are based solely on those predictions about the future that we are able to quantify. Those things that are not readily quantifiable are usually omitted, and in being omitted there is a danger that they may be ignored.” (p. xi)
This illustrates the dangers inherent in approaching management based solely on numbers, however obtained. This also explains why observational plant tours are still regarded as one of the most reliable methods for collecting data in manufacturing environments (Jones, Saunders & McLeod, 1988). The basis of any dashboard approach to management must therefore take into account the following four key issues:
(1) Limited Attention: Given the limited attention of managers and the costs inherent in sourcing certain data, the indicators displayed on the dashboard must be carefully selected using the CSF.
(2) Performance Measurement: The measurements used to monitor indicators or CSFs are crucial. The usefulness and effectiveness of the dashboard is totally dependent on the accuracy of the data used and the realism of the calculations presented to managers.
(3) Operator Training: It is critical that managers understand the assumptions built into the dashboard and the algorithms used to reach the results presented to them. They must also be fully aware of how data are collected and what limitations applied to the accuracy of the measurements. For instance, drill down facilities can make the difference between “using information to manage more intelligently and more effectively and making the same old mistakes but with more speed” (Meall, 1990). (4) Dashboard Layout: The layout of the dashboard has a direct impact on the understanding derived by managers. The interface of the dashboard must be consistent so that managers can visualize immediately where they should focus their attention as a matter of priority. Exception reporting and color coding (Watson, Rainer & Koh, 1991) can be used to achieve maximum visual impact.
It is also useful if the development of the dashboard can be operationalized as an evolutionary activity where managers can feed back their perception of the dashboards to developers so that the design can be improved over time and the indicators refined or replaced (as in the case of temporal CSFs). In this article, we propose a framework based on 11 questions to support this evolutionary process, and help managers and developers to establish a fruitful dialogue:
• Question 1: Who will use the indicators?
The answer may not be simple when not one, but a number of individuals are interested in monitoring certain indicators. However, managers should concentrate on monitoring the parameters most closely associated with their own performance or that of the areas directly under their control.
• Question 2: Can be mapped out to a specific objective at a higher level?
In the perspective of a top-down CSF exercise, indicators are mapped out to specific objectives pursued by top management. In a bottom-up scenario, it will help if indicators can be merged into higher level composite indicators presented to higher level managers. Developers can use the hierarchy of indicators as a blueprint for the drill-down facility to be built into the dashboard so that top managers can understand the underlying causes of poor or good performance.
• Question 3: How frequently will managers need to monitor each indicator?
Managers’ perception of how frequent significant or revelatory variations are likely to occur should be used as a guide for deciding how frequently indicators should be updated. The scope of the benefits that may arise as a result of the monitoring should also be considered if high costs are likely to be incurred.
• Question 4: What calculation methods are available? What unit of measurement will be used?
The choice of calculation method can greatly influence the variation of an indicator and shift the burden of responsibility from one area to another. It can also influence the way the performance of operators or workshops is measured. The choice of the unit of measurement is normally straightforward for quantitative analysis, but can become far more complex for less tangible CSFs that involve the estimations of qualitative factors. Customer satisfaction, for instance, will require vision and creativity if it is to be measured properly. Some quantitative measures may be applicable such as the number of complaints received per time interval, but other measures may have to be found that can act as surrogates of customer satisfaction.
• Question 5: What data source exists? What should be created?
Certain data may be missing from existing organizational information systems. Other data may reside in a proprietary system (e.g., a custom-built process control system) that does not integrate well with other systems. Significant investment in equipment and special devices (such as scanners and sensors) or in software such as OLAP and ROLAP (Relational OLAP) may have to be made.
• Question 6: How detailed should the analysis presented in the dashboard be? How can the indicators be broken down to be more meaningful?
Many indicators are too broad to be suitably presented as one figure, and some disaggregating may be required. Typical organizations sell multiple products in multiple markets. Thus, sales figures need to be disaggregated to present £ figures, volumes, and variances for each product on each market while also presenting the aggregated data. Multi-dimensional modeling can be used to support the organization and retrieval of such data.
• Question 7: What threshold values should be used to differentiate between adequate and inadequate performance? What comparisons can be made to assess the company’s performance?
Absolute measurement figures presented by a dashboard may not be meaningful to managers unless they can be examined in light of other data. Most companies already have a tight budget system in place, and this can be used as a source of normative values.
• Question 8: How can each indicator be represented for maximum visual impact?
Developers must seek to reduce information overload and use the latest graphical user interface (GUI) technology. Some software tool boxes are now available to help designers create displays and objects that mirror the type of controls normally found on dashboards. Gauges with specific color-coded threshold values can easily be created, and special charts can be made clickable to build intuitive drill down into the data. These speed up and facilitate the data reading of managers.
• Question 9: What action must be taken when good or bad performance is measured? Is there scope for corrective action to be taken based on the indicator?
Whenever good or bad results are presented, managers should know what avenues can be pursued. Reporting mechanisms (e.g., electronic mail) can be built into the dashboard to facilitate and accelerate the dissemination of interesting results and their discussion. In the longer term, increased familiarity with indicators and what their evolution means should have practical decision-making implications for all managers and staff. Thus, users’ reaction times to certain signals should be reduced and their responses should improve, especially in recurrent situations.
• Question 10: How will indicators be monitored/ archived in the long term?
A key element of our approach is the learning that can be achieved when CSFs are monitored over long periods of time. Staff and managers learn from regularly sampling their performance and that of their areas, and seeing it compared to other data, such as budgets and previous performance of industry standards. Greater learning will be derived if managers and staff set time aside to review and discuss indicators on a regular basis.
• Question 11: Is there any potential bias inherent in the methods and data used for calculations? What incentives are being given to staff?
The development of new performance measurement systems, such as a dashboard of indicators, should always be guided by consideration of the incentives given to actors and the behavior likely to result from the implementation of the underlying indicators. There may also be a change management side to the project, as managers negotiate with staff the implementation of the system. Staff may object to a certain type of measurement (which they may perceive to be threatening or invasive) or the implementation of devices dedicated to monitoring their work.
CONCLUSION
The case for managing the firm solely based on numbers has already been argued and lost. At this point, it is well established that managing firms cannot and should not be compared to the administration of a power plant. This does not mean, however, that the concept of control room does not have potential when applied to the management of organizations. Faced with increasingly complex situations and responsibility for the administration of increasingly complex business processes, managers have less and less time to spend monitoring the key factors of the business. The development of a dashboard can speed up this process and help managers catch far more information than they normally would without assistance.
Following the steps highlighted in this article will also give organizations a much better idea of what parameters they should worry about and how to measure performance. Peter Swasey, one of the directors of the Bank of Boston, commented that “what you don’t measure, you don’t manage” (McGill, 1990). The preparatory analysis work on the CSFs of the firm will give much confidence to organizational actors that they understand their business and have a comprehensive hold upon its vital functions, and the dashboard ultimately developed will provide flexible and speedy access to vital information, thereby freeing time for other key activities such as business or staff development. As a by-product, managers may also be able to use the analysis carried out for their dashboard as a blueprint for the incentive systems of their company.
KEY TERMS
Control Room: A special location in a plant where operators can monitor a process in great detail without having to physically be looking at it. This is particularly useful in dangerous environments.
Critical Success Factors: A methodology for managing projects and firms that concentrates on the areas where things must go right if the endeavor is to flourish.
Dashboard: Specific display of information that presents key information about a process or device. A dashboard may or may not be computerized.
Evolutionary Design: System development methodology where an ongoing approach is taken to analyzing the requirements of the application.
Interface: Portion of a computer application that is used by the user to communicate with the application. It is particularly important for a dashboard, because it may impinge on the ability of users to properly interpret the variations in the indicators shown to them.
Managing by Numbers: A school of thought that sought to demonstrate that firms could be managed solely based on watching key (mostly financial) indicators. This is now largely discredited.
Model: A simplified representation of reality that concentrates on predicting how a factor or a series of related factors would evolve based on the variation of a set of parameters. Also, a simplified representation of reality.
