INTRODUCTION
The purpose of this article is to re-address the vision of human-computer symbiosis expressed by J. C. R. Licklider nearly a half century ago, when he wrote: “The hope is that in not too many years, human brains and computing machines will be coupled together very tightly, and that the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today” (Licklider, 1960). Unfortunately, little progress was made toward this vision over 4 decades following Licklider’s challenge, despite significant advancements in the fields of human factors and computer science. Licklider’s vision was largely forgotten. However, recent advances in information science and technology, psychology, and neuroscience have rekindled the potential of making the Licklider’s vision a reality. This article provides a historical context for and updates the vision, and it argues that such a vision is needed as a unifying framework for advancing IS&T.
BACKGROUND
Licklider’s statement is breathtaking for its vision, especially considering the state of computer technology at that time, that is, large mainframes, punch cards, and batch processing. It is curious to note that Licklider did not use the term symbiosis again, but he did introduce more visionary ideas in a symbiotic vein. An article he co-authored with Robert Taylor titled The Computer As a Communication Device made the bold assertion, “In a few years, men will be able to communicate more effectively through a machine than face to face” (Licklider & Taylor, 1968). Clearly, the time estimate was optimistic, but the vision was noteworthy. Licklider and Taylor described the role of the computer in effective communication by introducing the concept of “On-Line Interactive Vicarious Expediter and Responder” (OLIVER), an acronym that by no coincidence was chosen to honor artificial intelligence researcher and the father of machine perception, Oliver Selfridge. OLIVER would be able to take notes when so directed, would know what you do, what you read, what you buy and where to buy it. It would know your friends and acquaintances and would know who and what is important to you. This article made heavy use of the concept of “mental models,” which was relatively new to the psychology of that day. The computer was conceived of as an active participant rather than as a passive communication device. Remember that when this article was written, computers were large devices used by specialists. The age of personal computing was off in the future.
Born during World War II, the field of human factors engineering gained prominence for its research on the placement of controls, widely known as knobology, which was an unjust characterization. Many important contributions were made to the design of aircraft, including controls and displays. With strong roots in research on human performance and human errors, the field gained prominence through the work of many leaders in the field who came out of the military: Alphonse Chapanis, a psychologist and a Lieutenant in the U.S. Air Force; Alexander Williams, a psychologist and naval aviator; Air Force Colonel Paul Fitts; and J.C.R. Licklider. Beginning with Chapanis, who realized that “pilot errors” were most often cockpit design errors that could be corrected by the application of human factors to display and controls, these early educators were instrumental in launching the discipline of aviation psychology and human factors engineering that led to worldwide standards in the aviation industry. These men were influential in demonstrating that the military and aviation industry could benefit from research and expertise of the human factors academic community; their works (Fitts, 1951) were inspirational in guiding research and design in engineering psychology for decades. Among the most influential early articles in the field that came out of this academic discipline was George Miller’s (1956) “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity to Process Information,” which helped to usher in the field of cognitive science and application of more quantitative approaches to the study of cognitive activity and performance.
An early focus of human factors engineering was to design systems informed by known human information processing limitations and capabilities, systems that exploit our cognitive strengths and accommodate our weaknesses (inspired by the early ideas represented in the Fitts’ List that compared human and machine capabilities (1951). While the early HFE practice emphasized improvements in the design of equipment to make up for human limitations (reflecting a tradition of machine centered computing), a new way of thinking about human factors was characterized by the design of the human-machine system, or more generally, human- or user-centered computing (Norman & Draper, 1986). The new subdiscipline of interaction design emerged in the 1970s and 1980s that emphasizes the need to organize information in ways to help reduce clutter and “information overload” and to help cope with design challenges for next-generation systems that will be increasingly complex while being staffed with fewer people.
There have also been theoretical developments in cognitive psychology that provide a foundation for Licklider’s vision. Central here is the work by Kahneman (2002, 2003). In his effort to reconcile seemingly contradictory results in studies of judgment under uncertainty, he has advanced the notion oftwo cognitive systems introduced by Sloman (1996) and others (Stanovich & West, 2002). System 1, termed Intuition, is fast, parallel, automatic, effortless, associative, slow-learning, and emotional. System 2, termed Reasoning, is slow, serial, controlled, effortful, rule-governed, flexible, and neutral. Cognitive illusions, which were part of the work for which he won the Nobel Prize, as well as perceptual illusions, are the results of System 1 processing. Expertise is primarily a resident of System 1 as is most of our skilled performance such as recognition, speaking, and driving. System 2, on the other hand, consists of conscious operations and is commonly thought of as thinking.
System 1 is effective presumably due to evolutionary forces, massive experience, and by constraining context. Most of the time it works quite effectively. System 1 uses nonconscious heuristics to achieve these efficiencies, so occasionally it errs and misfires. Such misfires are responsible for perceptual and cognitive errors. One of the roles of System 2 is to monitor the outputs of System 1 processes.
NEO-SYMBIOSIS: A VISION AND FRAMEWORK FOR CONDUCTING RESEARCH
Licklider’s notion of symbiosis does require updating. The term “man/computer symbiosis” is both politically incorrect and factually inaccurate. “Human/machine symbiosis” is preferable. There is also a problem with the term symbiosis itself. Symbiosis implies a co-equality between mutually supportive organisms. However, humans must be in the su-perordinate position. Dreyfus (1972, 1979, 1992) has made compelling arguments that there are fundamental limitations to what computers can accomplish, limitations that will never be overcome. In this case it is important that the human remain in the superordinate position so that these computer limitations can be circumvented. At the other extreme, Kurzweil (1999) has argued for the unlimited potential of computers. Should it be proven that computers, too, have this unlimited potential, then some attention needs to be paid to Bill Joy and his nightmarish vision of the future should technology go awry (Joy, 2000). In this case, we humans would need to be in the superordinate position for our own survival. Griffith (2005) has introduced the term neo-symbiosis for this updated version of symbiosis.
Kahneman’s two system theory plays a central role in neo-symbiosis. It is the System 2 processes that require computer support, not only with respect to the pure drudgery and slowness of System 2 processes, but also with respect to the monitoring of System 1 processes. In most cases, it is a mistake to assign System 1 processes to the computer. This was the fundamental error in many automatic target recognition and image interpretation algorithms that attempted to automate the human out of the loop. The perceptual recognition processes of most humans are quite good. System design should capitalize upon these superb processes and provide support to other areas of human information processing such as search (to overcome a tendency to overlook targets), interpretation keys to provide support for the recognition processes. Other types of System 2 support could include the augmentation (not replacement) of human reasoning processes, support to facilitate adjusting to changes in context to maintain situ-ational awareness and computational support.
A related approach is Joint Cognitive Systems (JCS’s) (Hollnagel & Woods, 2005; Woods & Hollnagel, 2006), which represents a specific implementation of cognitive systems engineering. As the term JCS implies, this approach views the human-computer system as a combination of human and machine cognition. Another way of looking at this is that the human is a component of the computer architecture (consistent with our view of neo-symbiosis). In their two volumes, Hollnagel and Woods have developed a sophisticated approach to system design, but it does not draw much from either cognitive psychology or cognitive neuroscience. Neo-symbiosis draws liberally from both cognitive psychology and cognitive neuroscience. In our view, neo-symbiosis is a subset of cognitive systems engineering that may be applied to enrich the field through its focus on human cognition and the supervisory role of humans in joint cognitive systems.
Another related approach is hedonomics. Hedonom-ics (Hancock, Pepe, & Murphy, 2005) can most easily be thought of as designing technology to climb Maslow’s (1970) Hierarchy of Needs. According to Maslow, human needs can be arranged in a hierarchy or pyramid beginning with physiological needs at the base, then proceeding up to safety, love and belonging, self-esteem, and ending with self-actualization at the top. An interesting exercise is to consider how technology can, and sometimes does, facilitate meeting these needs. Hedonomics is certainly in the spirit of neo-symbiosis. Both hedonomics and neo-symbiosis have the same destination. But in its present state, hedonomics presents what is effectively a brochure of the destination, whereas neo-symbiosis provides some direction as to how to get to that destination.
Augmented cognition (Schmorrow & Kruse, 2004; Schmorrow, Stanney, & Reeves, 2006; see also Greitzer, 2005) holds much relevance and potential for neo-symbiosis. The program was initiated within the Defense Advanced Research Projects Agency (DARPA) Information Processing Technology Office (IPTO) with the aim to monitor and assess the user’s cognitive state through behaviorally- and physiologically-derived measures acquired from the user while interacting with the system, and then to adapt or augment the computational interface to improve performance of the human-computer system. Clearly, the effort here is to address human information processing shortcomings, that is, to augment cognition. A theoretical framework is needed to identify what aspects of human cognition to augment. Greitzer and Griffith (2006) proposed neo-symbiosis, through Kahneman’s two system model of cognition, for that theoretical framework. Thought also needs to be given to how human cognition can augment machine cognition in other than a trivial sense. Such human-computer collaboration is needed to realize Licklider’s vision of an interaction between humans and computers so that produced levels of performance are achieved that have never been achieved before.
Although it is important to augment human cognition, it is equally important to consider how to leverage human cognition. Consider chess expertise, for example. Rather than trying to build chess playing systems that can defeat grandmasters, why not try to build systems with the grand master in the architecture to achieve unseen levels of chess playing performance. Tournaments could be run in which joint human/computer chess playing systems would compete. This calls to mind the Amazon Mechanical Turk Program (http://www.mturk.com/mturk/welcome) in which humans are invited to be used in programmatic ways by computers to solve problems that the computer can’t solve.
FUTURE TRENDS
We shall consider alternative future trends from two different perspectives, a pessimistic perspective in which Licklider’s vision is not fully realized, and an optimistic perspective in which desired outcomes and trends do occur. Let us first consider the trends we hope to see in the future.
A most significant facet of this trend will be the occurrence of a revolution in thinking. In a neo-symbiotic system, the human is superordinate. Because system design is theory driven, the system addresses human cognitive shortcomings and leverages human cognitive strengths. To do so requires a model of the human user. There are two levels of user models. One is a generic or nomothetic user model. Essentially,
Kahneman’s two system model of cognition provides the basis for a nomothetic user model, and there are many human factors handbooks that can assist in building nomothetic models. A second type of user model is an idiosyncratic user model, one that is specific to a particular human user. For example, the chess grand master system envisioned above requires an idiosyncratic model of a specific grand master. In the future, we hope to see systems that develop and refine models of specific human supervisor/partners (users) as a result of interaction with the computers. This is done to a limited extent today, but we envision that this will be taken to new levels of detail and sophistication. It is hoped that research in augmented cognition will become theory driven and will achieve advances to this end. That is, specific cognitive shortcomings would be remedied and specific cognitive strengths would be leveraged.
An article by Griffith and Greitzer (2007) develops a research agenda for neo-symbiosis. This is derived from Kahneman’s two system model. Note that the identification of neurological correlates, although potentially helpful and holding vast potential, is not a requirement, nor is it the only enabler for neo-symbiosis. The article shows how requirements can be written to facilitate neo-symbiotic design. The authors also discuss the need for more attention on metrics to assess the extent to which neo-symbiosis has been achieved
Griffith (2006) has argued that neo-symbiosis can be achieved over a wide range of technological sophistication and describes some of its social and political ramifications. In business and industry, for example, the goal of replacing workers with technology would be superseded by using technology to enhance the potential and productivity of workers though their interaction with the technology.
So the bright future for neo-symbiosis is one of increased productivity and fulfillment: That is, people will not only be more productive, but also more fulfilled. They will proceed further up Maslow’s hierarchy of needs. People will be happier. Life will be good.
But what if we ignore Licklider’s vision? One could argue that the destination will remain the same, but that the journey will take longer. It seems that one could make an argument to this effect along evolutionary lines. A less happy possibility is that we never arrive. We continue to use technology in a non-optimal manner, failing even to address rudimentary usability issues. We fail to achieve the potential and happiness offered by technology as a result of deficient concepts regarding how to think about technology. Yet another scenario is that technology advances, but humanity declines. That is, humans become mentally lazy and potentialities decline as computers are assigned the challenging work. Of course, the most nightmarish vision is that of Bill Joy (2000). Here machines become intelligent and decide that we are no longer needed. The future continues without us.
CONCLUSION
Symbiosis was a metaphor for a vision advanced by Licklider early in the new era of IS&T. Neo-symbiosis is an updating of that vision. We have argued that advances in cognitive theory and computer technology have provided the basis for the realization of that vision. Related approaches have been reviewed. The JCS approach combines human and machine cognition into the architecture. Neo-symbiosis agrees that they need to be combined into the architecture. Hedonom-ics argues that technology should be viewed as a means for human fulfillment, as does neo-symbiosis. Augmented cognition places heavy emphasis of physiological indices to enhance cognition. Neo-symbiosis is interested in more than enhancing cognition, however. It is also interested in leveraging human cognition so that, in Licklider’s words “… the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information processing machines.”. Neo-symbiosis not only provides a goal, but also provides a theoretical basis for achieving that goal.
We have also described several alternative futures. If one takes an evolutionary perspective, we might eventually achieve Licklider’s vision, but perhaps not as quickly compared to a more deliberate approach. We also speculated about some darker alternative futures in which humanity cedes the field to technology.
KEY TERMS
Augmented Cognition: This area of research seeks methods for addressing cognitive bottlenecks (e.g., limitations in attention, memory, learning, comprehension, visualization abilities, and decision making) to extend human information management capacity via technologies that assess the user’s cognitive status in real time.
Cognitive Systems Engineering: This is a design philosophy that advances a broad system design perspective employing modeling concepts from engineering, psychology, cognitive science, information science, and computer science, emphasizing human cognitive processes in system design.
Hedonomics: This is a design philosophy that considers a hierarchy of human needs in system design.
Human Centered Design: This is a design philosophy that emphasizes the needs and abilities of the user in the design of a system.
Human Factors: This is the field devoted to understanding and applying the properties of human capabilities to the design and development of systems with the aim of improving operational performance and safety.
Joint Cognitive Systems (JCS’s): This design philosophy regards a system as a whole comprising people and technology acting together.
Neo-Symbiosis: This is an updating of Licklider’s vision in which technology is placed in a subordinate role to the human.
Symbiosis: This is Licklider’s vision that in not too many years, human brains and computing machines will be coupled together very tightly, and that the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today.
Two System Theory of Cognition: Although there are a number of two system theories, this refers to Kahneman’s concept of Intuitive and Reasoning systems.
