INTRODUCTION
Audience Response Systems (ARS) are gradually being introduced into educational settings, having previously proved their value in business. Hand-held keypads allow students to enter data in response to questions or statements displayed on a public screen. The captured data is displayed on the public screen and enables both academic and students to immediately see how the whole group has responded. The anonymity afforded by an ARS encourages individuals to fully participate without fear of ridicule or loss of face.
The technology is simple to use by both students and academics, can be used with large or small groups and has applications in all topics of study and at all levels of study. ARS are highly portable, require very little set-up time and are easy to use by anyone who has had some experience with software such as PowerPoint.
BACKGROUND
ARS developed within the general area of Computer Supported Collaborative Work (CSCW) and have been used in the business community since the late 1980′s. Early work in the use of computer-based systems was concerned with providing managers with decision support tools that would provide support for, and improve effectiveness of, decision-making especially when dealing with semi-structured or unstructured situations. The systems were developed to be under the control of decision makers and support the process rather than attempt to automate it. Keen & Scott Morton (1978) initially described as Decision Support Systems they have also been described as Group Decision Support Systems (GDSS). Mallach (1994, p.7) defines a GDSS as “an information system whose primary purpose is to provide knowledge workers with information on which to base informed decisions”. Young (1989, p.8) describes the key feature of such systems as being that “… they are intended to interact with and enhance the special mental capabilities of the user, thereby facilitating learning, creativity.”. A key word in the name of these systems is “support”, that is, they enhance the process they are supporting by providing rapid data capture, effective processing tools and immediate feedback to the participants as part of a broader human process.
The initial systems were based on personal computers and often appeared in networked laboratory-type settings but smaller hand-held systems using keypads were developed. From the early 1990′s onwards these systems have been adopted by a growing number of businesses and are now most commonly described as ARS. The relatively low cost and ease of use meant that they attracted the attention of champions in organizations who often had a specific use for the system, often in the areas of quality self-assessment or team building. The initial emphasis on decision-making has now been broadened by recognition that these systems can provide support for a wide range of business processes. The feature of anonymity provided by these systems offers an opportunity for use in environments where there may be holders of widely differing levels of organisational power and authority engaged in discussion. The ease of use, portability, and flexibility in application provides an opportunity for groups to engage in a variety of reflective practices, with groups ranging from eight through to several thousand participants.
Table 1. Typical uses for ARS in business
Quality (Self assessed, Baldrige, EQA…) Control risk self assessment (CRSA)
Evaluating new product ideas Budget and capital resource allocation
Corporate governance Succession planning
Appointment interviews Corporate ethics
Team building Idea sharing, testing, approval
Conflict management Customer value, customer satisfaction
Employee and customer focus groups Stress management
Counselling related Time and Project Management
Readiness for change Benchmarking
Tapping employee wisdom Product development
Typical business uses for these systems are shown in Table 1. Many of them offer great potential for adoption in higher education learning environments where they can be used as support tools in a variety of subject areas and teaching and learning philosophies.
PRACTICAL SYSTEMS
In practice these systems comprise hand-held input devices that transmit data to a receiving device connected to a personal computer. Software processes the data and presents it in a variety of formats to the participants for discussion. Key components of the system are:
• Hand-held input devices. A variety of sizes and designs exist, the credit-card size keypad (Figure 1) being typical of the most recent development.
• Receiver. Utilizes infrared or other wireless communication media to collect data from the keypads.
• Software. Manages collection and processing of data and supports display of the data in a variety of presentational formats. The software may be embedded in other container software such as PowerPoint. The output from the system is usually displayed on a public screen via a data projector (Figure 2).
ARS IN HIGHER EDUCATION
The purpose of an ARS is to provide a system and set of tools that can support groups engaged in a face-to-face process that involves examining propositions, exploring alternatives and obtaining timely feedback from the participants engaged in the process. Draper and Brown (2004, p20) suggest that “The dream of personal teaching is really about adaptive teaching; where what is done depends on the learner’s current state of understanding”. ARS can provide timely feedback to support this adaptive teaching goal, but Draper and Brown make the point that this can only be achieved through appropriate pedagogic design and action and not through the technology alone. In one-to-one or small group settings the learning facilitator may have a sense of the current state of the learner if the learner feels sufficiently comfortable in revealing it. With large groups in more formal settings the availability of cues to the learning facilitator can be more limited. The immediate feedback that an ARS offers can publicly identify differences or similarities of opinion within groups and provide a trigger for further discussion or analysis of data and re-adjustment of pacing or content. ARS can be used with both large (hundreds of participants) and small groups to support lectures, workshops, seminars, and to explore a wide range of subjects. They can be used at undergraduate and postgraduate levels, and within traditional and post-modern paradigms. Subject areas that value discussion, debate, multiple interpretations and direct challenges to accepted wisdom can benefit from this technology, but equally an ARS can be used in subject areas where demonstration of understanding of a fixed body of knowledge is vital. ARS can be used for formative and summative assessment, in the gauging of preliminary level and subsequent stages of understanding of a subject and in the exploration of the concepts that underpin critical issues.
Figure 1. Credit-card size keypad (image courtesy of Keepad Pty Ltd)
Figure 2. Component parts of an ARS
Mitchell (2001) suggests that ARS can be used for mechanistic purposes such as monitoring class attendance via individual handsets, providing instant marking and feedback and for gathering data that can be used to support research activities related to classroom processes. McCabe, Heal and White (2001) used an ARS to support Computer Assisted Assessment (CAA) approaches with mathematics students and consider that it not only reinforced existing CAA activities but also served as a valuable tool for motivating higher levels of student learning. Hunt, Irving, Read and Knight (2003) used an ARS in a first-year information systems unit, in a decision-making subject in a third-year psychology course and also with second-year BSc Pharmacy students. In the pharmacy course questions were posed via the ARS and the resulting answers were displayed and discussed by the whole group. A key issue here is that what is being sought is not necessarily a “correct” answer but instead an examination and exploration of all possible answers and the reasons that individuals give for selecting a specific answer. The students expressed enthusiasm for the system, particularly in its ease of use, the ability to discuss answers immediately after making their choice and in the way it helped students identify where further reading was required. Importantly they also found it to be both easy and fun to use.
Post graduate HRM and MBA students using case-based approaches supported by an ARS indicated that the level of participation and number of ideas generated and explored was greater than usual and that the influence of individual personalities was greatly reduced (Jones, Gear, Connolly & Read, 2001). The students also observed that the technology was simple to use and to some extent became “invisible” to the users. Williams (2003) notes that students on an MBA course using an ARS were strongly in favour of the technology and had negative views about passive learning approaches that simply involved reading or listening. Uhari, Renko and Soini (2003) report that 80% of students studying a pediatric’s course felt that an electronic voting system helped improve their learning and enhanced questioning during lectures. Witt (2003) found that 87% of students studying in a statistics for psychologists course saw more benefits than disadvantages in the use of keypads.
In a session run by the author in 1995 an ARS was used to explore students’ evaluation of an engineering course (Banks, 2001). Students scored each item of the course feedback instrument using individual keypads instead of the normal paper and pencil approach. Once all items were completed they were worked through again, the students being shown the distribution of their scores and invited to comment. It quickly became evident that the criteria used to generate the scores differed widely from student to student and some discussion took place about what common and appropriate criteria might be. After some discussion the items were scored again and it was noted that greater consensus was evident in the second data set. This approach provided an opportunity for students to develop a common understanding of the criteria used in course evaluation and thus generated a more consistent interpretation of the evaluation instrument. The students found the opportunity to discuss their individual and collective views of the course to be a valuable experience.
The author has more recently used ARS to support peer review session in a Collaborative Information Systems Masters course (Banks, 2003). In this course students work in small groups collaboratively producing conference-style papers and peer review is an integral part of the learning process. Peer review was originally summative and was carried out by means of a paper-based instrument. In response to a student suggestion, a “diagnostic” electronic peer review session was introduced early in the course. This offers an opportunity for students to become aware of the perceptions of their peers of their performance against a number of agreed group performance indicators. The use of this diagnostic session allows problems to be surfaced and identified, solutions discussed and action plans developed before any major tensions affect the groups. The author has also used an ARS to support a “lost in the desert” scenario (Banks and Bateman, 2004) that allows groups to see how individuals and groups perform and to act as a trigger for discussion of issues of communication, trust, negotiation, and teamwork in general.
ISSUES IN THE USE OF ARS IN HIGHER EDUCATION
Benefits of Anonymity
In groups where a number of different cultures are represented there is a danger that in traditional face-to-face settings some students will not offer an answer for fear of “loss of face”. Some shy students may also feel inhibited in large groups and thus not feel able to contribute to the discussion. The anonymity afforded by an ARS provides an opportunity for these potentially disadvantaged students to fully participate in the learning process. The author has noticed that in small group peer review sessions, there is a tendency for overseas students to talk quite comfortably to the data on the screen rather than feel personally challenged. The system appears to offer a de-contextualisation of the student data and permits a more open discussion than has been experienced with traditional face-to-face peer review sessions. On the other hand, anonymity can mean that students enter data mischievously, safe in the knowledge that they cannot be identified.
More Than Just the Numbers
For in-class summative tests the ability to use an ARS to collect numeric data from the individual keypads is an obvious asset. However, in more discursive learning environments the distribution patterns of the data can form a trigger for critical discussion. The issue is not what the average or mean score is, but in trying to determine why a particular pattern of data is captured. Differences in scoring patterns by students suggest differing worldviews or interpretations of the data and this provides an opportunity for reflection, sharing of views, critical thinking and deep learning.
Educational Paradigm
This technology can be used with any subject, at any level of education and within any educational paradigm. The paradigm that appears to be common to most current users of ARS is that of constructivism.
Constructivist approaches are based on the premise that learners should engage in active construction of their own knowledge rather than passively receive the views of others. This process requires that learners engage in critical and reflective behaviour and this takes place in response to interactions with other individuals as part of a social sense-making process. The emphasis in terms of teaching moves away from a “sage on the stage” model with lectures as information delivery mechanisms, towards a “guide on the side” model where the focus is upon challenge or justification of prevailing views. The approach exposes students to new experiences and modes of thinking and enables them to examine their own ideas and to determine the extent to which the new experiences make sense in the light of these ideas. They are also encouraged to consider the number of possible alternative explanations for what they have experienced and to evaluate the usefulness of a number of different perspectives (Dufresne, Gerace, Leonard, Mestre & Wenk, 1996). This critical process can take place within a structured learning environment through “justificatory arguments” which are “brought forward in support of assertions, in the structures they may be expected to have, the merits they claim and the ways in which we set about grading, assessing and criticising them” (Toulmin, 1958, p.12). ARS can provide a valuable support mechanism for such approaches.
Question, Then Question Again
By asking a question a number of times and critically evaluating the distribution of the ARS responses it becomes possible to explore the reasons for the differing or changing responses. Gauging the responses of students through the use of an ARS allows for quick and multiple loops around the material if learning appears to be problematic. d’Inverno, Davis and White (2003) report that the use of an ARS suggests that typically around 40% fail to identify the correct answer to simple questions, and that if the same question is asked again around 20% still provide the wrong answer. (They do, however, suggest that there may be some deliberate entry of incorrect answers as not all students feel that the technology offers them benefit.)
Over-Use of the Technology
Over-use or inappropriate use may trivialise the technology and lead to student dissatisfaction. The software for ARS typically interfaces with PowerPoint, and the risk of building a mechanized performance rather than genuinely engaging the learner must be borne in mind. As with any technology it should only be used when it offers a tangible benefit to the process.
Accessibility/Equity for All Students
One potential problem with the introduction of an ARS is that of the provision of the keypads for students. Even though the cost of keypads is already low, and falling, the provision of one keypad per student clearly represents a large investment and technology management issue for an educational institution. Ownership of keypads by the institution also raises the problem of issuing and collecting the keypads before and after learning sessions, maintenance and so on. On way to overcome this problem is to make the students responsible for the keypads. This can be achieved through the currently developing approach of providing students with shrink-wrapped packages of text topics and keypad, the cost of the keypads being built in to the package purchase price.
FUTURE TRENDS
ARS have shown a dramatic growth in recent years. The increased availability of such technology combined with falling prices and innovative distribution models such as shrink-wrapping with text topics should encourage more educators to adopt ARS. The simplicity of use and natural integration with all subject areas and teaching styles should ensure the painless adoption of the technology across institutions. Such widespread adoption will mean that students feel that the purchase of personal keypad systems represents a good investment.
In the near future, systems that utilise personal digital assistant (PDA) technology and mobile phone technology will add text input, further increasing the versatility and power of these systems. There are many research questions to be explored as this technology is introduced. These will include the potential for the use of the technology to promote deep learning, the long-term reactions of students to the technology versus the initial novelty factor, the effectiveness at different levels of study and in different subjects, and many other areas of concern will offer many opportunities for investigation. The use of frequent in-course evaluations rather than a single exit evaluation will allow student concerns to be addressed in a more timely way and will also allow discussion to take place between teacher and learner about both the evaluation instrument and the meaning of the captured data.
The ARS itself will provide a vehicle to support the gathering of research data in actual teaching and practice. The availability of data immediately after a session has been conducted should allow timely adjustments to be made in pacing and content.
CONCLUSION
Considerable time and effort is being invested in distance learning via the web, but it is equally important that the benefits of technology are applied to support and enhance more traditional face-to-face learning environments.
ARS provide educators with an opportunity support their existing teaching and learning strategies in a way that provides them with improved, immediate and dynamic insight to the progress of learners. Students have an opportunity to engage with learning in an active way that helps them see how they, and their peers, are performing on a moment-to-moment basis. The opportunities for learning to be a shared experience are improved and there is the potential for engagement with subject material at a deep level.
KEY TERMS
Anonymity: A feature of an ARS that can protect the identity of a participant.
Audience Response System (ARS): An electronic system designed to support and enhance face-to-face group interaction by means of individual hand-held communication devices.
Computer Supported Collaborative Work (CSCW):Use of computer-based technology, including ARS, to support group processes.
Constructivism: An educational approach that takes the view that knowledge must be constructed within the cognitive structure of each individual.
Group Decision Support System (GDSS): A collection of hardware and software used to support decision-makers.
Keypad: A hand-held device that allows a participant to communicate data to an Audience Response System.
