Ena Howse, Donna Hamilton and Larry Symons
Queen's University
Summer 2000

Abstract
This study investigated the effects of a SMART Board on nursing students’ academic performance, group learning processes and user satisfaction. The SMART Board is a computerized whiteboard through which new ideas can be recorded, saved, recalled and integrated with other information. Because of these features, it was assumed that the SMART Board would facilitate interactive and collaborative learning and these effects would be evident in improved test scores, generation of ideas, satisfaction with group learning processes and user satisfaction with the SMART Board.

Participants were senior nursing students enrolled in a 12-week, applied management course, who used management interventions (concepts) during clinical practice, and subsequently presented oral reports of their analyses of concept application in group seminars. An intervention group of 15 students used the SMART Board to facilitate seminar discussions, while a comparison group of 15 students, not assigned to a SMART Board intervention, used a conventional method of oral presentation. To diminish intervention effects between the two groups, the comparison group completed a post-discussion evaluation exercise.

Although differences between the two study groups for the knowledge test and group processes were not significant, group mean scores were slightly higher for the intervention group. Significant differences were found between the groups for generation of ideas. Relative to the comparison group, the computer-assisted group not only generated more ideas, but also focused more closely on concepts. User satisfaction with the SMART Board was moderately high, reflecting a positive attitude toward the SMART Board. Further testing, including longitudinal and larger-scale studies, is required to extend knowledge in this area, and these studies should be designed to assess the actual transfer of knowledge to clinical situations.

Introduction
Studies on the role of computer-assisted learning in promoting concept development, interactive learning, collaborative learning and transfer of learning have produced modest support. However, the utility of technologies for improving the learning process is not fully understood (Carey & Kacmar, 1997). This study was conducted to determine whether use of a SMART Board would stimulate more interactive and productive exchanges about management concepts and subsequently improve critical thinking of students. Interest in critical thinking processes was based on a view that transfer of knowledge into practice is dependent upon the critical thinking that occurs during the acquisition of concepts (Halpern, 1998). Transfer of knowledge is a key educational goal and difficulty in transferring management concepts is well recognized (Champagne, 1999). Failure to transfer knowledge can have a negative effect on the functioning of work teams, productivity and job satisfaction (Newstrom, 1986; Baldwin and Ford, 1988; Broad & Newstrom, 1992). With new learning techniques, such as the SMART Board, some of these deficits could be reduced.

The SMART Board is an interactive whiteboard, which a learner can use with a computer alone or with a data projector to capture written or typed information on the Board, manipulate the data, store it and recall it later for integration with information from internet sources or data previously stored on a disk. Prior research suggests that computer technologies may enhance the extent, quality and depth of group discussion (Ocker & Yaverbaum, 1999), but findings on user satisfaction with computer-assisted group learning are mixed (Johnson, 1997; Ho, 1999). No studies were found that examined the impact of such technologies on collaborative learning of management concepts or the attitudes of students about the role of technology in this process. According to Griffith (1999), the extent to which people use technology may depend upon their understanding of its features and their ability to make sense of it. Therefore, we examined the effect of the SMART Board in enhancing face-to-face discussions, group processes and satisfaction with technology features on a group of undergraduate nursing students enrolled in an applied health-care management course.

Literature Review
Although findings on the impact of computers on learning are mixed, current studies show some evidence of productivity in group interaction, generation of ideas, test scores and satisfaction with technologies. Brief reviews of studies that are suggestive of success are cited here to illustrate the nature of previous work and the directions of current research.

Role of Computers in Concept Learning
Computer-oriented studies have focused on the associations between computer use and such cognitive outcomes as improved test scores and motivation. For example, Gilliver et al. (1998) showed that use of technology resulted in an eleven percent gain in productivity in an academic class. Emerging studies tend to address cognitive mechanisms that may account for improvement in concept learning. Phillips and Pierson (1997) speculated that software supports problem solving by shifting the cognitive load for low-level cognitive tasks, so that attention can be focused on more complex tasks. Deadman (1997) found that a computerized reflective writing exercise induced better reasoning skills than did teacher support alone. Similarly, Cohen (1997) reported that an interactive approach to learning through computers resulted in greater depth of learning for a group of students than that achieved by a control group.

Assessment of concept learning is challenging, considering cognitive psychology studies that point to many determinants including cognitive capacity, motivation, repetition, drill, feedback, establishing connections among ideas, modelling behaviour, level of concentration and integration of ideas as they emerge (Schacter, 1987; Cormier and Hagman, 1987; Norris, 1992). Further, the application or transfer of knowledge, which is a critical test of learning, appears to depend upon the quality of initial concept learning. If concepts are learned well, the student should retain useful cues that will trigger later retrieval of relevant knowledge from memory (Halpern, 1998). It is also believed that concepts are not well understood until the learner has progressed through specific cognitive and interactional phases. Bloom’s taxonomy (1956) indicates that higher-order, interactive-learning tasks (analysis, synthesis and evaluation) are prerequisites for effective learning and Kolb's typology of learning (1984) emphasizes the importance of direct experience, abstract conceptualization, active experimentation and reflection on experiences. These ideas indicate that attention to the complexity of learning is necessary in order to gain a better estimate of the impact of computers.

Role of Computers in Promoting Interactive and Collaborative Learning
Collaborative computing technologies promote interactive exchanges between the learner and the technology and among individuals in groups. According to Raatz (1993), collaborative computing allows groups to build common databases or repositories of information and together retrieve, replicate, edit and expand it. As a result, more effort can be focused on decisions and deeper critical thinking can occur. In a controlled study, Wegerif et al. (1998) found that coaching in exploratory talk, which involves constructive criticism, appeared to lead to an increase in the number of group interactions on the computer. The importance of collaboration is also emphasized by Ocker and Yaverbaum (1999), who examined differences between asynchronous (different contact times) computer-mediated communication and face-to-face collaboration. They found that both approaches were equally effective, but the computer-mediated approach was less favoured because of a lack of group interaction. In related group studies, Larsen et al. (1985) showed that cooperative learning groups exhibited higher levels of transfer of learning, and Yang (1999) demonstrated that a group who shared information and synthesized ideas in a collaborative computing context showed greater gains than a non-computing group by creating a broader network of signs and meanings in an assigned task.

Generation of Ideas in a Computerized Group-Learning Context
Studies on computer-enhanced group decision making have shown that group support systems appear to stimulate an increase in the production and quality of ideas (Dennis & Valacich, 1993; Valacich et al., 1994; Gallupe et al., 1992). Dennis and Valacich (1993) suggested that these effects might be due to the reduction of apprehension that results from being an anonymous participant. Some support for the effect of attitudes on participation is provided by Barling and Beattie (1983), who found that self-efficacy in group dynamics is associated with individual performance. Self-efficacy is also implicated in the use of technology in that the nature of interventions may affect the tendency to use it (Gist et al., 1989). Group support systems may also be effective because they provide structure for discussions (Siau, 1995). Additional support for the importance of structure is provided by Homrich (1997), who reported that students who used a structured group-support system to solve psychological cases proposed more treatment solutions than a face-to-face group. However, Reid et al. (1997) showed that despite effectiveness in generating ideas in a computing situation, participants expressed dissatisfaction with the computer medium for handling value-laden issues and preferred personal exchange and negotiation for this purpose.

Attitudes toward Computer Technology
User satisfaction is a key indicator of the utility of computing innovations. Early attitudinal studies on computing technology focused on users’ perceptions about hardware and software, and commonly showed that systems were unrefined and inefficient (Guinan et al., 1997). Satisfaction studies revealed a range of barriers, including anxiety, phobias and gender differences in adapting to technologies (Mahmood & Medewitz, 1989; Ager & Bendall, 1991). The reliability of early satisfaction studies has been questioned because of methodological deficits such as the inadequacy of measurement scales and small sample sizes, but recently, more valid and reliable tools have been produced (Chin et al., 1998) and attention is being directed towards the effect of particular features of technology on satisfaction (Chin et al, 1998; Griffith, 1999).

The accumulated findings suggest that computing technologies have the potential for enhancing concept learning in collaborative contexts, and studies that address a wide range of variables are now addressing underlying mechanisms that account for learning. In this study we build on some of these approaches to examine whether the SMART Board could induce more effective concept learning, greater generation of ideas, satisfaction with the group learning process and positive attitudes toward the technology itself.

Method
We used a comparative approach to study the influence of the SMART Board over a single academic term.

Setting and Sample
A total of 30 nursing students in the final year of an undergraduate program participated in the study and were randomly assigned to an intervention and a comparison group, comprising 15 students for each condition. Twenty-seven of the participants were female, three were male and all but three were in the 20–25 year age range. All students in the SMART Board intervention group had a wide range of experience with basic computer applications, such as word processing, data processing and graphics.

Procedure
Both groups participated in four 2-hour mandatory seminars over a 12-week period as part of a management field experience, which included testing of three management concepts (conflict resolution, motivation and work coordination) previously reviewed in a theory course. A single concept was reviewed in each seminar and four students, in succession, presented a 30-minute oral presentation on their individual findings. Testing involved construction of a specific learning objective, writing a plan for applying the concept and assessing the results. One example was to "examine the effect of a conflict-resolution strategy on staff satisfaction."

During oral presentations, all students were required to describe the events that occurred during testing and to use theory to support their interpretations. The intervention group received training in use of the SMART Board from a qualified trainer prior to the seminars and used the SMART Board to facilitate on-going discussion. They were expected to record questions and contributions from classmates on the SMART Board, draw on data stored on disks and integrate it with emerging ideas.

Those in the comparison group used a conventional presentation approach – explanation, followed by a question-and-answer period and use of overheads, slides or a blackboard to enhance presentations. To minimize the Hawthorne effect that might occur in the SMART Board group in response to receiving attention as a study group, members of the comparison group also completed an evaluative minute paper, which is a short description of the most important idea covered and an idea that needed to be addressed. Feedback on the minute paper was provided to students after the project was completed.

At the end of the last seminar, each group completed attitudinal surveys on the group learning process and self-efficacy in group discussion skills. In addition, the computer-assisted group completed a satisfaction survey on use of the SMART Board. In a separate joint session, all students completed an end-of-term multiple-choice quiz on concepts.

Measures
Knowledge gain was tested with a standardized, 50-item multiple-choice test on three assigned concepts. Items tested higher-order learning or the ability to use analysis, synthesis and evaluation skills.

An investigator recorded all ideas that emerged during presentations, included them in the total count and assessed them for relevance to the concept. Comments or questions that suggested alternate interventions, amplification of previous solutions to the problem, different arguments to support or negate conclusions, or synthesized ideas were judged to be relevant, as opposed to those which were extraneous or unrelated to a concept.

Assessment of group processes included a measure of self-efficacy in group skills and a tool to assess perceptions of group performance skills. The self-efficacy tool was developed according to Bandura’s (1997) requirements for task-specific assessments and included 10 items drawn from group concept-development tasks described by Halpern (1998). This tool included such items as "ability to engage group in discussion, able to hypothesize about group suggestions, and able to predict how group decisions might work." Each of these items was rated on a 10-point scale for perceived ability to perform a task and level of confidence for each task. Perceptions about the group learning process were measured on a 5-point scale adapted from Carey and Kacmar (1997) and included items that reflected factors such as satisfaction with flow of communication, level of cooperation and contributions to group discussions. Reliability for this questionnaire was high (Cronbach’s alpha = .78).

Attitudes towards the SMART Board were assessed with four components from the Questionnaire for User Interface Satisfaction (QUIS), Version 7.0, developed at the University of Maryland (Norman et al., 1998). It assesses subjective satisfaction with eleven technology features on a 9-point scale and has a high level of reliability. Four components of the tool (overall reactions, screen, learning, terminology and system information) were selected for this study. Overall reactions are sought for terrible versus wonderful, frustrating versus satisfying, ease of use, stimulation, adequacy and flexibility. Questions on the utility of the screen measures satisfaction with such characteristics as visual displays, fonts, highlights and layout. The terminology section assesses satisfaction with system messages, such as clarity of messages, performance of procedures and utility of error messages. A number of items were added to the latter section for features specific to the SMART Board, including attitudes toward ability to move data on screen, retrieve data from a disk, make slides and interact with PowerPoint. The learning section assessed users’ perceptions of their ability to learn complex tasks through system instruction, by trial and error, and how to correct mistakes. In addition, a microcomputer playfulness tool was used to estimate the participants’ tendencies to interact spontaneously, inventively and imaginatively with the SMART Board (Webster & Martocchio, 1992). Questions were also extracted from the QUIS tool to determine past experience with computers and demographic characteristics of participants.

Results
The mean score on a cognitive test of concepts was slightly higher in the SMART Board intervention group (73.7; SD 11.6) than that for the comparison group (69.2; SD 8.23), but a paired samples t-test showed no significant differences in scores between the two groups (Table 1). Similarly, findings for group processes, including self-efficacy ratings, did not differ significantly.

However, the total number of ideas generated by the intervention group was significantly higher than those for the comparison group (Table 2).

* sig. at 0.05 level

Observers noted that group discussions differed not only in quantity of ideas produced, but in the relationships of ideas to the concepts. In the test group, ideas were mostly related to the concept, whereas there was a tendency in the comparison group to produce ideas and questions that were extraneous and unrelated to the concept. The quality of ideas in both groups varied from simple questions for clarification to statements that supported or refuted points, with a slightly greater emphasis on argument in the intervention group. Although differences in the quality of discussion were apparent, neither group engaged in complex reasoning or synthesis of ideas.

Students in both groups tended to use the PowerPoint program to prepare slides for their presentations. Those in the intervention group tended to add blank slides in PowerPoint to capture ideas generated by group members and to manipulate this data through SMART Notebook software. This approach appeared to stimulate group interaction and interest.

Group Processes
Although not significant, the mean score for group processes, which reflected perceptions of the group interactions and contributions, was slightly higher in the intervention group (see Table 1). The mean score for self-efficacy in the intervention group was comparable to that of the comparison group (see Table 1).

Investigators noted that students consistently engaged in a collaborative process by helping each other to learn how to use the SMART Board. They assisted each other to load data, access data from PowerPoint and experiment with data manipulation through SMART Notebook. There was little evidence of anxiety or avoidance in response to working with the SMART Board. Also, the climate in the SMART Board group differed from the comparison group in that there was more dialogue and verbal exchange during oral presentations.

User Attitudes towards the SMART Board
The overall reliability of the QUIS tool was .79 and the overall mean on the 9-point scale for four components – overall reaction (mean 5.81), terminology/system (mean 6.72), screen (mean 6.04) and learning (mean 5.74) – was 6.08, suggesting a moderately high level of satisfaction with the SMART Board (see Table 3). All components, except learning, were significantly correlated (see Table 4).

*sig. at 0.05 level
**sig. at 0.01 level

The section on overall reactions represented global satisfaction for the SMART Board (see Table 5). On the 9-point scale, the highest mean was calculated for degree of stimulation versus dullness (6.7; SD 1.7) and the lowest was found for degree of satisfaction versus frustration (5.0; SD 1.7), suggesting that students discriminated among the items.

Screen satisfaction ratings assessed responses to characters on the screen, highlighting, and manipulation of data, as well as ease of storing and recalling information. The mean (6.72; SD .9980) was the highest recorded for all features. Some positive comments supported this rating, including "PowerPoint presentations came up clear and the right places," and "I really liked how the screen is bigger than a TV monitor and promotes participation of everyone in the group." On the other hand, it was noted that "writing on the screen was challenging…had to get used to the amount of pressure needed to write on the screen," and "the screen was hard to perfectly orient; wish that it was easier to write with pens."

Learning
The ratings for specific learning supports showed that students were generally satisfied with this feature, although it received a slightly lower rating than other features, as noted by means and correlations for the four features (see Tables 3 and 4). Several subjective comments indicated that "more time is needed to experiment with the system" and "watching others was helpful." However, only one student noted that it (SMART Board) was "not too effective; needed more time to learn how to use it."

SMART Board Playfulness
The overall mean for playfulness was 3.61 on a 5-point scale, suggesting that the SMART Board induced positive emotional responses (see Table 6). Scores ranged from 3.8 to 4.0 for creativity, imaginativeness, inventiveness and originality.

*scores reversed

Discussion
The results suggest that use of the SMART Board in group discussions resulted in greater generation of ideas and a moderately high level of satisfaction with the technology. However, significant gains were not demonstrated in cognitive testing, group discussion processes or self-efficacy in group process skills.

While cognitive test scores for the intervention and comparison groups did not differ significantly, the mean score of the intervention group was slightly higher. The equivalence of these test results may have been affected by a number of factors. For example, the test was administered at the end of the term as opposed to the end of a particular seminar, which allowed both groups extra time to prepare for the test and to share ideas during this time period. Comparable findings in group process behaviours and self-efficacy with group skills may be reflective of the stability of attitudes that might fit with the developmental level of students. On the whole, the students perceived their abilities and performance to be higher than that observed by investigators.

Greater generation of ideas by the intervention group is consistent with previous studies (Gallupe et al., 1992; Dennis & Valacich, 1993; Valacich et al., 1994). As suggested by Siau (1995), the structure provided by the technology may explain this finding. Students were able to focus attention on key ideas, keep them visible during presentations and return occasionally to previous ideas. Novelty may also be a contributing factor, since the interactive screen, colour, sound and animation appeared to stimulate enthusiasm. Some support for this view is found in the high ratings for SMART Board playfulness (see Table 6).

While ideas were more prolific in the intervention group, higher-order reasoning, such as synthesis and in-depth evaluation, was limited. Instead, there was a tendency to clarify ideas, expand on them and discuss their usefulness. Depth of discussion may have been limited by the short time frame of one-half hour for a presentation and by the nature of the presentation. In this situation, students had already completed individual analyses of the concept under discussion and were now reporting and reflecting on them. It is likely that a different style of reasoning would have emerged if the learning task were unfamiliar and complex.

In conclusion, it appeared that the SMART Board stimulated learning and user satisfaction in a seminar group. However, because of the small sample size, these results should be viewed with caution. Future research should be directed at larger samples and focus on more complex problems that demand higher-order reasoning. It would be useful, as well, to conduct longitudinal studies, which structure critical-thinking exercises and monitor the actual transfer of skills to the workplace.

References
Ager, A. and Bendall, S. (Eds.). (1991). Microcomputers and clinical psychology: Issues, applications and future developments. The Wiley series in clinical psychology. Chichester: John Wiley & Sons.

Baldwin, T. and Ford, J. (1988). Transfer of training: A review of directions for future research. Personnel Psychology, 41, 63-105.

Bandura, A. (1997). Self-Efficacy. New York: W.H. Freeman and Co.

Barling, J. and Beattie, R. (1983). Self-efficacy beliefs and sales performance. Journal of Organizational Behaviour Management, 5, 41-51.

Bloom, B. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: McKay.

Broad, M. and Newstrom, J. (1992). Transfer of training. Reading, MA: Addison-Wesley Publishing Co., Inc.

Carey, J.M. and Kacmar, C.J. (1997). The impact of communication mode and task complexity on small group performance and member satisfaction. Computers in Human Behaviour, 13(1), 23-49.

Champagne, F. (1999). How evidence influences managers. A presentation at Closing the Loop, The third international conference on the scientific basis of health services. Toronto, October 1-3.

Chin, J.P., Norman, K. and Shneiderman, B. (1998). In Norman, K.L, Schneiderman, B.A., Harper, B.D., and Slaughter, L.A. Questionnaire for user interaction satisfaction, Version 7.0. College Park: University of Maryland.

Cohen, V.L. (1997). Implications for learning in a technology-rich school. Journal of Artificial Intelligence in Education, 8(2), 153-174.

Cormier, S. and Hagman, J. (1987). Transfer of Learning. San Diego, CA: Academic Press.

Deadman, G. (1997). An analysis of pupils' reflective writing within a hypermedia framework. Journal of Computer Assisted Learning. 13(1), 16-25.

Dennis, A.R. and Valacich, J.S. (1993). Computer brainstorms: More heads are better than one. Journal of Applied Psychology, 78, 531-537.

Drakeford, C.A. (1996). High school guidance counselling by computers: A study of the Therapeutic Learning Program. Dissertation Abstracts International, Section A: Humanities and Social Sciences. 56(7-A): 2564.

Gallupe, R.B., Dennis, A.R., Cooper, W.H., Valacich, J.S., and Bastianutti, L.M. and Nunamaker J.G., Jr. (1992). Electronic brainstorming and group size. Academy of Management Journal, 35, 350-369.

Gist, M.E, Schowerer, C. and Rosen, B. (1989). Effects of alternative training methods on self-efficacy and performance in computer software training. Journal of Applied Psychology, 74(6), 884-891.

Gilliver, R.S., Randall, B. and Pok, Y.M. (1998). Learning in cyberspace: Shaping the future. Journal of Computer Assisted Learning, 14(3), 212-222.

Griffith, T.L. (1999). Technology features as triggers for sensemaking. Academy of Management Review, 24(3), 472-488.

Guinan, P.J., Cooprider, J.G., and Sawyer, S. (1997). The effective use of automated application development tools. Systems Journal, 36(1), www.research.ibm.com/journal/sj/361/guinan.html.

Halpern, D.F. (1998). Teaching critical thinking for transfer across domains. The American Psychologist, 53(4), 449-455.

Ho, J.L. (1999). Technology and group decision process in going-concern judgements. Group Decision and Negotiation, 8(1), 33-49.

Homrich, A.M. (1997). Effects of Group Support System interactive computer technology on task-group processes of psychology case presentations. Dissertation Abstracts International, Section B: The Sciences and Engineering, 58(6-B), 3316.

Johnson, D.L. (1997). The effect of task difficulty on accounting-based decisions in face-to-face versus computer-mediated group settings. An experimental investigation. Dissertation Abstracts International, Section A: Humanities and Social Sciences 58(4-A) 1357.

Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.

Larsen, C., Dansereau, D., O'Donnell, A., Hythecker, V., Lambiotte, J., and Rocklin, T. (1985). Effects of metacognitive and elaborative activity on cooperative learning and transfer. Contemporary Educational Psychology, 10, 342-348.

Mahmood, M.A. and Medewitz, J.N. (1989). Assessing the effect of computer literacy on subjects' attitudes, values and opinions toward information technology: An exploratory longitudinal investigation using LISREL model. Journal of Computer Based Instruction, 16(1), 20-28.

Newstrom, J. (1986). Leveraging management development through the management of transfer. Journal of Management Development, 5, 33-45.

Norris, S. (1992). Generalizability of critical thinking: Multiple perspectives on an educational ideal. New York: Teachers College Press.

Norman, K.L., Shneiderman, B.A., Harper, B.D., and Slaughter, L.A. (1998). Questionnaire for user interaction satisfaction, Version 7.0. College Park: University of Maryland.

Ocker, R.J., and Yaverbaum, G.J. (1999). Asynchronous computer-mediated communication versus face-to-face collaboration: results on student learning, quality and satisfaction. Group Decision and Negotiation, 8(5), 427-440.

Phillips, R.J. and Pierson, A. J. (1997). Cognitive loads and the empowering effect of music composition software. Journal of Computer Assisted Learning, 13(2), 74-84.

Raatz, S. (1993). Collaborative computing. The CPA Journal Online. http://www.nysscpa.org/cpajournal/old/14628716.htm.

Reid, F.J.M., Ball, L.J., Morley, A.M. and Evans-Jonathan, S.B.T. (1997). Styles of group discussion in computer-mediating decision making. British Journal of Social Psychology, 36(3), 241-262.

Schacter, D.L. (1987). Implicit memory: History and current status. Journal of Experimental Psychology: Learning, Memory and Cognition, 13, 501-518.

Siau, K.L. (1995). Group creativity and technology. Journal of Creative Behaviour, 29(3), 201-216.

Webster, J. and Martocchio, J.J. (1992). Microcomputer playfulness: Development of a measure with workplace implications. MIS Quarterly, 15, 201-226.

Wegerif, R., Mercer, N. and Dawes, L. (1998). Software design to support discussion in the primary curriculum. Journal of Computer Assisted Learning, 14(3), 199-211.

Valacich, J.S., Dennis, A.R., and Connolly, T. (1994). Idea generation in computer-based groups: A new ending to an old story. Organizational Behavior and Human Decision Processes, 57(3), 448-467.

Yang, S.C. (1999). Divulging intertextual processes in the problem-solving Hypermedia. Journal of Interactive Learning Research, 10(1), 87-120.

Biographies

At Queen's for 22 years, she has duties that include overseeing the ongoing appropriate adaptation of computer technology in the classroom and in learning at Queen's, and the training of professors on new technology to improve teaching. She is in charge of the Learning Technology Unit (LTU), where professors can come to use computer equipment and software for instructional purposes.

Back to Introduction
Back to Project Library