Vol. 16, No. 4, December 2000

Special Issue: approaches to the design of software training

Guest Editors: M. Bannert & P. Reimann, University of Landau and University of Heidelberg

Guest Editorial

M. Bannert & P. Reimann, Universität Kobelenz-Landau und Heidelberg

Email: bannert@rhrk.uni-kl.de

According to software producers, modern computer application programs are not only becoming more and more user-friendly but also easier to learn to use. However, the complexity of applications’ software is increasing dramatically and frequent updates of products often require the (re-)learning of program usage. Consequently, for the majority of computer users, a minimum of training is an indispensable prerequisite for the effective use of software

But how should users be supported when learning to use a software product and how should instructions be designed in order to achieve effective software training? To date, only a few psychological studies have investigated these research questions in any systematic fashion. However, various fields of psychology can and should make contributions to this issue, which will not only allow the formulation of recommendations and guidelines for instructional approaches to software training, but also provide significant knowledge concerning theoretical and educational issues beyond software training.

Hence, the objective of this Special Issue is to discuss this new topic of psychological research from a theoretical and methodological perspective based on empirical studies. The focus lies on the learning of application programs and the design of adequate instructional support, and the implications for software training practice. The acquisition of programming languages is not considered.

Most papers emerged from the Symposium ‘Instructional Approaches and Design of Software Training’ which was organised at the EARLI Conference 1999 in Gothenburg, and from the poster session about software training held at the DGP-Conference 1998 in Germany. The selection of original research reports covers a broad spectrum of issues, in which each contribution views the topic from the perspective of computer-based learning and instruction.

Martin-Michiellot and Mendelsohn investigate design recommendations for computer manuals based on the Cognitive Load Theory. In particular, they analyse whether different instruction formats affect learning time and performance due to induced cognitive load. The main result of their study is that learning to use a CAD-package with a so-called juxtaposed or an integrated manual — both presented without a computer — takes about half the time when compared to a conventional manual — presented with a computer –— without any loss in learning performance.

A similar topic is treated by Van der Meij who focuses on captured screen images in software documentation. In his experimental study, which also addressed Cognitive Load Theory, problems related to splitting attention between manual, screen, and input device were investigated when learning the use of software with manuals. As expected, the inclusion of screen images in the software documentation for Windows 95 influenced learning time and learning outcome. Van der Meij proposes a taxonomy for screen images with four crucial design dimensions in order to help designers of software manuals to improve upon the use of screen images and in order to guide further research on this issue.

Dutke and Reimer analyse the design of on-line help-systems to support task-based system-exploration for which two distinct cognitive phases are assumed: schema acquisition and mental model building. The aim of their study was to assess the effects of on-line help formats on learning performance. In two studies using graphics software similar to ‘Paint Brush’ they varied the help format: Operative help, which includes a list of actions which have to be carried out in order to reach the goal, should be best suited for schema acquisition; function-oriented help, which gives the explanation about how the specific function works, should be especially appropriate for mental model building. Experimental results confirmed that learning performance was better when using operative help in the context of solving similar tasks. However, when the type of task changes, users exploring the system with function-oriented help were better.

Reimann and Neubert present an exploratory study in which the use of a variety of information resources (i.e. the user interface and program itself, manuals, on-line help, or examples provided in the manuals) by novice learners was investigated. In particular, they asked how novices make use of the worked-out examples often provided in manuals and during training. Based on earlier research on the self-explanation effect, thinking aloud data from novices obtained when learning to use a spreadsheet program were analysed with respect to examples elaboration and utilisation during problem solving. Important effects of self-explaining comparable to findings in other domains were found.

The aim of Lazonder’s paper is to identify users’ needs for learning web-based information seeking. Searching in the Web is described by a process model including the stages: goal formation, strategy selection, strategy execution, and monitoring. Information search processes of novice and expert users were analysed in a qualitative case study. In particular, their actions and verbalisations were recorded when performing three Web search tasks. Minimal performance differences between novices and experts were identified. The results indicate that users should be instructed in monitoring skills and system knowledge when learning information searching.

Bannert reports on an investigation into the effectiveness and feasibility of individual learners’ support in software training. Text-processing courses in real classroom settings were analysed in a quasi-experimental study using two kinds of user interface. Half of the courses used a conventional software interface, whereas the other half used a training wheels interface in which all irrelevant functions were blocked. In half of the courses all instruction was given by a human tutor whereas the participants in the other half learned with printed self-learning material. Results indicate that participants using the ‘training wheels’ learned significantly faster than those using the ‘standard interface’. However, they did not have better learning outcomes. Students in the ‘self-learning material’ condition learned significantly faster and achieved significantly better learning outcomes than students in the ‘human tutor’ group. No group differences in learner satisfaction occurred.

The training wheels approach was also investigated in Leutner’s study and the paper presents the so-called double-fading support approach (DFS) to software training. Two types of user support when learning a complex software system were faded out gradually during the course: locking software’s functionality and detailed guidance. Two experiments investigated if both fading-out support measures do lead to increased problem solving achievement when practice problems with a CAD system were solved. Results indicate the effectiveness of the DFS-Approach for an deeply structured CAD-menu. As expected, participants working with the initially reduced software outperformed participants from the full functionality software group; additionally, participants of the slowly faded guidance group outperformed participants from the other fading guidance-groups. However, less effectiveness of the DFS-approach was obtained for an icon-based CAD-system. Here, it seems that the two factors of user support over-compensated each other.

Wiedenbeck, Zavala and Nawyn analyse the effects of different practice methods in exploration-based software training. The aim of their study is to understand the effects of different hands-on practice methods by analysing the activities undertaken during training. Three practice methods were varied by using an exercise, an exploration, and a combined training approach for learning Hypercard. Effects of these methods were analysed with respect to learning time and learning activities. The outcomes of the study indicate that participants of the exploration-based method failed to practice and to repeat the basics of the materials and failed to extend their knowledge. The lack of systematic repetition and going beyond the information given may have led to subsequent difficulties in skill acquisition of entirely exploration-based training approaches. The study points out the necessity of conducting process analysis which is often ignored in training studies. In addition, and more important for the topic of this Special Issue, the study demonstrates the limits of entirely exploration-based learning. Most learners were not able to explore the software totally on their own in such a way that they were able to use it effectively. Thus, instructional approaches to software training are nevertheless important measures for software users — despite the increasing usability of modern software products.

This issue concludes with comments by Van Merrienboer who classifies the papers on the basis of a general Instructional Design Approach. His review distinguishes papers mainly concerned with task and content analysis (‘What to teach’) from papers focusing on design and selection of instructional methods (‘How to teach’) and from papers dealing with instructional message design (‘How to present what is taught’). Based on this classification Van Merrienboer points out three recent trends in software training: from narrow function-oriented learning tasks to rich task-oriented learning problems in real-life contexts; from purely procedural description to scaffolding for deep system understanding; and from low-load instruction formats to instructions that induce and support deep information elaboration. Finally, he concludes that software training should no longer be treated as a separate field of instructional research and design, but rather should be integrated in training measures of professional skills.

Finally, we want to thank all our contributors for their co-operation when working on this Special Issue and meeting its deadlines. Also, Jeroen van Merrienboer helped us considerably in the editorial process and with his insightful comments. Furthermore, we wish to thank Bob Lewis, for not only providing us with the opportunity to publish our topic as a Special Issue of JCAL, but also for supporting us throughout the entire process. He and the contributors made this work professionally satisfying — it was really a pleasure for us working with all of them.

Journal of Computer Assisted Learning (2000) 16, 4, 281-283.

Cognitive load while learning with a graphical computer interface

S. Martin-Michiellot & P. Mendelsohn, Technologies de la Formation et de l’Apprentissage, Université de Genève

Email: Silvere.Martin-Michiellot@tecfa.unige.ch

Computer software packages are hard to learn but little is known about how to help new users. A study was made in which 30 learners had to learn the basic commands of a CAD software package using manuals with different formats of presentation of instructions. Ten learners learned using a conventional manual plus a computer, 10 learners learned using a manual with juxtaposed screen images but no computer and 10 learners learned using a manual with integrated screen images but no computer. Each learner was tested with a practical and a written test and the learning time was measured. There was no statistical difference between the groups of learners for the score of the practical and written tests. However, it was found that learners using a manual with juxtaposed or integrated screen images had learned twice as fast as learners using a conventional manual. The discussion questions the use of conventional manuals to learn application software and stress as the key role of the graphical environment in the learning process.

Keywords: Application software; Cognitive load; Computer; Comparative study; Instruction; Interface; Visual manuals; Undergraduate

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 284-293.

The role and design of screen images in software documentation

H. van der Meij, Department of Instructional Technology, University of Twente

Email: meij@edte.utwente.nl

Software documentation for the novice user typically must try to achieve at least three goals: to support basic knowledge and skills development; to prevent or support the handling of mistakes, and to support the joint handling of manual, input device and screen. This paper concentrates on the latter goal. Novice users often experience split-attention problem due to the need to (almost) simultaneously attend to different media. Existing research indicates that split-attention problems can be prevented or reduced by the presence of screen images in the manual. Research is yet unclear about the optimal design of these pictures. This study examines three design styles. Forty-eight novice users received one of the three manual based on these styles. The manuals were an introduction to Windows 95. The users of the most successful manual needed 25% less training time and had a 60% better retention. The most important characteristics of the design style of this manual were its use of full screen images (instead of partial ones) and a two-column lay-out in which the instructions and screen images were presented side-by-side in a left-to-right reading order. The discussion focuses on the tension that exists between theory and practice. Special attention is given to the contributions of a taxonomy of screen images and cognitive load theory.

Keywords: Application software; Cognitive load; Screen images; Training; Tutorial; Visual manuals

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 294-306.

Evaluation of two types of online help information for application software

S. Dutke. & T. Reimer, Freie Universität Berlin and Universität Potsdam

Email: dutke@rhrk.uni-kl.de

A recent trend in application software design is to extend online help systems in order to support exploratory and self-paced learning. Two different information formats, lists of action steps that have to be taken to achieve a goal (operative help) and explanations about how a function works (function-oriented help), were evaluated to assess their effects on learning performance. In two experiments, adult computer novices (N = 60 and N = 20) learned to use experimental graphics software by task-based exploration. After one half of the tasks, the type of tasks changed so that the acquired action schema could no longer be applied. Results indicated that participants who had access to function-oriented help were coping more successfully with the schema change than participants who received operative help. The relevance to the design of software training and online help systems is discussed.

Keywords: Application software; Control group; Exploratory learning; Individual; Mental model; Online help; Schema; Training

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 307-315.

The role of self-explanation in learning to use a spreadsheet through examples

P. Reimann & C. Neubert, University of Heidelberg & UBS Bank

Email: peter.reimann@urz.uni-heidelberg.de

This papers describes an exploratory study into the early phase of getting to know end-user software during which users make use of a variety of information resources, including the user interface/program itself, manuals, on-line help, examples provided in the manuals and other sources. In particular, how do novices make use of the worked-out examples often provided in manuals and during training? Building on earlier research on the self-explanation effect, thinking aloud data from 10 participants were analysed to see how examples were studied and how they were used during problem solving. Important effects of self-explaining comparable to findings in other domains were found in this study. For instance, those participants who self-explain with the goal to discover meaning prove to be better problem solvers than those who do not self-explain or who focus more on syntactical aspects of examples.

Keywords: Application software; Discovery learning; Examples; Human–computer interaction; Self-explanation; Spreadsheets; Training

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 316-325.

Exploring novice users' training needs in searching information on the WWW

A.W. Lazonder, Department of Instructional Technology, University of Twente

E mail: lazonder@edte.utwente nl

Searching for information on the WWW involves locating a Website and locating information on that site. A recent study implied that novice users’ training needs exclusively relate to locating Websites. The present case study tried to reveal the knowledge and skills that constitute these training needs. Fourteen pre-university students, classified as novice (n = 7) or experienced WWW-user (n = 7) performed three Web search tasks. Their actions and verbalisations were recorded. Between-group comparisons showed minimal performance differences. However, qualitative analyses of novice users’ search performance suggest that their ability to locate Websites could be enhanced by instructing monitoring skills and advanced system knowledge.

Keywords: Application software; Case study; Information literacy; Information systems; Internet; Post-secondary; Qualitative; Training

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 326-335.

Effects of training wheels and self-learning materials in software training

M. Bannert, University of Landau

Email: bannert@rhrk.uni-kl.de

The role of software training is becoming increasingly important due to the grow-ing variety and complexity of modern software products. This paper focuses on the effectiveness and feasibility of two different kinds of individual learners' support in software training in classroom settings. This research question was investigated by analysing 11 text-processing courses which were conducted on the basis of a quasi-experimental research design. Firstly, the user interface was varied, i.e. half of the courses used the regular standard user interface of the software, whereas the other half used a training wheels interface in which all irrelevant functions were blocked. Secondly, in half of the courses all instruction was given by a human tutor whereas the participants in the other half received written self-learning material. A total of 72 university students majoring in different fields participated. Dependent measures were learning time, learning outcome, and learner satisfaction. Participants from the 'training wheels' condition learned significantly faster compared to the partici-pants of the 'standard user interface' treatment. However, they did not have better learning outcomes, and moreover, they did not accept the training wheels interface. Students in the 'self-learning material' condition learned significantly faster and achieved significantly better learning outcomes than students in the 'human tutor' group, whereas no group differences in learner satisfaction occurred.

Keywords: Applications software; Discourse analysis; Experimental; Individual; Self-learning materials; Training; Training wheels

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 336-346.

Double-fading support - a training approach to complex software systems

Detlev Leutner, Erfurt University of Education, Germany

Email: leutner@ipsych.ph-erfurt.de

A new approach to software training is presented, the so-called Double-Fading Support (DFS) approach. According to this approach, which is based on Carroll’s training-wheels idea and on cognitive theories of skill acquisition, two types of user support when learning to use a complex software system — locking the software’s functionality and detailed guidance — are faded out gradually during the training course, so that the learners are able to use the complex software with minimal instructional support at the end of the training. Two 30-hour training experiments with two different CAD software systems and CAD-inexperienced university students were conducted. The results of Experiment 1 with 88 participants indicate the effectiveness of the DFS-approach for CAD software with a deeply structured menu system. Participants working with the initially reduced software outperformed participants of the full software functionality group; additionally, participants of the slowly faded guidance group outperformed participants receiving medium, fast or no fading of guidance at all. Results of Experiment 2 with 120 participants, however, indicate less effectiveness of the DFS-approach for an icon-based CAD software in which most of relevant functions are permanently visible to the user. It seems that the two factors (fading out the locking of software’s functionality and fading out detailed guidance) overcompensate each other.

Keywords: Application software; Empirical evaluation; Help systems; Training; Tutorial; Undergraduate

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 347-357.

An activity-based analysis of hands-on practice methods

S. Wiedenbeck, J.A. Zavala. & J. Nawyn, University of Nebraska

Email: susan.wiedenbeck@drexel.edu

The success of exploration-based training is likely to be strongly influenced by what activities the learner undertakes during training. This paper presents a study of the activities undertaken during training by 51 experienced computer users learning to use an application package through exercises, exploration or a combined approach to training. Results suggest that exploration learners practice procedures selectively, fail to consolidate skills through repetition, and do not devise activities which extend their knowledge beyond the scope of the training materials. It is argued that these characteristics may lead to subsequent difficulties in performance.

Keywords: Application software; Exploration; Exercises; Minimalist training; Training; Undergraduate

This paper is one of a selection of papers derived from a symposium at the EARLI conference in 1999 and from a poster session at the Deutsche Gesellschaft für Psychologie conference in 1998.

Journal of Computer Assisted Learning (2000) 16, 4, 358-365.

The end of software training ?

J.J.G. van Merrienboer, Open University of the Netherlands

Email: jeroen.vanmerrienboer@ou.nl

The contributions to this Special Issue are discussed within an Instructional Design framework. A distinction is made between contributions concerned with task and content analysis, design and selection of instructional methods, and instructional message design. Three major trends indicate a shift in focus:

• from narrow learning tasks for operating the software per se to rich learning tasks for using the software in the context of meaningful task performance;

• from methods that stress procedural descriptions and related exercises to methods that stress guided exploration and scaffolding and so aim at deep system understanding, and

• from low-load presentation formats to formats that also evoke active processing and elaboration of presented information.

These trends indicate the end of software training. It has grown up and is no longer fundamentally different from regular professional skills training.

Keywords: Application software; Individual; Interface; IT-use; Manuals; Screen images; Training

Journal of Computer Assisted Learning (2000) 16, 4, 366-375. Invited paper.