BRADLEY BREN CAHOON

Computer Skill Learning in the Workplace: A Comparative Case Study

(Under the direction of BRADLEY C. COURTENAY)

Twelve computer users in three organizations were interviewed in a qualitative study of how adults learn personal computer skills in the context of workplace activities. Transcripts and observational data were used as the basis for organizational case studies; a cross-case analysis identified general categories, events, and processes of individual, workgroup, and organizational learning.

Generally, individual learning began with a social situation requiring adaptation to computers. The most skillful participants progressed to adapting computer resources to work requirements. The transition from novice to experienced user, a process that a learner may enact many times, depended on asking questions, self-directed learning, learning more than one software application or computer system, and adapting computer resources to work requirements.

All participants indicated that informal learning through mutual problem solving and coaching had been more important in their skill development than formal training. Each workgroup included one or more individuals identified by peers as local experts, on whom the group relied for support, and negotiated implicit or explicit rules governing the distribution of computer work and skills. In some groups, these rules encouraged self-directed learning to reduce demands on local experts. In other workgroups, most computer tasks were delegated to local experts, with other members exhibiting little self-directed learning. These results were consistent with prior research on situated learning (Lave & Wenger, 1991).

Organizational expectations and support appeared to motivate self-directed learning and skill development. Members of bureaucratic service organizations developed less skill than members of a flatter, more profit-oriented organization. Informal learning was also influenced by conflicts between formal organizational structures and informal networks of computer users.

Recommendations for computer training include extending the duration of workshops, focusing on rehearsal of work tasks rather than enumeration of software features, training workgroups together, and involving participants in program planning. Alternatively, instructional technology labs can support self-directed learning and reduce conflicts in motivation by distancing learning goals from immediate work goals. Future research in this area could focus on quantitative assessments of transfer of training and the development of network-based systems to support collaborative learning.

COMPUTER SKILL LEARNING IN THE WORKPLACE:

A COMPARATIVE CASE STUDY

by

BRADLEY BREN CAHOON

B.A., The University of Georgia, 1983

M.A., The University of Georgia, 1985

A Dissertation Submitted to the Graduate Faculty

of the University of Georgia in Partial Fulfillment

of the

Requirements for the Degree

DOCTOR OF EDUCATION

ATHENS, GEORGIA

1995

© 1995, 1998

Bradley Bren Cahoon

All Rights Reserved

ACKNOWLEDGEMENTS

The research presented in this dissertation could not have been conducted without the support, encouragement, and cooperation of many people. I am especially grateful to the individuals and organizations who participated in the study and to the hundreds of computer training participants at the Georgia Center for Continuing Education whose experiences led me to examine this problem. Helen Mills and the other members of my department were unfailingly supportive during my research. Bradley C. Courtenay and the other members of the advisory committee–Ron Cervero, Sharan Merriam, Michael Orey, Edward G. Simpson, and Karen Watkins–improved this study greatly through their comments, criticisms, and insights. Other faculty and fellow graduate students provided valuable feedback throughout the evolution of the research. Joe Cafiero read various drafts and participated in a long-running e-mail dialogue that helped shape my thinking about the subject.

My wife, Lynn Cahoon, contributed to this study on many levels: assisting with transcriptions, proofreading, making editorial recommendations, discussing her own learning experiences, and listening patiently as I tried to explain to her and myself what I was looking for and what I found. For these reasons, and for her loving kindness, this dissertation is dedicated to her.

TABLE OF CONTENTS

Page

ACKNOWLEDGEMENTS iv

CHAPTER

1 INTRODUCTION

Context of the Problem 1

The Problem 3

Purpose of the Study 5

Significance of the Study 6

Definitions of Terms 8

2. REVIEW OF THE LITERATURE

Introduction 14

Quantitative Studies 15

Qualitative Studies 27

Organizational Studies 34

Situated Learning 40

Informal and Incidental Learning 45

3. METHODS

Introduction 50

Design of the Study 51

Population and Sampling 54

Data Collection 57

Data Analysis 60

Strengths and Limitations 68

4. FINDINGS

Case 1: Leisure Activities Department 72

Case 2: Outpatient Therapy Department 75

Case 3: Regional Publishing, Inc. 84

Cross-Case Analysis 90

5. DISCUSSION AND RECOMMENDATIONS

Summary 124

Discussion 125

Recommendations 135

REFERENCES 147

APPENDICES 156

CHAPTER I

Context of the Problem

Fifteen years ago, personal computers were a technological novelty. Now they are pervasive in organizations of all kinds and sizes, as much a part of the fabric of working life as coffee cups and telephones. Popular images of "the computer" in the mass media have shifted from monolithic mainframes attended by white-coated technicians to desktop systems that ordinary people use to work, play, and communicate. Personal computers are already so common that they are achieving a degree of invisibility. Just as electric lights, automobiles, and televisions have become unquestioned parts of day-to-day life in our culture, so personal computers are beginning to be taken for granted, almost as if they had always existed.

Many factors account for the rapid adoption of the personal computer. Perhaps the most obvious is the astonishing rate at which computers have become more cost-efficient. The performance of microprocessor and memory chips has improved exponentially since 1970, while at the same time the cost of computers has declined by half every two to three years (Brynjolfsson, 1993, p. 72). Constant innovations in design and manufacturing fuel a personal computer industry that shows every sign of continuing its explosive growth for decades to come (Cringely, 1992; Freiberger & Swaine, 1984; Ryan, 1990).

The potential for continuous improvement in price/performance ratios is inherent in integrated circuit technology, as miniaturization allows more transistors to be packed into smaller chips. Personal computer development has been further accelerated by consumer demand for more and better systems, as organizations purchase and install computers at a steadily increasing rate (Brynjolfsson, 1993). This hunger for computing power reflects global economic trends, particularly the increasing importance of information in the creation of wealth (Toffler, 1990). More than ever, the success and survival of organizations depends on their ability to obtain, process, and distribute time-critical information. For this reason, the personal computer has become a central tool of the modern workplace.

However, installing personal computers is not sufficient to achieve organizational goals. The ubiquity of the personal computer does not indicate that it has been assimilated painlessly into working life or that its presence has transformed every employee into a Tofflerian "knowledge worker." As Brynjolfsson (1993) notes in a review of the "productivity paradox" literature, "Delivered computing power in the U.S. economy has increased by more than two orders of magnitude since 1970, yet productivity, especially in the service sector, seems to have stagnated" (p. 67). He describes 17 empirical studies that report lower-than-expected returns on computer investments in services, manufacturing, and other sectors of the economy.

Brynjolfsson identifies four possible explanations for these findings: problems of measuring productivity accurately; the possibility that computerization tends to redistribute market share among firms rather than increasing total economic output; poor managerial decisions about computer acquisitions; and time lags between current investments and eventual benefits. The last of these dilemmas – the inevitable delay between an organization’s acquisition of computers and its members’ effective use of those computers – is closely linked to the complex problem of computer skill learning.

To realize the potential of personal computing, organizations have to ensure that workers actually use their computers, that they develop the levels of skill necessary for their jobs, and that they are able to adapt to ongoing innovations in hardware and software. Thus the pressures on working adults to learn about computers are increasingly urgent. So far, however, advances in this area of training and education have come much more slowly than advances in the technology itself.

The Problem

In spite of manufacturers’ claims of "user friendliness," many adults find that learning to use a personal computer is time-consuming, frustrating, and charged with anxiety (Filipczak, 1994; Howard, 1994). Unlike other technologies that have seen similarly rapid adoption, such as the telephone, the computer is not designed for one specific task. It is adaptable through programming to almost any activity, and this generality makes it more conceptually abstract. The physical design and environmental settings of personal computers rarely provide clues about their use, and the diversity of software means that skills learned on one program may not transfer to another. Considering the pressing need of organizations and individuals to meet these challenges, surprisingly little appears to be known about computer skill learning.

The majority of publications on computer skill learning (e.g., articles in newspapers and popular magazines, how-to books) are based on expert opinion or the promotion of specific products. Their proliferation provides some indirect evidence about the attitudes of adult learners; for example, the recent popularity of best-selling books with titles such as DOS for Dummies (Gookin, 1993) suggests that many book-buyers identify with the image of computer learners as "dummies" and may welcome a humorous approach to an anxiety-provoking situation, even as they align themselves with the targets of the humor. While content analysis of such works could prove an interesting study in itself, these books and articles offer little theoretical insight into the cognitive and social processes of computer skill learning.

A review of the scholarly literature reveals many empirical studies related to this subject, but so far no coherent theoretical basis for computer training appears to have emerged. One reason for this lack of theoretical support for educational practice is that research on computer skill learning is scattered across several disciplines, and scholars in one area often overlook related work published outside their own fields. In a helpful but incomplete review of the literature on computer skill acquisition, management professor Urs Gattiker observes that "limited cross-feeding and integration of research results is occurring between work in education and training, personnel psychology, and ergonomics, as well as in cognitive psychology" (Gattiker, 1992, p. 570). Gattiker’s review suffers from similar limitations, overlooking relevant studies in human-computer interaction (HCI), artificial intelligence, and anthropology.

As discussed in greater detail in Chapter II, the empirical literature on computer skill learning provides intriguing data and some powerful explanatory concepts, but often raises more questions than it answers. For example, a number of experimental studies suggest that popular training methods such as classroom instruction, computer-assisted instruction (CAI), and manuals are not consistently effective (e.g., Carroll, 1990; Carroll & Mack, 1984; Czaja et al., 1986 & 1989). Searches of several computerized databases, including ERIC, PsycINFO, Business Index, Current Contents Plus, and SocioAbstracts, found no studies that tried to assess the extent to which computer training transferred to actual work practices. One experimental study indicates that transfer between computer tasks may be much more limited than is assumed by most instructional designs (Singley & Anderson, 1989). Survey-based and ethnographic studies of workplace computer use suggest that managerial support for training tends to be minimal (Bikson, 1987; Olsten Forum for Information Management, 1993) and that workers rely more heavily on informal support from colleagues than on formal training and support for developing skills and solving problems (Bullen & Bennett, 1991).

Given the paucity of the research base, the data suggesting that computer training is limited in both availability and effectiveness, and the growing need to help adults develop computer skills, a critical problem emerges. How do adults actually construct the computer skills they use in the workplace, and how can this process be facilitated?

Purpose of the Study

The purpose of this study was to examine how adult learners develop personal computer skills in the context of workplace activities. This goal required investigating learning as adults experience it rather than under controlled experimental conditions, since what is at issue is adults’ adaptation to the practical demands and constraints of computing in organizational settings (Brown & Duguid, 1991 & 1992). The following research questions were used to explore this problem:

Significance of the Study

A better understanding of the ways adults experience computer skill learning could provide a theoretical basis for the planning, design, and delivery of improved computer instruction. Prior research suggests that cognitive and affective factors such as conflicts in motivation exert a strong influence on success or failure in skill learning (e.g., Carroll, 1990; Elliott & Dweck, 1988). Literature searches discovered only one prior study based on in-depth interviewing of adults about their subjective responses to computer skill learning (Howard, 1994). The present study identifies general categories that provide a context within which experimental research could be reassessed, synthesized, and adapted to the requirements of educational practice.

By focusing on individual histories of skill learning rather than on specific instances or techniques of training, this study points toward facilitative interventions that fall outside the scope of current computer training. For example, if informal peer interactions influence learning more strongly than classroom instruction, new organizational strategies may be necessary to shift instructional responsibilities from HRD departments or contract trainers to an informal network of "local experts" within the organization (cf. Brown & Duguid, 1991 & 1992; Lave & Wenger, 1991; Marsick & Watkins, 1990).

Current demographic and economic trends suggest that the significance of computer skill learning will increase. Large numbers of workers are being displaced through corporate downsizing; at the same time, the skill requirements for new jobs are becoming more complex and information-intensive. By 1989, more than 75 percent of the Americans who will be in the workforce in the year 2000 were already working (Miller, 1989). Younger workers cannot replace older ones each time skill requirements change; retraining and continuing education are the only cost-effective means of filling highly skilled jobs. Moreover, the pace of technological innovation is so rapid that those who already have computer skills must continuously learn new ones as organizations upgrade to new hardware and software. Facilitating computer skill learning may be the most economically important area of adult and continuing education during the next 10 to 20 years. Improvements in this area can help organizations achieve better returns on computer investments and can also empower workers to participate more productively in the workplace and exercise greater leverage in the labor market.

The potential significance of this study extends beyond current personal computer systems, if these are viewed as merely one example of technological innovation in the workplace. Other information technologies, some scarcely imaginable now, are likely to continue to transform working life. Already fax machines, pagers, modems, notebook and hand-held computers, local- and wide-area networks, and video teleconfencing are extending the concept of the "workplace" to include virtual communities whose members work together but rarely share the same physical space or time. For these early settlers of the electronic frontier, ongoing adaptation to technology is a fact of life. Moreover, such adaptation has become a defining activity of many professions (e.g., software developers, providers of information services, designers of interactive media, engineers, technology consultants, and researchers). A better understanding of computer skill learning will be valuable in easing the migration to a future in which technology is no longer part of the workplace, but in a sense has become the workplace.

Definitions of Terms

The diversity of systems, users, and organizational contexts makes computer skill learning an extremely broad topic. Below, "personal computer," "applications," "novice," "experienced user," and "computer skill" are defined for the purposes of this study.

A personal computer as considered here is a small, self-contained computing device with a video display, keyboard, and at least one disk drive, designed for use by one person. Personal computers have often been called "microcomputers," but this term fails to suggest that the unique quality of these systems is their use by individuals. This study focuses solely on desktop systems that have achieved widespread market acceptance. "Personal computer" as used here will refer either to IBM PC-compatible systems (i.e., computers using microprocessors based on the Intel 80x86 architecture) or Apple Macintosh computers. An informal review of industry publications such as InfoWorld and PC Week indicates that these two platforms account for all but a very small percentage of the personal computers in current use. Unless otherwise indicated, the terms "computer" and "personal computer" will be used synonymously.

The popularity of personal computers is largely a result of the commercial availability of personal productivity software or applications. Applications are computer programs designed as tools for word processing, desktop publishing, the creation of databases and spreadsheets, and other typical tasks. Unlike specialized "vertical" software, such as an airline reservation system or a point-of-sale system, applications are generic products that are commonly used in a wide range of situations; their functions are not defined by the specific information processing needs of a single organization. For example, skills in word processing or spreadsheet manipulation are more likely to transfer from one work setting to another than are skills in operating a corporation’s customized inventory system. McLean, Kappelman, and Thompson (1993) report survey results that indicate that working with spreadsheets, graphics, word processing, databases, desktop publishing, and electronic mail are the most common activities of personal computer users in organizations. Many empirical studies of computer skill learning have focused on word processing programs, which are the most common applications and often serve as a paradigm for personal computing in general.

Novice computer users are those whose ability to perform work-related tasks with personal computers and applications software is constrained by lack of knowledge or experience, while experienced users have achieved a level of skill that allows them to perform such tasks routinely. The distinction between novices and experienced users is central to this study; therefore, a working definition of skilled computer performance is necessary to recognize and evaluate the learning that occurs in this transition. For that reason, an extended discussion of computer skill follows, drawing on theoretical constructs from prior research on human-computer interaction to operationalize the concept in terms of specific behaviors and competencies.

Colley and Beech (1989) note that "activities are said to be skilled when the performance of them has reached a level where it appears to be effortless, where it is almost always accurate and where additional practice makes little apparent improvement" (p. 1). Skilled performance is goal-directed, uses feedback to correct errors, and is improved by practice. Colley and Beech distinguish perceptual, motor, and intellectual (i.e., cognitive) skills:

Personal computing performance requires the development of skills in all three of these categories. Computer novices must learn to filter their perceptions of the information on the display screen to identify important details; they must use motor skills such as typing and mouse operation to respond to these perceptions; and they must learn to interpret their perceptions in order to choose appropriate responses. Thus, in making the transition from computer novices to computer users, learners practice several types of skill simultaneously.

The effortlessness and accuracy of expert performance indicate that some computer skills are habitual or automatic. Examples include typing, in which the ability to locate keys becomes tacit with sufficient practice, and word processing, in which experienced users can perform complex actions such as deleting a sentence without conscious awareness of the individual steps involved. Such tacit knowledge is described as procedural, to distinguish it from conceptual or declarative knowledge; declarative knowledge is learned and expressed symbolically (for example, as facts or lists) while procedural knowledge is expressed by performing an action. Some aspects of procedural learning can be represented quantitatively, particularly the relationship between the time required to perform a task and the time that has been spent practicing it. According to Wærn (1989), who reviews quantitative studies of computer skill learning,

This function is often graphed as the well-known ‘learning curve.’

However, learning to use a computer involves much more than developing procedural skills, as is evident when a procedure fails to produce the expected results. Problem solving in such a situation requires the skillful use of declarative knowledge about the system. In the words of Wærn, this involves "learning the characteristics or nature of the problem space as well as learning how to search efficiently within it" (Wærn, 1989, p. 82). In computer performance and many other domains, experts appear to possess highly-organized knowledge about the problem space that allows them to reason their way toward solutions even when they lack sufficient information to choose among procedures (Bédard & Chi, 1992).

Cognitive scientists have attempted to account for this kind of performance by positing the existence of mental models that allow reasoning by simulation (Gentner & Stevens, 1983; Greeno, 1989). HCI researcher Donald Norman describes interaction with a computer as a cyclical process of execution and evaluation, in which the user must form an intention, decide how to execute it, act, and then evaluate the resulting state of the computer system. The ability to sustain this process depends on the user’s mental model of the system, which focuses attention on relevant aspects of the situation and allows the construction of explanations about the system’s behavior (Norman, 1986).

Research on procedural knowledge and mental models is helpful in operationalizing the concept of personal computer skill along several behavioral dimensions:

• Efficiency: In performing a task, novices are more likely to make mistakes and to perform actions unrelated to task goals, while experienced users tend toward optimal efficiency.

• Speed: Novices must think through each step of a procedure, while experienced users have automated many procedures and can execute them rapidly without awareness of individual steps.

• Attention: While novices are often distracted by irrelevant aspects of the system’s state, experienced users can focus their attention on salient details.

• Parallel processing: The efficiency of novices declines steeply when they attempt more than one task, while experienced users are often able to sustain high levels of performance on simultaneous tasks.

• Problem solving: Novices are frequently unable to recover from mistakes or deal with unfamiliar events, while experienced users demonstrate lower error rates and stronger problem-solving skills.

• Transfer: Beginners have difficulty in transferring skills to new systems and programs, while experienced users have a greater ability to interpret unfamiliar tasks in terms of similarities to previous experiences.

These differences between skilled and unskilled performance suggest both possible objectives for computer instruction and criteria for evaluating its outcomes. However, the processes through which novices become skillful users remain obscure. The studies described in Chapter II attempt to illuminate these processes with varying degrees of success.

CHAPTER II

Introduction

As yet, computer skill learning has received little attention in the literature of adult education. Rachal (1993) and others have focused on the use of computer-assisted instruction (CAI) to support adult basic education and teaching in other content areas. However, the use of computers as a medium for the delivery of instruction is tangential to the problem of learning workplace computer skills. Most relevant publications have been dated by the recent evolution of the technology (e.g., Gerver, 1984 & 1986; Heermann, 1986).

The purpose of this review is to describe and evaluate empirical research on computer skill learning, in order to clarify what is already known about the cognitive processes and social situations underlying this learning and to provide a theoretical basis for this study. With a few exceptions, all the studies cited here are data-based, have been published within the last decade, and deal with commercially-available personal computers and software. These criteria have allowed the inclusion of experimental studies, ethnographic observations, interviews with and surveys of personal computer users, verbal protocol analyses, and organizational case studies.

The first section of this chapter reviews quantitative studies of individual computer skill learning, most of which are based on experimental or quasi-experimental comparisons of various instructional treatments. However, methodological problems limit the application of many of these studies to educational practice.

The second section reviews qualitative studies employing verbal protocol analysis and phenomenological interviewing, which present a different and potentially more practical view of individual skill learning.

The third section assesses organizational studies, presenting data on the social contexts in which most adults learn to use computers. The concluding sections discuss the development of computer skills in terms of theories of situated learning and informal and incidental learning, which provide the conceptual framework for this study.

Quantitative Studies

The studies reviewed in this section used experimental methods to collect data about how adults acquire computer skills. Subjects were grouped according to personal characteristics and instructional treatments, trained, and then compared on various measures of performance. In many of these studies, the treatments were identical to methods commonly used by providers of computer training and support: classroom instruction, CAI, video, and printed manuals. These studies would seem to be of immediate relevance to educational practice, but many do not fulfill this expectation. Their findings suggest that the learning associated with these forms of instruction is insufficient to account fully for the ability to use a personal computer in daily work. While many of the studies report similar results, discrepancies in their designs make direct comparisons between them difficult. A discussion of designs and results will identify some common methodological problems.

Two studies by Sara Czaja and her colleagues are representative of many others in their design and their findings. Czaja et al. (1986) evaluated three training strategies for teaching novices to use the WordStar word processing application: computer-assisted instruction (with commercially-available tutorial software), use of a printed manual, and instructor-led lecture and drill. The subjects were 135 women with basic typing skills and at least a high school education. In an experiment lasting eight hours, all subjects were trained on the same set of WordStar operations during the morning, reviewed this training briefly after a lunch break, and then attempted to complete a series of realistic word processing tasks. Performance was evaluated on number of tasks attempted and completed, average time per task, and average number of errors per task. Computer-assisted instruction was found to be the least effective means of instruction, which the researchers attribute to the passivity enforced by programmed instruction. However, they also note that "none of the methods evaluated in this study proved to be effective in teaching people word processing. On the average, people made approximately 7-11 errors per task, which appears too high..." (Czaja et al., 1986, p. 215).

In a 1989 paper, Czaja et al. analyzed the same data in terms of age-related differences in learning. The subjects were equally divided among three age groups: 25-39, 40-54, and 55-70. No significant interaction was found between age and training method. All subjects in the computer-assisted instruction condition performed less well, regardless of age. The subjects in the older group consistently performed less well than younger subjects in number of tasks attempted, time on tasks, and time-averaged errors. The authors suggest various possible explanations for the lower performance of older learners, including age-related differences in spatial memory. However, they overlook the possibility that older learners may require longer periods of training to achieve the same skill levels as younger learners (Zandri & Charness, 1989).

The design of this experiment, involving only a few hours of training prior to two hours of testing on six complex tasks, reflects an unrealistic assessment of the inherent difficulty of personal computer performance. One could compare it to providing novice automobile drivers with a few hours of instruction and then asking them to drive unassisted to various locations. Unfortunately, the treatments in most experimental studies suffer from the same lack of duration.

Two similar studies by Gist, Rosen, and Schwoerer (1988 and 1989) tested the effectiveness of observational learning and behavioral modeling (Bandura, 1986) in the development of computer skills. In the 1988 study, older and younger groups received three hours of spreadsheet training through either a tutorial or modeling approach. The tutorial group used CAI software to complete exercises and solve problems, while the modeling group worked with the same content but saw procedures demonstrated by an actor on videotape before each exercise. Both age groups performed better under the modeling condition; the older group scored less well on a performance test regardless of training condition, a difference that may reflect the three-hour time constraint.

In their 1989 study, Gist et al. explored the effects of behavioral modeling on self-efficacy (the perception of one’s own ability to learn or perform a particular task). Participants were categorized as high, medium, or low in self-efficacy prior to participation in three hours of spreadsheet training, with instructional treatments similar to those of the 1988 study. Participants were randomly assigned to behavioral modeling and computer-assisted instruction conditions, tested at midpoint for self-efficacy, and tested at the end of training for performance and satisfaction. Again, behavioral modeling led to consistently higher performance scores for all groups. Low self-efficacy participants reported fairly high efficacy at midpoint under the modeling condition, while under the CAI condition they reported much lower efficacy. Both these experiments support the use of video in training as a means of demonstrating and situating computer skills and tend to reinforce the findings of Czaja et al. on the relative ineffectiveness of CAI.

Zandri and Charness (1989) also compared the effect of different instructional treatments on younger and older subjects, but achieved different performance results. Forty-six subjects participated in four self-paced training sessions on the multi-purpose application Sidekick. Each session lasted a maximum of three hours. Treatments included working individually and working with a partner of the same age cohort; participants in the partnered condition were required to take turns at the computer keyboard during problem-solving exercises. All received written instructions and were allowed to work through the assigned exercises at their own pace.

Older adults asked for help two to three times as often as young adults and took twice as long as younger ones to complete the training, but achieved nearly equal performance results on a post-training performance test. Initial attitudes toward computers showed little correlation with final performance, and partnered training was found to be at least as effective as individual training. This study suggests that older adults can achieve the same levels of computer skill as younger ones if they are given enough time (through self-pacing) and have the opportunity to ask questions. All participants had near-maximal scores for performance, indicating that self-paced, problem-solving activities of sufficient duration are likely to lead to stronger skill development.

In another comparison of WordStar training methods, Frese et al. (1989) drew on the work of Donald Norman and others who have studied the user’s construction of a mental model of a computer system. Frese et al. categorized training methods on two dimensions, passive-active and sequential-integrated. The passive-active dimension describes the extent to which the learner is encouraged to construct and test hypotheses about the system, while the sequential-integrated dimension describes the extent to which the training relies on rote instruction in simple procedures or on high-level explanations of how procedures are organized within the system. A combination of active and integrated instruction was hypothesized to support the development of an accurate mental model.

To test this theory, the experimenters trained 15 subjects in three groups, each with his or her own computer and experimenter. All subjects were trained by editing an example text. The "sequential" group was given passive instruction with written materials detailing exact step-by-step procedures. The "hierarchical" group received a more detailed manual and diagram that presented the organization of commands within WordStar. The "hypotheses" group received no written material; they were encouraged to hypothesize about how to fix errors in the sample text and to experiment with different solutions. The correct commands were then provided by the experimenter.

Two-hour training sessions on two days were followed by a two-hour testing session. Subjects were assessed on their ability to recall commands, time-weighed error scores, keystroke efficiency, transfer of skill to a previously-unused command, prior typewriting skill, and other measures. As predicted, the hypotheses group outperformed the sequential and hierarchical groups on all measures, though not all differences were statistically significant. The hypotheses group showed the largest degree of variance on corrected error time, keystroke efficiency, and transfer. As in the studies by Czaja et al. and Gist et al., instructional methods requiring more self-directed problem solving produced better results than programmed instruction.

A study by Charney, Reder, and Kusbit (1990) further explored the relation between self-directed, exploratory learning and achievement of computer skills. Sixty-five subjects used the same training manual to learn 12 commands for the spreadsheet application VisiCalc. One group learned through guided exploration–no problems were posed to them, but they were encouraged to try out the commands on sample spreadsheets. The other group worked with six tutorial exercises, in which both problems and step-by-step solutions were provided, and six problems that required them to attempt to discover solutions, with feedback from the experimenters. The exploration learners did not spend as much time on (self-generated) exercises and seemed to have difficulty in posing realistic problems for themselves. Problem solving was slower, but appeared more effective than exercises in which each step was provided; problem-solving instruction also produced the best scores on a performance test given two days later. The researchers employed regression analysis to determine that this outcome was not simply the effect of longer training time in the problem-solving condition, and suggest that a completely exploratory method may be relatively ineffective for computer novices who lack sufficient knowledge of the domain to set appropriate learning goals.

Quantitative studies of computer learning vary widely in the degrees to which they attempt to assess factors of prior knowledge, motivation, and learning style. While few would dispute that these variables are likely to have a strong influence on learning outcomes, quantitative designs seem ill-equipped to offer much insight into their effects.

For example, Gattiker (1990) attempted to assess how individual differences such as academic ability and gender affect the acquisition of workplace computer skills. The subjects of the study were university students enrolled in a management course on end-user computing. Thus, in spite of a large number of subjects (n=347) and a fairly complex use of multiple regression analyses, the results of the study have more bearing on performance in computer classes than in the workplace. The study found that previous enrollment in a survey course in computer science was of limited benefit to subjects with below-average academic ability (as indicated by GPA) in their performance in the subsequent computer course. However, female students in this group seemed to show more ability to transfer knowledge from the earlier class to successful performance in the management class. The researcher found that performance of lower-ability students was significantly correlated to the amount of time they spent on lab practice–certainly not a surprising result. Gattiker’s findings in this study are too inconclusive to have a strong bearing on training practice.

In a similar effort to understand motivational factors, Webster and Martocchio (1993) examined the effects of task labeling (as work or play) and trainee age on learning outcomes. The 68 subjects were university employees enrolled in an institutionally-sponsored class on the mail-merging feature of WordPerfect 5.0. Participants were randomly assigned to work or play treatments and completed pre-training surveys designed to assess computer anxiety, computer attitudes, confidence, motivation, computer experience, and pretraining WordPerfect knowledge. Subjects in each treatment heard lectures and completed practice exercises on the same features of the software. In the play treatment, the instructor relied on a script using language that had previously been tested for its associations with play, while in the work treatment the researcher used work-related language to describe and explain the same training. The investigators hypothesized that the play treatment would produce more positive learning outcomes and that this effect would vary with the age of the participants. At the end of training, participants completed a questionnaire to assess post-training motivation and a ten-item objective test on the mail-merge operations.

The play treatment was found to be somewhat more effective for younger participants, but older participants showed no significant difference between treatments. As in most other studies, younger participants tested significantly higher on training outcomes than older ones. The researchers showed praiseworthy caution by controlling for training expectations, computer experience, and previous WordPerfect knowledge as covariants in their analysis. However, the use of a ten-item multiple-choice test as a measure of performance provides little indication of the participants’ ability to use the mail-merge procedure in practice. The researchers did not report the duration of the training activities (one can infer that training lasted less than one day), nor did they describe the opportunities participants had to practice with the WordPerfect application during training. This study provides little useful data about an important variable in computer learning: the effect of task labeling, or perceived goals, on motivation and performance (cf. Elliott & Dweck, 1988).

Bostrom, Olfman, and Sein (1990), as part of a series of computer training studies, attempted to assess the influence of learning style on training outcomes. Bostrom et al. distinguished two basic categories of training: exploration-oriented, emphasizing trial and error, learner control, and realistic tasks, and instruction-oriented, emphasizing programmed activities, instructor control, and enumeration of software features. To assess the interaction of these forms of training with learning styles, Bostrom et al. conducted four experiments in which subjects learned computer software tasks under varying instructional conditions. Prior to training, all subjects completed the Kolb Learning Style Inventory (Kolb, 1976) and were categorized in terms of the basic learning style dimensions abstract/concrete and active/reflective. The authors acknowledged that the disputed validity of this instrument limits the generalizability of their results, but reported a consistent pattern of findings across the four experiments: "In general, abstract learners performed better than concrete learners. In most cases, active learners performed somewhat better than reflective learners" (Bostrom et al., 1990, p. 113). The lack of significance in all but one of the four studies is attributed to varying degrees of experimental control, but the researchers also note that differences in preferred learning style do not prevent individuals from learning under other conditions: "A person’s learning preference does not preclude or negate what he or she learns" (p. 114). One implication of this is that, while some people may be predisposed to learn computer skills because of a learning style suited to abstract concepts and active experimentation, training methods can be adapted to the needs of concrete and reflective learners as well–for example, through the use of analogies to provide concrete references for computer structures and functions.

The same distinction between exploration-oriented and instruction-oriented training informs a 1993 study by Davis and Bostrom dealing with the interaction between training method and computer interface design. The experiment compared the effects of both training methods on learning to perform a set of basic file and directory tasks on IBM-compatible and Macintosh computers. Findings supported those of other experiments that indicate that the mouse-based graphical user interface (GUI) supports faster learning and performance than a command-based interface. The visual display of files and directories as icons that can be moved with the mouse appears to allow novices to construct a more accurate mental model of the computer system in a shorter period of time. Subjects learning to work with files and directories by typing verbal DOS commands had significantly lower and more variable performance scores. The type of interface used had a much stronger effect on learning outcome than training method, reflected in the low variability of performance scores under the GUI condition. Given the increasing popularity of the GUI as a standard on both Macintosh and IBM-compatible personal computers, the findings of this study are encouraging. However, the failure to find significance for differences in training method suggests once again the low effectiveness of formal training methods and the limited value of quantitative findings as guidelines for educational practice.

Some of the methodological problems of these studies are apparent. In his review of the computer skill acquisition literature, Gattiker (1992) notes that most experimental studies have failed to assess adequately subjects’ prior knowledge, perceptual speed, or motor abilities. Further, most research has been conducted with students in university settings (e.g., Gattiker’s own 1990 article), and thus does not support inferences about workplace learning: "Specifically, these propositions should be tested in the context of continuous education efforts undertaken to advance skills held by employees" (p. 565). Webster and Martocchio (1993) conducted the only study reviewed here that met this criterion, but its performance measures were unrelated to actual work tasks.

A second design flaw of most experimental studies of computer instruction has been their short duration. Total training time is usually in the range of two to four hours. Gattiker argues that this is inadequate:

Gattiker also notes that adults who have not been formally schooled for many years can be overwhelmed by long, continuous training sessions, such as a full-day experiment; intermittent training of the same total duration may be more effective, particularly for those with low initial knowledge or skill.

A third common design problem in experimental studies is their lack of consistent measurements of performance (learning outcomes). To some extent, this is an inevitable consequence of the variety of hardware and software used in the studies. However, it is an inherent limitation of experimental designs that performance must be assessed under controlled conditions that have little in common with the workplace situations in which most computer novices are required to develop their new skills.

The quasi-experimental methods discussed above are not the only possible approach to rigorous quantitative measurement of computer skill learning. Recent advances in brain research suggest that the computer itself, through imaging technologies such as positron emission tomography (PET), can reveal surprising relationships between learning and brain activity. Studies by Richard Haier and colleagues (Goldsmith, 1994; Haier et al., 1992a; Haier et al., 1992b) used PET scans to measure glucose metabolic rate (GMR) in the brains of subjects playing the computer game Tetris, an activity that requires eye-hand coordination and visual attention skills similar to those used in other computer tasks. Subjects were scanned after an initial session learning the game and then again after several weeks of practice. While average game performance increased sevenfold, GMR declined; more skillful performance was strongly correlated with lower levels of energy use. Haier et al. attribute the brain’s increased efficiency to the development of optimal cognitive strategies, which, once automatized, lead to the use of fewer extraneous brain areas and an overall decrease in the glucose used by the brain. The authors acknowledge that Tetris requires attention and speed, but "may not be an adequate probe of working memory" (Haier, 1992b, p. 425), which appears to play a critical role in many cognitive skills. These results appear to be consistent with other research on computer skill learning that emphasizes increased efficiency as a result of automatic processing (e.g., Singley & Anderson, 1989). However, the use of PET scans to study brain activity during learning is still in its earliest stages and seems to have no direct application to educational practice at this time.

Quantitative methods are attractive to many researchers interested in computers; the objects of study can also serve as instruments for data collection, statistical analysis, and even visualization of brain activity. However, the value of this work for trainers and educators depends on the researchers’ ability to explain the instructional implications of their data. Most quantitative studies have not taken a sufficiently broad view of the problem of computer skill development to provide useful insights about the experiences of typical adult learners.

Qualitative Studies

Many of the strongest factors influencing computer skill learning, such as motivations, expectations, goals, self-efficacy, and prior knowledge, do not lend themselves to quantitative measurement or statistical analysis. Data about these mental states may be more appropriately collected and analyzed through qualitative methods such as observation, interviews, and case studies (Bogdan & Biklen, 1992; Merriam, 1991). Qualitative data are human expressions of meaning such as speech and other actions; their form is generally verbal rather than numerical (Dey, 1993). In many respects, qualitative research provides a more integrated view of the process of computer skill learning than does quantitative research. This section reviews qualitative studies employing verbal protocol analysis and phenomenological interviewing.

Although it differs from most qualitative methods in its reliance on experimental conditions, verbal protocol analysis (Ericsson & Simon, 1984) is a common strategy in research on computer learning that has produced rich qualitative data. Learners are presented with various computer tasks and are encouraged to ‘think aloud’ about their actions, strategies, and problems; the sessions are recorded on audio- or videotapes, which serve as data sources. Analyses of recordings focus on reconstructing the learners’ cognitive processes rather than on measuring their performance (although simple frequency counts of actions or behaviors can provide a means of triangulating data from verbal reports). The crucial difference between protocol analyses and other experimental approaches is that learners’ subjective reports about their computer experiences are accepted as valid and valuable information.

Robert (1989) is representative of verbal protocol studies of computer learning. Robert was impressed by the tendency of Macintosh users to prefer to learn through unassisted exploration rather than by referring to manuals. In his experiment, subjects who had some previous computer experience but no Macintosh experience worked on their own with the computer for two hours without access to manuals or other assistance. Throughout, the subjects were encouraged to verbalize their thoughts and problem-solving strategies. In analyzing videotapes of these sessions, Robert observed subjects using strategies of trial-and-error, experimentation, repetition, analogy, and induction. They encountered numerous problems in the course of exploration: getting lost, being overwhelmed with information, being naive about the repercussions of their actions, misinterpreting system cues, becoming snarled in tangles of errors, and getting sidetracked. Robert concluded that exploratory learning is facilitated or hindered by the user’s prior knowledge of the task domain, the complexity of the computer system, and the quality of its interface design.

Studies of subjects with prior computer experience are helpful in assessing the levels of understanding achieved by typical personal computer users. Briggs (1990) and Santhanam and Wiedenbeck (1993) used verbal-protocol designs similar to Robert’s to assess the knowledge and skill of people who used personal computers on a regular basis but who had no special computer expertise. Both studies found that users who are competent in basic word processing often exhibit gaps in their comprehension of computer systems.

Briggs (1990) notes that many people teach themselves to use word processing applications but questions the success of these self-directed learning efforts. She argues that such learners tend to acquire procedural skills in word processing without adequate mental models of their computer systems. To test this hypothesis, Briggs studied the ability of users of varying experience to teach themselves a new word processing application on an unfamiliar computer. In addition to audiotaping the subjects’ questions and comments, she collected quantitative data in the form of recorded keystroke counts during two document-editing tasks.

Briggs found that few subjects were able to make use of their previous word processing experience to ask for relevant information; those who had used more than one word processing application were most successful in verbalizing questions that helped them accomplish the tasks. Often users lacked sufficient explicit awareness of system events to monitor and improve their own performance: "Users have a tendency to act before they think" (Briggs, 1990, p. 396). This study suggests that purely self-directed learning is not sufficient as a means of becoming a competent user, a finding consistent with the results reported by Charney, Reder, and Kusbit (1990).

Santhanam and Wiedenbeck (1993) were interested in the word processing performance of users who were neither novices nor experts. They studied 14 discretionary users–"lawyers, managers, professors, and university officials" (p. 205)–who had experience editing substantial documents. Subjects used their preferred computer system and software to perform a set of tasks identified as "routine" and "non-routine" by the experimenters. These subjects were compared to a smaller number of experts (secretaries and technicians who used the software daily) and novices (no experience with the software) on the same tasks. All subjects were videotaped and were guided to verbalize their thoughts as they performed the tasks. The performance of the novice and expert subjects provided triangulation for the data-coding categories used with the discretionary users.

The discretionary users were remarkably homogenous in their performance. They tended to display expert-like behavior on routine tasks, although they often used sub-optimal methods of accomplishing them, preferring to rely on a small set of familiar skills rather than attempting to learn new skills. On non-routine tasks, their behavior was strongly novice-like and showed that their conceptual knowledge of the systems and commands was shallow. These users appeared to have achieved a steady state of knowledge and showed little motivation to learn abstract concepts that would support expert-level reasoning about system behaviors. This study points toward a "plateauing" effect in the development of computer skills.

These findings are largely consistent with the influential work of John M. Carroll (Carroll, 1990), whose central thesis is that adults want to accomplish meaningful tasks and use prior knowledge about these tasks to try to make sense out of computer systems. Thus learners are vulnerable to a motivational double-bind; their eagerness to accomplish work-related goals often interferes with their development of a more coherent understanding of the computer system.

Carroll (1990) supports this view with data from a series of studies originally reported in Carroll and Mack (1984) and other articles. In the 1984 study, the subjects were ten temporary workers with extensive experience in routine office activities but no prior computer or word processing experience. They were asked to imagine a scenario in which a word processing system had just been introduced to their office and they were to learn it in order to train colleagues, working with the printed manuals and tutorials provided with the system. Each subject spent four half-days teaching herself to use the system and was then tested on a simple transfer task requiring her to type, revise, and print a one-page letter. Throughout, subjects were encouraged to think aloud, with experimenters intervening occasionally to prompt them to verbalize their thoughts. The self-training sessions were recorded on videotape, which was the primary data source.

The conditions of this experiment closely parallel the circumstances in which many workers first encounter personal computers. The researchers’ goal was to identify and categorize critical incidents in the learning process as the basis for generalizations about typical experiences of computer novices. In this study and others, Carroll and his colleagues describe patterns of behavior that will be familiar to anyone who has attempted a similar learning task. Learners consistently "jumped the gun" by plunging into tasks before they had read instructions. Their half-hearted efforts to coordinate their actions with step-by-step tutorial exercises were often thwarted by errors that left the computer system in a state out of synchronization with the training materials. These episodes often created tangles of errors in which the original deviations from the training sequence were forgotten or unrecoverable. Errors were also caused by inappropriate analogies drawn from prior knowledge (e.g., of typewriters). Even more striking was the tendency of learners to diverge completely from programmed instruction to explore system features on their own, often referring to several manuals simultaneously in an effort to construct an instructional agenda around personal goals.

Carroll (1990) argues that classical instructional design methods based on the decomposition of knowledge into basic concepts and principles (e.g., Gagne, Briggs, & Wagner, 1988) are ineffective in supporting the learning needs of adult computer novices. As an alternative, he proposes a "minimalist" design method that emphasizes less reading, more realistic tasks, and anticipation and exploitation of learner errors. He supports the effectiveness of such instruction with data from various experiments testing "minimal manuals," "task scenarios," and a "training wheels" interface in which advanced features of an application are blocked to prevent novices from becoming entangled in errors before they have mastered basic procedures.

Carroll’s work is especially important in its recognition that the need to gain computer skills is situated within a context of social incentives and constraints. This approach to the problem of computer training acknowledges that human relationships, rather than hardware and software, drive the process of learning.

Phenomenological studies of computer skill learning are relatively rare, but their findings have been similar to those of protocol analyses. Boland (1985), in a review of his interview-based studies of system design, argues that surveys and other quantitative methods are less effective than a hermeneutic approach that treats "the design and use of information systems [as] the text that we must try to understand" (p. 194). Boland emphasizes that all phases of computer use are interpretive activities that should be understood as dialectical processes of sense-making, in which users, programmers, and researchers are engaged in the social construction of meaning.

This philosophical orientation, which is more characteristic of most qualitative research in education than the comparatively rigid procedures of protocol analysis, is also evident in Howard’s 1994 study of adult computer users. Based on in-depth interviews with eight computer users, this study attempts to identify common themes in adults’ first encounters with personal computers. Not surprisingly, Howard’s subjects reported experiences similiar to those of Carroll’s: their motivational conflicts, anxieties, humiliations, and gradual adaptation are poignantly described in their own words and as interpreted by the researcher. Howard outlines several stages in this process: initial excitement about the potential benefits of computing; frustration caused by unexpected complexities; and accommodation to the practical requirements of computer use.

Howard’s study demonstrates both the depth and the dangers of the phenomenological method. His primary data are drawn from only four interviews, with an additional four interviews serving as data for triangulation. This sampling procedure seems insufficient to support useful generalizations, especially since the study does not build on earlier data-based research. Howard moves quickly from his limited data to broad assertions about the reproduction of cultural values. His analysis of recurrent subjective experiences of computer skill learning prefaces a highly-charged discussion of the ways in which the mediation of experience by technology reinforces values of "efficiency, accuracy, and progress" (p. 36) at the cost of more humane values of reflection and contemplation. While the philosophical arguments presented are challenging and valuable in themselves, their relationship to the interview data is tenuous and unpersuasive. As a result, the study is satisfying neither as an empirical study nor as an essay of opinion. However, it does demonstrate that interviews can produce data about the emotional and interpersonal conflicts inherent in computer skill learning.

Organizational Studies

A fundamental weakness of most of the research reviewed to this point is that the initial encounter between an adult learner and a personal computer usually occurs not in a laboratory but in the workplace. Most adults are motivated to learn about computers–willingly or not–by the expectations of managers and colleagues. No account of computer learning that fails to consider the obstacles and opportunities created by its organizational context can serve as an adequate guide for the practice of adult educators.

From the perspective of sociology, the acquisition of computer skills by individuals is an element of larger processes of organizational learning, through which

Computerization offers new possibilities for explicitly recording and transmitting organizational knowledge and even for capturing the decision-making skills of high-performing workers in expert system programs (Harmon & Sawyer, 1990). However, the inclusion of particular personal computers and applications in organizational routines can also lead to "competency traps," in which familiarity and habit block the adoption of more mature technologies. Levitt and March (1988) indicate that such learning can lead "an organization, industry, or society to persist in using a set of procedures or technologies that may be far from optimal" (p. 323). Examples include the standard QWERTY keyboard, originally created to slow down workers who were exceeding the mechanical limits of early typewriters; while alternative layouts offer potentially higher levels of keyboarding speed, the inefficiencies of the QWERTY design are apparently now too deeply embedded in educational and business practices to eliminate. The popularity of relatively primitive software such as MS-DOS threatens to produce similar effects. However, the price-performance ratios of computer systems improve so rapidly that fixation on any single standard seems unlikely to last.

One implication of the rapid evolution of personal computers is that their successful use in organizations depends on how workers respond to the stress created by continuous pressure to learn. Bikson (1987) summarizes the results of various RAND Corporation studies on organizational adaptation to computer-mediated work. These findings indicate that workers tend to respond positively to the initial introduction of computers in the workplace, in spite of the fact that few organizations take any interim measures to manage workload while users are learning a new system. Bikson emphasizes, however, that the short-term stress of introducing computers is less significant than the subsequent, ongoing increase in demands on cognitive resources:

Bikson points out that only about one-third of the managers surveyed believed that their organizations had a continuing role in facilitating users’ adaptation to electronic information tools, although more than 70 percent believed that moderate to major changes in the use of these tools would occur in their groups over the next two to three years. The implication is that employees are often left on their own to learn about new computer systems. Bikson suggests that workers’ learning can be enhanced by informal knowledge sharing within workgroups and by training that promotes the development of cognitive models of computer systems rather than rote performance, thus supporting problem solving, generalization, and adaptation to next-generation tools.

The issue of computer skill learning is complicated by debate about the effects of computerization on the social environment of work. Some writers contend that computers lower the cognitive demands of work, fragmenting jobs and reducing the quality of working life. Braverman (1974) framed the terms of what has come to be called the "deskilling debate," arguing that automation serves to increase managerial control by lowering skill requirements, making it possible to hire less educated and experienced workers and thus reduce labor costs. However, it is not clear that such effects are inevitable consequences of computerization.

Kraut (1987) observes that research on the deskilling issue has produced a range of contradictory results and attributes this to several causes. First, case studies have often focused on organizations using technology at "different epochs and stages of maturity" (p. 14). For example, studies of centralized, minicomputer-based word processing centers cannot be compared directly to studies of individuals using word processing applications on personal computers. Second, "researchers have ignored important level-of-analysis distinctions about the units of work that are being skilled or deskilled" (p. 17). Often the deskilling of a particular task is accompanied by the reorganization of tasks within a job; spellchecking may reduce the lexical skills once required of secretaries, but the availability of laser printers and multiple fonts demands learning new typographical skills. Third, the introduction of computers can have the effect of redistributing tasks across job categories, so that skill needs are shifted within organizations: "In universities, professors do much of their own typing with word processing equipment and personal computers, transferring some of the most onerous of data entry tasks from secretaries to professionals while at the same time giving the professionals more control over the form and content of their writing" (p. 19). Such transfers of skill requirements would seem to have generally positive effects on job quality–provided that resources are available to support learning.

Citing early social analyses of the telephone, Kraut argues that attempts to predict and evaluate the effects of technological change on the workplace have been notably unsuccessful. Our limited understanding of how personal computers affect the quality of working life may encourage overly-simplistic interpretations of the social impact of the technology. Similarly, Rule and Attewell (1989) argue that "computing has become a kind of projective device for social scientists and other social critics" (p. 131). In a survey of 184 private sector organizations in and near New York City, they found the majority of computing activities to be straightforward conversions of conventional processes such as payroll, invoices/billing, accounts receivable and payable, word processing, order entry, general ledger, financial statements, mailing lists and labels, and so on. At each site, computers were adapted to existing organizational functions and goals, which remained largely unchanged.

However, some researchers have observed more radical effects on organizations. In her case study of several corporations adapting to new technology, Zuboff (1988) observed the emergence of new patterns of collaborative work, in which learning computer skills coincided with learning new forms of organizational behavior. She concluded that the introduction of computers into an organization can serve to transform the culture of work by creating

The ability of computers to create a new kind of work environment is particularly evident in organizations using groupware applications. These programs are designed to support workgroup collaboration, typically including functions such as electronic mail, document exchange, scheduling, and time logging. Bullen and Bennett (1991) interviewed 223 people in 25 organizations using groupware, ranging from large corporations to small businesses. Their findings provide interesting insights about the role of organizational activities and values in the individual’s acquisition of personal computer skills. Some features of the groupware applications were quickly and generally adopted, while no one bothered to learn other features:

When organizational training efforts focused on the procedural mechanics of groupware applications, users tended to learn only the most basic skills and to employ the tools in unimaginative ways. In some cases, where managerial expectations for improved performance were high but groupware introduction involved no re-examination of current work processes, the pressure of having to learn a new tool had a negative effect on productivity. Where trainers presented groupware in terms of its ability to accomplish the organization’s unique work tasks or to make workflow more efficient, people made better use of the applications.

Successful adoption of groupware and other personal computer applications seems to depend less on formal training than on advocates in management who champion their use and on departmental "gurus" to whom co-workers can turn as learning resources. This finding is consistent with other studies that suggest that one-on-one problem solving episodes are the most popular form of computer learning in organizations. Adult learners tend to prefer to ask another computer user a question rather than rely on manuals or on-line help, perhaps because such exchanges situate new knowledge in the context of shared activities:

Two key points emerge from the literature on computer learning in organizations. First, training and learning are not equivalent. As shown by the experimental studies of computer instruction reviewed above, people do not generally achieve significant levels of practical skill solely through formal training, if they are fortunate enough to receive any. Most personal computer learning occurs in the workplace in the course of day-to-day activities. Second, such learning is usually a communal activity, in which computer knowledge is developed and shared within workgroups. Such sharing serves not only to accomplish organizational goals but to construct the social identities of the participants. These points are elaborated in the conclusion of this chapter, which discusses theories of situated learning and of informal and incidental learning as a conceptual framework for the study of computer skill learning.

Situated Learning

To support practice effectively, educational theories should explain and predict learning in ways that can guide the planning, design, and delivery of instruction. Good theories of computer skill learning must explain both successful and unsuccessful instruction. For example, Carroll (1990) presents evidence that some manuals and CAI tutorials can actually impede adults’ progress in gaining practical computer skills, while studies such as those of Czaja et al. (1986 & 1989) suggest that short-term classroom training can be equally ineffective. Yet people do learn to use personal computers and succeed in getting meaningful work done. These findings can be understood in terms of an emerging body of theory, often called situated learning, that claims that knowledge is inextricably involved with the social circumstances of its use (Lave & Wenger, 1991).

Situated learning has roots in a variety of social scientific research traditions, including cognitive anthropology, activity psychology, and critical theory (Chaiklin, 1993). It is not a single, unified learning theory, nor are the most prominent researchers in this movement in unequivocal agreement about its goals, methods, or principles.

The person most strongly identified with situated learning is the anthropologist and educational researcher Jean Lave, whose central concern has been to move the study of cognition and learning out of laboratories and classrooms into settings of everyday activity. Some of her early work focused on the methods of calculation used by adults in grocery shopping, food preparation, and other ordinary situations; in these studies, she found that adults rely on procedures that exploit the physical materials at hand rather than on the arithmetic algorithms they were taught in school (Lave, 1988; Lave, Murtaugh, and de la Rocha, 1984). With Etienne Wenger, she has elaborated the concept of situated learning into a theory of "legitimate peripheral participation" that emphasizes the relationship between learning and participation within social and historical "communities of practice" (Lave & Wenger, 1991). In a recent article, Lave articulates a set of premises that indicate the basic approach of situated learning theorists:

Lave insists that situated learning theory itself, like the practices it describes, should be seen as the shared construct of a community rather than as a reified body of knowledge.

John Seely Brown, Allan Collins, Paul Duguid and others have collaborated on a series of papers that apply concepts of situated learning to a variety of domains: the culture of traditional schooling (Brown, Collins, & Duguid, 1989); the use of "cognitive apprenticeships" to teach reading, writing, and mathematics (Collins, Brown, & Newman, 1990); the development of intelligent tutoring systems (Brown, 1990); organizational learning (Brown & Duguid, 1991); computer skill learning and system design (Brown & Duguid, 1992); and educational technology (Brown & Duguid, 1993). These articles will provide the primary sources for the following discussion of situated learning.

Situated learning can be described in terms of three premises: its rejection of the idea of general transfer of knowledge; its insistence that cognition is socially distributed rather than solely an attribute of individual minds; and its social and historical analysis of communities of practice.

The general transfer of knowledge is implicitly assumed by almost all forms of institutionalized education. In this view, knowledge can be abstracted from the situations in which it is used, presented to students in a generalized, symbolic form, and subsequently applied by them to tasks outside the instructional setting. This belief informs most of the methods and tools of schooling, including lectures, textbooks, and examinations. Unfortunately, there is little empirical evidence supporting general transfer. Singley and Anderson (1989) review research in this area, beginning with the studies of Thorndike (1906), and find no basis in the psychological literature or in their own experiments to support claims that generalized knowledge can be transferred to situations that do not share identical elements with the situations of its acquisition. Instead, knowledge tends to be highly use-specific. Singley and Anderson offer the "extreme example" of someone who has been told the rules of chess:

If computer knowledge and skills are inseparable from the situations in which they are applied, the frequent failure of computer training to improve performance in other settings is more comprehensible. Having rehearsed a procedure or heard about a concept in a classroom does not guarantee that one will be able to use it appropriately in practice.

Another element of situated learning theory is its location of cognition within material and social environments rather than inside peoples’ heads. This view appears to oppose mainstream cognitive psychology and artificial intelligence, which attribute problem-solving performance to processes occurring within individual agents. That such processes occur is not disputed; however, ethnographic studies in settings as diverse as aircraft carrier navigation (Hutchins, 1993), industrial production (Darrah, 1992), copy machine repair (Orr, 1990) and tailoring (Lave & Wenger, 1991) suggest that people rely more heavily on interactions with co-workers, tools, and materials to do their jobs than on the application of knowledge from memory. Situated learning theory holds that locating cognitive skill solely within individual minds ignores the way people communicate with each other and use their physical environments to distribute the load of memory search and problem solving. Tools and workplaces provide resources that reduce demands on participants’ memories and logical abilities; communication with other workers frames problems, narrowing the search for solutions.

The significance of this view for research on computer skill learning is primarily methodological. It suggests that rather than relying on laboratory experiments or surveys, researchers are more likely to understand how people learn to work with computers through interviews, fieldwork, and document analysis. These methods allow the collection of better data about the social contexts of computer use and may shed light on the ways learning occurs in the process of work.

A third element of situated learning theory is its social and historical analysis of communities of practice. If the knowledge required for work is seen as constructed by groups rather than belonging to individuals, it becomes important to understand how the organizational context of an activity has evolved historically. Work practices occur within larger cultural and economic processes that shape the behavior and understandings of participants, often in ways of which they may be unaware. These processes influence learning by shaping perceptions of who can legitimately participate in which activities, how conflicts of interest between "newcomers" and "old-timers" should be resolved or repressed, and what kinds of social identities participants will adopt (Lave & Wenger, 1991).

In the context of computer skill learning, these issues are closely tied to processes of organizational innovation. An understanding of how people learn to work with computers seems to be inseparable from the changes that computers create in organizational structure and behavior (Zuboff, 1988). The implementation of new information tools carries with it new expectations for productivity (Bikson, 1987) and the possibilities of deskilling and redistribution of skills (Kraut, 1987). In turn, organizational innovation has to be understood in the context of global economic changes that make computerization a matter of competitive survival (Toffler, 1990).

These elements of situated learning theory may account for aspects of computer skill learning that have otherwise received little attention. In its emphasis on everyday practice, situated learning has similarities to a number of theories of adult learning, such as Schön’s concept of the "reflective practitioner" (Schön, 1983). Another theory that deals more directly with the communication of knowledge through social interactions is informal and incidental learning (Marsick & Watkins, 1990).

Informal and Incidental Learning

Informal and incidental learning theory, like situated learning, is based on the premises that training and learning are not equivalent and that most learning occurs in the course of work practices. It addresses learning at several levels: individual, workgroup, organizational, and professional. Framed as a theory of human resource development, it is developed around cases and situations familiar to trainers in many kinds of organizations and is thus more adaptable to applications in adult and continuing education than most research on situated learning. Its basis in action science (Argyris, Putnam, & Smith, 1985) provides a framework for organizational interventions that encourage people to reflect on their attitudes, beliefs, and practices. The categories of informal and incidental learning are useful in analyzing the kinds of computer skill learning that occur without formal training.

Informal learning occurs primarily outside institutionally-sponsored instruction. It is goal-oriented and intentional; it can be planned, but is often spur-of-moment. It includes self-directed learning, as in the case of someone who "fools around" with a new computer program, perhaps dipping into books or on-line help to construct an ad hoc curriculum, trying things out and learning along the way. Such learning is motivated by real-world tasks and agendas (Carroll, 1990). Alternatively, an informal learning episode might involve networking within a workgroup or organization, identifying people who are expert users of a particular program in order to solicit their help in resolving a problem (Bullen & Bennett, 1991).

Perhaps the most common informal computer learning situation is one-on-one coaching or mentoring by a co-worker. An employee may give a new worker an orientation to the organization’s electronic mail system through an impromptu tutorial session. Such spontaneous training episodes may communicate only the most rudimentary knowledge and skills, but they do so in the same setting in which those skills will be practiced (i.e., at a specific office computer). They may also initiate interpersonal relationships that can support future instruction and problem solving.

In contrast, incidental learning is unintentional, a by-product of another activity. Often learners are not aware that they are constructing knowledge during these incidents. Through force of repetition or observation, they can absorb ideas about computers that they might not be able to verbalize unless encouraged to reflect on their attitudes and actions. Incidental computer skill learning includes learning through errors or problem-solving episodes. For example, in the process of producing an annual report, a computer user may teach herself how to add bullets to text or how to create a chart from a spreadsheet, even though she began the task with no explicit intention of learning these skills. Computer users seem to "pick up" a variety of skills in this way, often as the result of situations that place new demands on their knowledge of computers and software.

Incidental learning also includes the assumptions, beliefs, and attributions about computers that people unintentionally communicate to one another. For example, by listening to conversations and observing the behavior of colleagues, computer novices may come to believe that having a particular type of personal computer is a sign of social status; that women, minority members, or older people cannot learn computer skills as readily as stereotypical "white guys in ties"; or that all computer experts use technical jargon to conceal their own lack of understanding and to control less knowledgeable users. As these examples show, incidental learning can be fraught with errors; however, whether or not they are valid, such concepts are easily communicated in social interactions and are likely to influence the learner’s subsequent motivation and behavior.

Incidental learning episodes, over an extended period of time, appear to constitute a socialization process through which people construct personal identities as computer users. In addition to learning how to operate equipment and software, becoming a computer user involves learning to see oneself in certain relationships to computers, other users, and organizational environments. One goal of computer skill learning is adaptation to experiences within a community of practice:

Categories like "novice," "user," and "expert" are more valuable as indexes of social identity than as measurements of skill. Little research has been done on how people construct identities as computer users in response to participation in work practices. If theories of situated learning are accurate, a longitudinal study might find evidence of ongoing changes in perceptions, beliefs, attitudes, and goals during the course of new users’ adaptation to the requirements of computer-based work. Brown and Duguid (1992) discuss the related issue of the difficulties experts can face in acknowledging their need to resume the role of novice in response to changes in technology.

One side effect of the introduction of personal computers in workplaces may be a reduction of status barriers between novices and experts. Brown and Duguid (1992) suggest that

To summarize, theories of situated learning and of informal and incidental learning appear to be consistent with many of the findings of experimental, survey-based, and ethnographic research on computer skill learning. This body of research identifies several variables as critical factors in the development of workplace computer skills:

• Practice effects. The factors that best predict the quality of skill performance are the recency and duration of prior performance. Skills that are constantly rehearsed tend to become automatic and error-free.

• Self-direction. Even when presented with structured training materials, adults learning new computer skills tend to be self-directed in their attempts to apply the skills to work-related problems immediately.

• Specific transfer via mental models. The success of such self-directed learning efforts appears to depend heavily on the transfer of specific knowledge from prior experience with similar systems. Successful transfer can occur when the learner already possesses a mental model that is strongly analogous to the new situation.

• Limited general transfer. Transfer of general, verbal knowledge (such as a lecture or a computer manual) to practical tasks tends to be poor.

• Preference for informal learning. Adults rely heavily on informal interactions with other members of their workgroups for the development of computer skills.

• Socialization. In addition to mastering operations and procedures, adult learners experience a socialization process through which they develop new identities in relation to computers and their organizational setting.

In the following chapter, this conceptual framework provides the basis for the design of an interview-based study exploring how computer users interpret their learning experiences and how interactions with their workgroups and organizations influence their learning.

CHAPTER III

Introduction

The purpose of this study, as outlined in Chapter I, was to examine how adult learners develop personal computer skills in the context of workplace activities, in order to understand how educators, colleagues, and organizations can facilitate this learning. Chapter II discussed potentially valuable constructs from prior research and identified methodological difficulties that limit the applicability of earlier empirical studies to educational practice. One conclusion drawn from the review of the literature was that computer skill learning is studied more appropriately within contexts of organizational behavior than under artificially-controlled conditions. These findings about computer skills were shown to be consistent with theories of situated learning and informal and incidental learning, which provide a conceptual framework for this study.

This chapter describes the methodology and procedures used to address the following research questions:

Design of the Study

This research used interview transcripts and on-site observations as data sources for a comparative case study of computer users in three organizations. This section discusses the rationale for applying a qualitative case study design to the research questions.

Traditionally, as their name suggests, computers have been associated with quantitative methods, so it is not surprising that many studies of computer skill learning have relied on techniques such as statistical analysis of computer-recorded keystroke data (e.g., Card, Moran, & Newell, 1983), expert systems programmed to simulate human learning behaviors (e.g., Anderson, 1989), and quasi-experimental comparisons of instructional treatments (e.g., Czaja et al., 1986 & 1989). These methods have been effective in testing hypotheses about well-defined problems, such as the relationship between duration of practice and efficiency of performance. However, they are ill-suited to the discovery of new concepts or theories or to the study of subjective states and processes such as goals, intentions, understandings, attitudes, beliefs, or emotions. Educational research problems centering on discovery or on subjective states lend themselves more readily to qualitative approaches such as interviewing, observation, and document analysis (Bogdan & Biklen, 1992; Merriam, 1991).

Some of the most influential research on computer skill learning has employed verbal protocol analysis, a quasi-experimental design in which participants are encouraged to think aloud while performing computer tasks (e.g., Carroll, 1990). This method has been used with considerable success to collect verbal data about routine word processing activities (Santhanam & Wiedenbeck, 1993) and adaptation to unfamiliar systems (Briggs, 1991). While these experiments demonstrate the value of verbal data for the study of computer skill learning, their generalizability is limited by their laboratory settings. Given evidence that most skill learning is situation-specific (e.g., Singley & Anderson, 1989; Chaiklin & Lave, 1993), findings based on laboratory protocols cannot be assumed to apply directly to the day-to-day experiences of computer users in offices and other work settings. Therefore, this study employed in-depth interviews, conducted at the participants’ places of work, to elicit contextually-appropriate accounts of practical learning.

Talking to people is perhaps the most obvious means of learning about their perceptions and understandings. Discourse between speakers has a methodological face validity because each of us relies on it daily to understand and predict the behavior of others. As a formal research technique, however, interviewing requires explicit decisions about procedures for sampling, collecting and analyzing data, and assessing validity.

The interviewing strategies used in this study were influenced by psychological research on the use of narratives and explanations in discourse (e.g., Antaki, 1988; Edwards & Potter, 1992; Gergen, 1988; Potter & Wetherell, 1987; Mishler, 1986a and 1986b). These strategies contrast with survey-style interview methods that emphasize problems of standardization: "how to ask all respondents the same question and how to analyze their responses with standardized coding systems" (Mishler, 1986a, p. 233). Survey-style interviewing tends to suppress narrative and explanatory ‘digressions’ in order to produce a homogenous body of data that can be subjected to statistical manipulation.

Discourse-analytic methods reflect a more phenomenological view. Rather than attempting to produce a general, ‘objective’ account based on aggregations of standardized responses, such strategies emphasize that any discourse between speakers acts to construct a negotiated social reality (Edwards & Potter, 1992). This view of interviewing is consistent with the theoretical perspectives of situated learning and informal and incidental learning, which emphasize the importance of interpersonal relationships in the creation and exchange of knowledge.

In a discussion of research on information systems, Boland (1985) argues that phenomenological or hermeneutic methods can produce more useful data than quantitative approaches because interpretation is the defining activity of computer deployment and use:

A primary concern of this study was the influence of workplace relationships and other organizational factors on individual skill learning. Relevant interview data could be provided only by computer users working together within an organization. To support generalizations across organizational types, it was necessary to collect such data within several organizations of differing missions, sizes, and computer resources. Thus, while the central problem of the study was to clarify how individuals learn computer skills, the research was structured as a comparative case study of three organizations. This design allowed within-case analyses of individual skill learning at each organization as well as a cross-case analysis that develops a general interpretation of computer skill learning.

An additional rationale for an interview-based design is its value as a contribution to the knowledge base. A review of the literature discovered only one similar research design, a study of first-time adult computer users based on phenomenological interviews (Howard, 1994). Howard’s study, while it moves too hastily from the interpretation of empirical data to philosophical speculation, is a useful demonstration of the value of this under-utilized approach. By presenting and analyzing additional interview data about learning in organizational settings, the present study may facilitate efforts to synthesize prior findings and support computer training practice.

Population and Sampling

The most basic decision in an interview-based study is the selection of participants (Johnson, 1990). In qualitative research, this process has different goals than the probabilistic sampling methods employed in quantitative research:

Accordingly, informants are chosen on the basis of their "theoretical qualifications ... in terms of such things as status, role, position, expertise, category or subgroup membership, dimensions, and even knowledge" (Johnson, 1990, p. 38).

The population of interest in this study includes adults who use personal computers routinely in their work but who are not academically trained computer professionals. ‘Routinely in their work’ in this context means that a user’s computer tasks are recurrent and necessary to achieve basic functions of the user’s job. Individuals who have received or are pursuing degrees in computer-specific disciplines (e.g., computer science, electrical engineering, management information systems, instructional technology) were excluded on the grounds that such people’s experiences are not representative of the majority of personal computer users in the workforce. The selection of participants was intended to include a sufficiently broad range of individuals, organizational types, and computer installations to warrant generalizations about this population.

To collect data about diverse organizational situations, the sample had to include both public- and private-sector organizations of varying sizes and computer resources. My work in computer training at the University of Georgia Center for Continuing Education helped me make initial contacts and establish the credibility of the research. Three organizations agreed to participate in the study: a non-teaching department of the university; a privately-owned hospital; and the corporate office for a chain of regional newspapers.

In the case of the university department, access was facilitated by my previous contacts with the staff as advisor to a student sports club. I sent an invitational letter (Appendix 1) to ‘Mel Daniels,’ the facilities manager for ‘Leisure Activities,’ who identified other potential participants during a follow-up telephone conversation. ‘Elizabeth Burton,’ a secretary at ‘County Hospital,’ received the same letter and then identified two other interview participants when we talked on the telephone. Elizabeth was familiar with my department at the Georgia Center through her informal activities as a coordinator of computer workshops for hospital staff; however, I had no direct participation in this training nor any contact with the interview participants at the hospital prior to this study. Access to ‘Regional Publishing, Inc.’ was facilitated by my department head, Dr. Helen Mills, through her membership in a civic group with the president of that organization. The president identified ‘John Gilliam,’ RPI’s controller and computer systems administrator, as an appropriate contact person. Again, the invitational letter was followed by a phone conversation in which John and I discussed the selection of other interview participants.

In both the letter and the telephone contacts, I described the research problem and emphasized my desire to talk to computer users who held different positions within the organization, who represented various levels of computer experience, and who had opportunities to interact with one another in the course of their computer work. I worked with each contact person to select potential participants who met these criteria and who could make themselves available for a 30-60 minute interview.

At Leisure Activities, I interviewed four of the department’s six staff members (as a result of audiotaping problems, only three of these interviews provided transcript data). At RPI, I interviewed four people out of a staff of nine. The hospital’s department of ‘Outpatient Therapy’ has a staff of about 40 people, most of whom do not use computers. I interviewed three computer users in that department and two other hospital employees: a secretary in another department who had once been a temporary clerical worker in Outpatient Therapy, and the director of internal auditing. In all three cases, I interviewed about half of the computer users in each workgroup, including managers, professionals, and support staff. A total of 12 people of different job requirements, educational backgrounds, ages, and genders participated in the interviews. Participants ranged in computer skill from near-novices to an experienced network administrator. Background data about the participants are presented in Tables 2, 3, and 4 of Chapter IV.

In the cases of Leisure Activities and Outpatient Therapy, the first round of interviews was followed by a second set of additional interviews. Discussions with members of the advisory committee early in the data collection process made clear the need to ensure that all the cases paralleled each other in terms of the number of individuals interviewed and the representation of different organizational positions (i.e., support staff, professionals, and supervisors). In the case of one organization (the hospital) individuals outside the department who had relevant information were also interviewed. These second rounds of interviews reinforced the parallelism of the cases and also allowed further observations of the workplace environments. A second round of interviews at RPI was unnecessary, since these criteria were satisfied in the first set of interviews.

Personal computers used in these organizations included networked and non-networked DOS, Windows, and Macintosh computers running a variety of standard word processing, spreadsheet, database, and desktop-publishing applications. Currently these are the personal computers and types of applications most widely used in the United States.

Data Collection

Interviews were conducted on-site over a four-month period. All but two were conducted in participants’ offices or work areas so that they had access to their computers during the interviews and so that I could observe their work environments. In the other two interviews, another room in the department was used for greater privacy, but I was still able to observe the participants’ work areas.

At the beginning of each interview, participants were asked to sign a consent form approved by the University’s Institutional Research Board. This form described procedures for protecting the anonymity of their responses. Participants were advised that they could terminate the interviews at any time or withdraw their interviews from the study at a later date. They also received instructions on how to contact me for further information or copies of the findings.

The interviews included two stages. First I filled out a survey-style data sheet (Appendix 2), collecting basic information about the participant and the organization. The second stage was an in-depth, audiotaped interview lasting 30-90 minutes, using a simple interview guide to elicit stories, explanations, and statements about experiences of computer skill learning.

The interview guide (Figure 1) served primarily as a memory aid. The questions were derived from a pilot study, from informal conversations with many computer users, from the research questions for this study, and from recommendations by the advisory committee. Participants were invited to talk about events that they perceived as important in their computer skill learning, about their strategies for coping with computer problems, and about the positive and negative effects of their organizational settings on their learning. The structure of the interview guide did not determine the content or sequence of all questions and responses during the interviews. Alternative lines of questioning were explored if potentially relevant issues surfaced. For example, in the case of County Hospital, participants’ comments about the MIS department led to extensive discussion about user-support issues not anticipated in the interview guide. Throughout the interviews, I avoided the use of jargon or other language that might direct responses and used restatement and rephrased questions to prompt for elaboration and to confirm understandings (Merriam, 1991).

As soon as possible after each round, the interviews were transcribed by me or by an assistant. I reviewed printed transcripts and checked them against the audiotapes to verify or correct questionable readings. Pseudonyms were substituted for personal and organizational names throughout the transcripts, and some other references that might allow identification of individuals were also edited. The 12 resulting text files (totaling about 42,000 words) were the core data for the study.

Observations conducted on-site at the workplaces provided an additional data source to triangulate interview data. The form in Figure 2 was used to record observations after each site visit. These notes served as memory prompts while writing the "thick" descriptive passages prefacing each of the three case analyses in Chapter IV (cf. Geertz, 1973).

A walk-through tour of each organization early in the interview process revealed material details of office design, distribution of computer resources, and patterns of computer activity. By interviewing most participants at their desks, it was also possible to observe details of their personal work environments. Several participants responded to questions by using their computers to demonstrate or display the software and files being discussed. Such observations provided useful validity checks for participants’ statements about their skills and attitudes.

How are personal computers used in the organization?

Who was using computers during the visit?

Describe any incidents of computer use that involved more than one person.

Describe the overall computer installation of the site.

Describe the participant’s work space and computer system.

Describe any use of the computer during the interview.

Describe any reference materials in the work area.

Describe any characteristics of the participant that influence computer use (e.g., a disability).

Data Analysis

Unlike quantitative methods, procedures for the analysis of qualitative data cannot be completely specified prior to data collection, since the form of the data is unpredictable. As a result, qualitative studies do not usually meet one of the most common standards of research reliability: the ability to be replicated. However, advocates of qualitative methodologies contend that much quantitative research emphasizes reliability at the cost of construct validity, resulting in the production of elegantly-tested findings that have little or no practical relevance (Edwards & Potter, 1992; Mishler, 1986b; Potter & Wetherell, 1987). One of the primary goals of this and other qualitative studies is to achieve valid results by using methods that respect the socially constructive nature of discourse and practice.

Traditional standards of scientific inquiry and scholarly publication require that the processes and findings of research be presented in such a way that an informed audience can judge its value. For the qualitative researcher, these standards impose an obligation to document as clearly as possible how data were collected and analyzed and how specific interpretations are warranted (Kirk & Miller, 1986). This section identifies presuppositions that shaped the research and its findings, outlines the procedures used to carry out within-case and cross-case analyses, and describes the measures taken to assess the validity of the findings.

In an interview, both the researcher and the participant contribute to the construction of meaning. In spite of all efforts at objectivity, the data collected can be heavily influenced by the questions asked and by the participant’s perception of the researcher’s motives and expectations in asking those questions (Mishler, 1986b). Thus the agenda and concepts that guided the research effort must be stated explicitly, so that readers of the study can assess their possible influence on what data was collected and how it was analyzed.

Chapter I describes the context of the research problem, states the purpose of the study, and defines several critical concepts: ‘personal computer,’ ‘applications,’ ‘novice,’ ‘experienced user,’ and ‘computer skill.’ These premises and concepts were accepted as given. The questions in the interview guide (Figure 1) reflect the assumption that certain events and experiences are central to computer skill learning, e.g., ‘getting started,’ ‘coping with problems,’ and ‘learning within the organization.’ The phrasing of these questions presupposed that participants would tend to describe their computer experiences in the form of stories and explanations. Just such stories and explanations comprise the bulk of the data.

The initial stages of data analysis were concurrent with data collection. As interviews at each organization were completed, the tapes were transcribed and the printed data read repeatedly and annotated. A tentative analysis of each case was made before conducting interviews in the next organization, which allowed emerging themes to be tested across the cases.

The first set of interviews, conducted at the university’s Leisure Activities department, provided an opportunity to test analytic strategies and to receive feedback from two members of the advisory committee. Following the strategy proposed in the prospectus for this study, excerpts from the interviews were analyzed in terms of structuralist theories of narrative (Barthes, 1982; Chatman, 1978; Ducrot & Todorov, 1979). The analysis described each excerpt’s internal structure (how narrative elements such as characters, actions, happenings, and settings were combined and organized) and pragmatic function (how the narrative expressed the narrator’s beliefs, attributions, goals, and intentions). Discussions of this analysis with the advisory committee members produced a list of common themes that was useful in analyzing data from the other cases.

This initial effort persuaded me that structuralist analysis would not be applicable for the data in this study. Designed for the detailed explication of dense, written narratives, these methods cannot be imposed on the looser structure of speech without distorting the data. While structuralist procedures were useful in identifying passages of talk as narrative or explanatory units, I decided to concentrate on the identification of recurring themes, concepts, and experiences by making constant comparisons between and within the interview texts (Dey, 1993; Glaser & Strauss, 1967).

After completing and transcribing interviews for the second case, County Hospital, new themes were identified and others were modified. Some of the categories of learning experience described by the Leisure Activities participants also applied to the data for the second case. The desire to see if these categories would recur in a smaller, more profit-driven organization led to the selection of Regional Publishing, Inc., as the last of the three cases.

With all 12 transcripts in hand, the final within-case and cross-case analyses were prepared on a Macintosh computer using the applications Microsoft Word and FileMaker Pro. The within-case analyses were conducted in the same sequence as the original site visits. For each case, the transcripts were reviewed and selected excerpts from the interviews were copied from the Word text files into FileMaker Pro database records. Each database record included fields containing the following information: a code that identified each record; the participant’s name; a category; researcher comments; and the text of the excerpt. Figure 3 shows the database record for an excerpt from one of the County Hospital interviews.

Alexandra

Becoming a Novice

Alexandra was thrown into computer use when she started a previous job. None of the staff received any formal training or support. "As we needed it, we'd try to figure out how to use it."

I took a job as an administrative assistant and they sat me down in front of it and they said, "Here is your computer." I didn't even know how to turn it on. I was very lucky to be hired for the job. Basically, you know, someone showed me how to get around and then I took it from there. Sat with the books and pretty much taught myself -- used help a lot.

As each record was entered into the database, it was assigned to a category. The category descriptors were phrases that served to link records about related forms of experience. Throughout the within-case and cross-case analyses, the list of categories was iteratively reviewed and revised in order to create new categories only when necessary to accommodate the data and to combine related categories whenever their contents could be subsumed under a more inclusive classification. The comments field was used to summarize the excerpt and explain its category assignment. These comments provided data for an ongoing audit of the analysis, making it possible to review the evolution of the categories and the reasons for various interpretations.

Once the data for a case were placed in the database, it became possible to find and sort the records by participant or by category. In writing the within-case analyses, Word and FileMaker Pro were used simultaneously, allowing the copying of data into the research report and the ongoing revision of the category list. The Find, Replace, and Index features of FileMaker Pro simplified the process of modifying category descriptors and redistributing records among the categories.

After the within-case analyses were completed, the database contained a total of 113 records. The first step in the cross-case analysis was to create a list of the category descriptors in Microsoft Word. The Search feature of FileMaker Pro was used to determine how many records had been assigned to each category, and this information was used to sort the list of descriptors according to the frequency of records in each category. This reordering helped to focus attention on categories that contained only one or two records. These categories were subsu