© 2008 INFORMS
Using Self-Regulatory Learning to Enhance
E-Learning-Based Information Technology Training
Radhika Santhanam
Gatton College of Business and Economics, University of Kentucky, Lexington, Kentucky 40506, [email protected]
Sharath Sasidharan
Lewis College of Business, Marshall University, Huntington, West Virginia 25755 [email protected]
Jane Webster
Queen’s School of Business, Queen’s University, Kingston, Ontario Canada K7L 3N6, [email protected]
T
echnology-mediated learning methods are widely used by organizations and educational institutions to deliver information technology training. One form of technology-mediated learning, e-learning, in which the platform is the tutor, is quickly becoming the cost-effective solution of choice for many corporations. Unfor-tunately, the learning outcomes have been very disappointing. E-learning training makes an implicit assumption that learners can apply a high level of self-directed learning to assimilate the training content. In contrast, based on perspectives from social cognitive theory, we propose that instructional strategies need to persuade learn-ers to follow self-regulated learning strategies. We test our ideas with participants who were trained through e-learning to design a website. Our findings indicate that participants who were induced to follow self-regulated learning strategies scored significantly higher on learning outcomes than those who were not persuaded to do so. We discuss our findings, and suggest that the interaction among information technology features, instruc-tional strategies, and psychological learning processes offers a fruitful avenue for future information systems training research.Key words: e-learning; laboratory experimentation; information technology training; self-regulatory learning;
social cognitive perspective; pretraining scripts; website development; self-efficacy
History: Ritu Agarwal, Senior Editor; H. Raghav Rao, Associate Editor. This paper was received on August 19,
2005, and was with the authors 8 months for 3 revisions.
Introduction
Almost every employee today has to be skilled in using information technology (IT); therefore, organi-zations continue to invest significantly in IT training for their employees, and universities offer many IT training courses for students (Agarwal and Ferratt 2001, Bureau of Labor Statistics 2004, Homer and Povar 2004, Piccoli et al. 2001). Given the extensive deployment of IT training in corporations and institu-tions, it is not surprising that information systems (IS) researchers expend substantial efforts to identify the most effective training methods and strategies (e.g., Agarwal et al. 2000, Bostrom et al. 1990, Compeau et al. 1995, Compeau and Higgins 1995a, Johnson and Marakas 2000, Olfman and Mandviwalla 1994a, Lim
et al. 1997, Santhanam and Sein 1994, Venkatesh 1999, Yi and Davis 2003).
In recent years, IT training is increasingly being delivered through electronic means such as tech-nology-mediated learning (TML) methods instead of through face-to face interactions between learners and trainers. TML as a training vehicle is growing rapidly because it is seen as a cost-effective way to deliver training at convenient times and at remote locations to large numbers of employees and students (Zhang et al. 2004). A study by the Sloan Consortium finds that educational institutions use TML extensively: 81% offer at least one fully online or blended course, and 34% offer complete technology-based degree pro-grams (Allen and Seaman 2005). In practice, however,
TML has not provided the benefits that were orig-inally anticipated; corporations, educational institu-tions, and IS researchers are thus very motivated to search for ways to improve TML effectiveness (Alavi and Leidner 2001, Gupta and Bostrom 2005, Moller 2002, Olfman et al. 2006, Sasidharan and Santhanam 2006).
IS researchers emphasize the exigency of conduct-ing research on TML for several reasons. First, this research can build on and contribute to the cumu-lative knowledge developed by IS research on the interaction of IT and human problem solving and learning processes (Alavi and Leidner 2001, Leidner and Jarvenpaa 1995). Second, because IT training is an integral component of IS research, existing find-ings on how best to develop and test IT skills could contribute to our understanding of whether IT infras-tructures are effective IT training platforms (Gupta and Bostrom 2005, Olfman et al. 2006). It can also help us research IT software aspects that provide new opportunities for the control and pacing of user learn-ing. Finally, enhancing IT skills of students through TML is a research topic of growing interest to IS aca-demics as evidenced by research articles, specialized journals, and special interest groups on IT education (Hardaway and Scamell 2005, Leidner and Jarvenpaa 1995, Markus 2005). Consequently, there is a need to study the role of TML as an effective IT training deliv-ery mechanism (Gupta and Bostrom 2005, Olfman et al. 2006, Salas and Canon-Bowers 2001, Zhang et al. 2004).
We believe that IS researchers, with their multi-disciplinary focus and ability to integrate social and cognitive processes with technology affordances, are uniquely qualified to study TML. In this research, we address the role of TML as an IT training mecha-nism in the following manner: (1) We draw on a key learning-through-technology framework (Alavi and Leidner 2001) that has not received much empirical attention; (2) we examine one specific TML environ-ment and the underlying instructional strategies in order to identify methods that can improve IT train-ing outcomes; and (3) we propose and test interven-tions based on social cognitive (SC) theory (Bandura 1991) that can induce learners to self-regulate their learning and enhance their IT skills via TML training. Combined, this approach provides a foundation for
future work in both learning through IT and learning about IT.
We focus our attention on the specific TML envi-ronment, referred to as e-learning, where the learner interacts primarily with the IT platform rather than with other learners or instructors (Jonassen and Reeves 2001, Jones and Paolucci 1999, Zhang et al. 2004). As we elaborate in the next section, the instruc-tional strategy in this type of training infrastruc-ture is anchored on self-directed and independent learning. The implicit assumption is that learners are able to meet the demands expected of the instruc-tional strategy, apply high levels of learner con-trol, and self-direct their learning. Reports indicate, however, that learners are not able to apply the anticipated high levels of learner control, are not motivated to learn, and tend to use inadequate learn-ing strategies (Bell and Kozlowski 2002, Brown 2001, Rossett and Schafer 2003). Therefore, we argue that if e-learning is to become an efficacious IT train-ing method, instructional strategies should be mod-ified to include interventions that persuade learners to follow self-regulated learning (SRL) strategies. In the next section, we describe how the social cogni-tive perspeccogni-tive on self-regulation can help design interventions that modify instructional strategies in e-learning-based training environments. We then report on an experiment in which we provided manipulations to encourage learners to self-regulate while learning to use Website-development software. We conclude the paper by discussing the implications of the study for practice and future research.
Research Framework
In this section, we first explain the different terms used in reference to TML. We then review TML research and describe the learning-through-technology frame-work proposed by Alavi and Leidner (2001). Finally, we propose that e-learning-based training could be improved by paying attention to self-regulation, and elaborate on how it could be applied by proposing specific hypotheses to be tested.
Characteristics of TML and E-Learning
TML refers to an environment in which IT is used to mediate/support teaching and course delivery and
includes a variety of learning environments: differ-ent functions and features of IT may be selectively applied, instructional packages may be bundled in various ways, and learners may be provided with dif-ferent levels of control (Benbunan-Fich 2002, Bostrom 2003, Jonassen 2004). E-learning describes a TML environment in which a single user interacts with technology and attempts to self-direct and complete a training course (Zhang et al. 2004).
In Figure 1, we list a few commonly used terms and corresponding descriptions related to e-learning. Vir-tual learning is a broad term to refer to computer-based environments with a wide range of resources made available to learners (Anohina 2005, Piccoli et al.
Figure 1 E-Learning-Related Terms and Descriptions Virtual Learning
• An encompassing term denoting computer based instructional environments that are relatively open systems with a wide range of resources that learners can use to interact with other learners and instructors. Some researchers refer to this as Web-based learning or online learning if educational content is transferred via the Web browser and computers connected to the intranet.
• Delivery of educational content via a Web browser over the public internet, a private intranet, or an extranet. • Training via computers connected to the World Wide Web.
Technology-Mediated Learning (TML)
• A term used in instructional environments where information technologies are used to support course delivery and to manage the teaching and the learning process. It can refer to a broad set of applications such as learning labs, teleconferencing systems, computer-mediated communications, collaborative learning systems, etc. Though some IS researchers refer to this as e-learning, most disciplines make a distinction between TML
(learning with computers) vs. technology-based learning (learning from computers). When information technology is used primarily to support the learning process but course content is not necessarily delivered via computers, it tends to be referred to as TML.
Technology Based Learning/Training (TBL/TBT)
• A term used when the delivery of course content is via computer technology. This is thus referred to as learning from computers. Within this are two groupings of terms: distance learning and e-learning.
Distance Education/Learning
• Separation of teacher and learner in space and/or time • Provision for two-way communication
• Educational institution typically provides certification on course completion • May involve some classroom teaching as well
• May also include learners engaged in group learning E-Learning
• User engaged in self-paced learning with learner in control. Learning package is delivered or transacted through electronic means, sometimes referred to as computer microworlds.
• If e-learning is conducted through stand-alone computers, it is generally referred to as Computer-Based Training/Computer-Assisted Instruction.
• If e-learning is delivered through the internet, intranet (or extranet), or a Transmission Control Protocol/Internet Protocol network, it is referred to as
Internet-based training/learning.
• If e-learning involves use of communication technologies to facilitate discussions among learners, it is referred to as collaborative e-learning.
Note. Figure adapted from Anohina (2005). Descriptions obtained from Anohina (2005), Alavi and Leidner (2001), Piccoli et al. (2001), Jonassen (2004), and Keegan (1996).
2001). Technology-based learning, or learning from computers, encompasses both distance learning and e-learning because technology is used in both as the pri-mary medium to deliver course content (Gupta and Bostrom 2005, Jonassen and Reeves 2001, Jones and Paolucci 1999). In distance learning, the teacher and learner are separated in space and time, but they can communicate with one another; in many implementa-tions, learners can communicate with other learners as well. Among TML contexts, e-learning provides max-imal control to learners so that they can learn in an independent manner. However, as seen in Figure 1, in e-learning the learner may be unable to communicate with the instructor or with other learners.
An e-learning infrastructure typically consists of a database-centric learning content-management sys-tem that lists the course catalogs (from which the learner can choose a specific course), registers the learner, and manages the interaction between user and system (Bostrom 2003). A typical IT training course, such as an introductory course on website develop-ment, might consist of several sequential learning modules, with each module providing instructional information on one topic related to designing a web-site. The first module may consist of general infor-mation on websites, the second on features of the specific software; the third may introduce procedures to develop a website, and so on. Advanced courses could comprise a dozen such modules.
Prior Research on TML
The general challenges of TML-based training have been addressed by researchers in other disciplines, but little attention has been paid to IT skill develop-ment courses (e.g., Bernard et al. 2004, Jonassen 2004, Jones and Paolucci 1999). Factors impacting distance learning effectiveness have been examined even from the days of correspondence courses, with recent stud-ies focusing on technology-mediated environments and their implications for the new roles of teach-ers and students (Berge and Mrozowski 2001, Keegan 1996, Noffsinger 1926). A meta-analytic review of over 200 research studies on distance learning concludes, among other findings, that learning outcomes can be enhanced by paying attention to specific features of the instructional environment and to the proper use of instructional strategies (Bernard et al. 2004). Other reviews of TML also point to the importance of pay-ing attention to instructional strategies and to the spe-cific features in a TML environment. A review that summarizes research findings comparing TML con-texts involving groups of learners versus individual learners concludes that TML courses are not supe-rior overall, but that group learning with TML has more favorable effects than individual learning from TML. This suggests that feedback, the social con-text, instructional strategies, and interaction among learners can maximize learning outcomes (Lou et al. 2001). Unfortunately, many commercial technologies are based on an individual learning model, not on a collaborative learning model (Benbunan-Fich and
Hiltz 2003, Richardson and Swan 2003). This points to the importance of paying attention to instructional strategies in specific TML environments, and to the special challenges in contexts where learners have to learn independently without opportunities to collab-orate with other learners. As is evident from Figure 1, e-learning, the technology examined in this study, represents such an environment where learners do not get opportunities to interact with other learners.
A review of TML research in IS indicates that, sim-ilar to other disciplines, IT skill development has not been given much attention (Alavi 1994; Alavi et al. 1995, 2002; Coppola et al. 2002; Leidner and Fuller 1997; Sasidharan and Santhanam 2006). Most of these studies test TML in collaborative learning contexts and, as in other disciplines, conclude that using TML methods does not automatically lead to superior learning outcomes: attention must be given to instructional strategies and specifics of the TML context. Hence, as articulated by Alavi and Leidner (2001), learning outcomes can be increased by paying attention to the interaction of three key factors in spe-cific TML environments, namely, information technol-ogy, instructional strategy, and learners’ psychological processes. Except for a study by Piccoli et al. (2001), little research has addressed these factors and their impact on learning outcomes.
Based on Alavi and Leidner’s (2001) framework, we argue that, because IT features in e-learning often do not permit learners to interact with other learn-ers or instructors, the IT platform becomes the dom-inant mode of communication with the learner. The accompanying instructional strategy relies on self-directed learning with the assumption that learners can independently regulate their learning and absorb the training content. In other words, the conditions of training are established to evoke a high level of self-directed learning: the training design is based on an objectivist learning model, which assumes that learn-ers learn best in an isolated and intensive manner by regulating their own learning (Gagne 1977, Gagne et al. 1992, Leidner and Jarvenpaa 1995). But such a design imposes a very high burden on the learner: to be motivated and focused on learning without any guidance from human instructors or other learners. Even in e-learning environments where learners can interact with other learners and instructors, learners
complain that they find it difficult to take on the responsibility for directing their own learning (Piccoli et al. 2001). Empirical observations attest that learn-ers situated in e-learning environments do not ade-quately self-direct their learning, nor do they exercise high levels of learner control, which might explain the higher drop-out rate (Allen and Seaman 2005, Bell and Kozlowski 2002, Brown 2001, Zhang et al. 2004). Therefore, as described below, we propose that the instructional strategy, or the sequence of activities that systematically exposes learners to experiences to help them acquire knowledge, must be modified to enhance the effectiveness of e-learning-based training. Self-Regulation and E-Learning
The interaction between IT features and instructional strategies necessitates that the learner exhibits an e-learning strategy that includes high learner control, self-discipline, and self-motivation. We propose that learning outcomes will be enhanced if instructional strategies are modified to include interventions that instruct learners to follow self-regulatory learning strategies that include, among other things, encour-aging learners to believe that they can learn through e-learning training, enhancing their motivation to learn, formulating appropriate goals for the course, and devising methods for organizing course con-tent. In other words, instructional strategies should include interventions that instruct learners to apply self-regulation in their learning.
Self-regulation refers to a general skill that keeps people focused on a task, helps them monitor their task-completion progress, and explains success in a broad range of phenomena, for example, management of chronic illnesses, training for sports, treatment of obsessive behaviors, and learning in academic set-tings (Bandura 1991, Boekaerts et al. 2000). Partic-ularly in academic environments, researchers have found that students who self-regulate their learning reach higher academic achievements irrespective of their courses of study (Pintrich and DeGroot 1990, Zimmerman et al. 1992, Zimmerman and Schunk 2001). SRL involves strategies by which learners actively engage in learning and apply intentional efforts to manage and direct their learning activities. Zimmerman and Schunk (2001) review various the-oretical perspectives on SRL, including the operant,
Vygotskian, constructivist, volitional, phenomenolog-ical, information processing, and social cognitive. All of these perspectives tend to view SRL as something purposive that involves the use of specific strategies, but they differ on the factors they view as being rel-evant to a learner’s use of SRL. For example, oper-ant theorists emphasize the role of external factors, suggesting that learning responses are ultimately con-trolled by external reward/punishment contingencies such as verbal coaching and reinforcement. Informa-tion processing theorists emphasize the role of three types of memory (sensory, short term, and long term) in the SRL process. Learning strategies in this per-spective emphasize learners’ ability to cluster bits of information into larger units and develop their capacity to process information. In contrast with the external factors highlighted in the operant theory, the social cognitive perspective takes an agency perspec-tive and highlights individuals’ roles and their own ability to enact self-regulation. It states that learners by themselves can self-regulate through their specific use of strategies toward learning, and emphasizes that several factors, such as learners’ self-efficacy beliefs, their motivation to learn, and their learning goals and strategies, must be addressed in combi-nation (Kauffman 2004, Pintrich and DeGroot 1990, Schunk 2001, Zimmerman 1989). We focus on this perspective because it takes a collective approach to addressing SRL, and the SC theory has been used fre-quently in IS research.
The Social Cognitive Perspective on SRL
The salient aspects of the SC perspective include, among other factors, learners’ motivations, their out-come expectancies, and their perceived self-efficacy beliefs (Schunk 2001, Zimmerman 2000). Learners’ perceived self-efficacy beliefs (i.e., beliefs about their capabilities to learn) are essential factors that affect all phases of self-regulation, and are formed based on prior observations or prior performance or through some form of persuasion (Schunk and Ertmer 1999, 2000; Zimmerman et al. 1992). The SC perspective also emphasizes the situation-specific nature of SRL; that is, learners may not engage in self-regulation equally in all learning environments (Schunk 2001). For example, research indicates that, although high achievers tend to use SRL strategies, they may not
apply them in every situation (Schunk and Ertmer 2000, Zimmerman and Martinez-Pons 1990). There-fore, SC theorists emphasize the situational task goals of the learner, such as completing a specific home-work assignment, and situation-specific self-efficacy beliefs, such as beliefs in their abilities to solve fraction problems in arithmetic (Schunk 2001, Zimmerman and Schunk 2001). Although some SRL strategies may gen-eralize across settings, learners must know how to adapt to situation-specific domains and feel competent in doing so.
In line with the situation specificity of SRL, fore-thought is described as an important period that occurs before learning and sets the stage for action. Motivations and preparations during forethought in-fluence learners’ level of engagement with the learn-ing task and impact learnlearn-ing outcomes (Schunk and Ertmer 2000). During learning, outcomes can be im-proved by providing performance-based feedback and facilitating learners’ metacognitive monitoring (Bell and Kozlowski 2002, Schmidt and Ford 2003). But addressing only metacognition or other isolated
Figure 2 Phases and Beliefs in Self-Regulatory Learning
Performance or volitional control Forethought task analysis motivational beliefs Self-reflection and adaptation
Phase Structure and Sub-Processes of Self-Regulation Cyclical self-regulatory phases
Forethought Performance/volitional control Self-reflection
Task analysis Self-control Self-judgment
Goal setting Self-instruction Self-evaluation
Strategic planning Imagery Causal attribution
Seft-motivation beliefs Attention focusing Self-reaction
Self-efficacy Task strategies Self-satisfaction/affect
Outcome expectations Self-observation Adaptive defensive
Instrinsic interest/value Self-recording Goal orientation Self-experimentation
Note. Adapted from Zimmerman (2000). This article was published in the Handbook of Self-Regulation, edited by M. Boekarts, P. Pintrich, and M. Zeidner, Attaining self-regulation: A social cognitive perspective, 13–39, copyright Elsevier, 2000.
single components of learning may not be enough; instead, a collective (i.e., multifaceted) approach is useful because learners who discover a lack of pro-gress through metacognitive monitoring also need some motivational regulation to continue their learn-ing efforts (Pintrich and DeGroot 1990, Zimmerman et al. 1992).
Most individuals use some level of self-regulation, but they differ on the quality and extent to which they apply it in specific contexts (Zimmerman 2000). This depends on their knowledge of SRL strategies, their decisions to use known strategies, and their ability to use these strategies skillfully (Schunk and Ertmer 1999). Thus, adopting an SC perspective considers not only learners’ choices of cognitive strategies, but also their fears, doubts, confidence, self-beliefs, and sense of personal agency in specific performance contexts (Zimmerman 1995, Zimmerman and Martinez-Pons 1988). Based on our adoption of the SC perspective on SRL, we believe that because learners’ use of SRL has to be activated in a specific situation (such as while undergoing e-learning-based training),
context-specific interventions that seek to enhance learner motivation and that instruct learners to follow SRL strategies could improve learning outcomes.
Development of Hypotheses
We draw on a process model of SRL (shown in Fig-ure 2) to design an instructional strategy that can help to persuade learners to apply SRL during e-learning training (Zimmerman 2000). In the model, SRL can occur through three cyclical phases: forethought, voli-tional control, and self-reflection. Although this is a process model suggesting how SRL could be affected, it can be observed from Figure 2 that the model includes the multifaceted aspects of SRL. In the fore-thought phase, the learners’ learning goals, motiva-tion to learn, self-efficacy beliefs, and learning plan should be addressed. In the performance/volitional control phase, the learner focuses on the learning task, applies cognitive strategies like note taking, and organizes study materials, thereby self-directing their learning. During self-reflection, learners conduct metacognitive monitoring of their learning progress and adapt their strategies.
One could use this model to design an instruc-tional strategy such that a pretraining intervention occurs at the forethought phase, when learners get engaged in task analysis through which they under-stand the learning plans and goals for the train-ing session. Hence, before the learner begins the e-learning tutorial, trainers could provide a task anal-ysis for the training session by describing learning plans and goals for the session. During the fore-thought phase, learners must also develop motiva-tional beliefs toward training and, once again, trainers could influence these motivational beliefs. This could be achieved if trainers were to provide learners with scripts that inform them about the various modules in the e-learning course, the goals and elements of each module, and a list of what they should have learned by the end of each module. The scripts could also be designed to boost learners’ motivational beliefs, out-come expectations, and interest in learning through e-learning.
When learners start on the actual learning task, they should engage in performance/volitional con-trol activities such as attention focusing and self-recording, and self-observational strategies such as
note taking. Similarly, scripts could also be used to influence these activities by instructing learners to apply the cognitive strategies of self-directing, record-ing, note takrecord-ing, etc. As learning progresses, learners should engage in metacognitive activity and evaluate whether their strategies are working and, if neces-sary, adapt. Either of two methods can be effective in accomplishing this: (1) learners could self-evaluate and reflect of their own volition or (2) they could be provided with external evaluations and feedback (Zimmerman and Kitsantas 1997). Therefore, one way to help learners reflect on their strategies could be via external feedback emphasizing the success of the learners’ self-regulatory strategies. The feedback could be used to set goals for the rest of the training session and enhance learners’ motivation. This type of feedback and positive evaluation is an integral aspect of self-regulation that reinforces the use of SRL strate-gies and sustains motivation and self-efficacy beliefs (Schunk 2001, Schunk and Ertmer 1999). Therefore, we propose two hypotheses:
Hypothesis 1A. In an e-learning-based IT training
session, presession interventions designed to increase the learner’s use of self-regulatory learning strategies and accompanying beliefs will enhance learning outcomes.
Hypothesis 1B. In an e-learning-based IT training
ses-sion, interventions that provide positive feedback to learners on their use of self-regulatory strategies will enhance learn-ing outcomes.
Learner Characteristics
Researchers propose that certain dimensions of SRL can be influenced by stable dispositional traits: if a learner has a higher ability to set goals, to apply meta-cognitive strategies, and to self-regulate, etc., those skills could help the learner apply SRL and achieve higher learning outcomes (Ford et al. 1998, Schmidt and Ford 2003, Winnie 1995). In IS research, individ-ual learner traits are noted as influencing training outcomes (Agarwal et al. 2000, Bostrom et al. 1990, Webster and Martocchio 1992). Studies on distance learning also indicate that learner characteristics may influence learning outcomes and should be consid-ered (Bernard et al. 2004). Therefore, it is worthwhile to examine whether individual characteristics could influence e-learning-based training outcomes.
Because the focus of this study is on SRL and the training interventions attendant on SRL, we were interested specifically in examining learner character-istics that might relate to SRL or to behaviors in a new learning environment. As shown in Figure 2, individ-ual goal orientation could influence performance and volitional control. Beaubien and Payne (1999) describe goal orientation as an important individual motiva-tional variable that explains how people develop com-petence in new learning and performance situations. Although first understood as a two-factor structure, goal orientation is now described as comprising three factors: learning orientation, performance approach, and performance avoidance orientations (VandeWalle 1997, Zweig and Webster 2004). For the purposes of this study, the learning orientation factor is relevant because e-learning training represents a new learning task. People with high learning orientations are more likely to find new learning tasks challenging and to view learning performance as indicative of mastery (Brett and VandeWalle 1999, Fisher and Ford 1998, VandeWalle 1997). Therefore, we propose the follow-ing hypothesis:
Hypothesis 2A. In an e-learning-based IT training
session, learning orientation will be positively associated with learning outcomes.
Self-efficacy beliefs are important motivational beliefs in SRL (see Figure 2). In IS training research, self-efficacy beliefs are identified as a critical predictor of learning outcomes (Agarwal et al. 2000, Colquitt et al. 2000, Compeau and Higgins 1995b) and, as described earlier, task-specific self-efficacy beliefs are considered to be relevant (Bandura 1997). In this study, learners were trained through a computer-based program, so it can be expected that individ-uals’ self-efficacy beliefs regarding learning through computers may influence learning outcomes. In addi-tion, individuals differ in their ability to self-regulate learning, because this ability is developed over a life-time through social and other influences (Schunk and Zimmerman 1997). Users may understandably have differing self-efficacy beliefs regarding their ability to self-regulate learning and this could influence learn-ing outcomes. Hence, we propose the followlearn-ing:
Hypothesis 2B. In an e-learning-based IT training
session, computer-learning self-efficacy will be positively associated with learning outcomes.
Hypothesis 2C. In an e-learning-based IT training
session, self-efficacy for self-regulatory learning will be pos-itively associated with learning outcomes.
Research Method
To test our hypotheses, we chose a laboratory exper-iment as the suitable approach. We performed a pilot test before we conducted the experiment (see Appendix 1).
Facilities, Target IT Application, and Participants Using an e-learning platform used in training em-ployees and students, we conducted the experiment at a large public university. The IT training pack-age provided a course on website design and devel-opment with Microsoft FrontPage software. In this course, learners proceeded through several modules as they underwent training in the conceptual ele-ments and procedures used to set up websites. The training laboratory was similar to an IT training room, with 16 standalone computers in individual cubicles. Each participant, wearing a headset, completed the course independently, without interference from other learners. Training was interactive and delivered by audio and video components. The participant sequen-tially activated and completed the learning modules.
For our experiment, we selected eight consecutive modules from the course, providing enough informa-tion content to develop a website with linked pages and internal and external links. Each screen within a course module displayed a typical FrontPage inter-face with a box that described the screen layout and menu options. The participant was asked to choose one of the menu options, to type text, and to exe-cute appropriate actions. The participant viewed the results. If satisfied, the participant hit the forward button and proceeded to the next screen, but could move backward or forward within the module at any time. After a module was completed, the participant moved on to the next one. Undergraduate business students participated over four semesters and were given course credit for volunteering. Students who had web-design experience were excluded, but were given course credit through other means. We next describe our experimental procedures and then pro-vide details on the interventions and measures used in this study.
Experimental Procedures
Immediately after students volunteered for the study, they were given a background information survey that measured their individual learner traits and de-mographics (see Appendix 2). They were then as-signed to one of the training sessions. Prior to starting the e-learning training, participants received either a treatment or control script. During the training ses-sion, they received feedback in the form of another treatment or control script. Thus, the combination of treatment or control scripts prior to and dur-ing traindur-ing resulted in four different conditions: treatment-treatment (T1-T2), treatment-control (T1-C2),
control-treatment (C1-T2), and control-control (C1-C2).
We refer to the scripts provided prior to training as pretraining scripts and those provided during train-ing as midpoint scripts (see Figures 3 and 4).
At their scheduled training session, participants were randomly assigned to receive either the treat-ment (T1) or the control (C1) pretraining script.
They were given eight minutes to read the pretrain-ing scripts and then they completed manipulation check measures on motivation to learn and computer-learning self-efficacy beliefs, described below. Next, participants received training through four modules of the e-learning system. As a pretext to provide par-ticipants with feedback, we asked them to answer five easy-to-answer multiple-choice questions on website design. The purpose was not to determine partici-pants’ learning scores, but to demonstrate that our feedback was based on their performance. Therefore, after completing four modules, participants answered these questions and received feedback from the mid-point script (treatment [T2] or control [C2], randomly
assigned). After reading this script, they continued through the next four modules. At the end of the eight modules they were asked to answer questions testing their declarative knowledge. Next they were asked to complete a hands-on performance task (see Appendix 3): to develop a website in about 20 min-utes (the time determined in the pilot study).
We collected data through 16 experimental ses-sions. Participants spent an average of about 2 hours and 20 minutes completing the assignment: the back-ground information survey took about 20 minutes, the learning session with the midpoint feedback took
about 90 minutes, and the final testing through com-prehension and development of a website took about 30 minutes.
Interventions and Measures
Background Information Survey. We created a survey that gathered background information (in-cluding demographics such as age, gender, and com-puter experience) and measured individual learner traits. Measures of learner traits included learning orientation, ( = 085), computer-learning self-efficacy measure ( = 092), both provided by Zweig and Webster (2004), and a self-efficacy for SRL mea-sure ( = 087) from Zimmerman et al. (1992) (see Appendix 2).
Pretraining Scripts. The pretraining treatment script was based on the self-regulation model pro-vided by Schunk and Zimmerman (1998) (Figure 2). After pilot testing and modifications, the final pre-training treatment script (T1) provided task analysis
and learning goals for the session (see Figure 3a). We asked participants to take notes, pay attention, and stay focused. We also attempted to boost their motivational beliefs by influencing their self-efficacy beliefs and their outcome expectations from the learn-ing exercise (e.g., with statements such as “You are a very capable learner”). The pretraining control script (C1) was about the same length to control for the
amount of information presented to the participants (Webster and Martocchio 1995). It provided only gen-eral information on communication technologies (see Figure 3b).
Midpoint Scripts. One group of participants re-ceived the midpoint treatment script (T2), which eval-uated their performance and asked them to focus on learning goals, to pay attention, and to monitor their learning progress, all elements of SRL. Consistent with other research on feedback (e.g., Martocchio and Webster 1992), all participants in this group received positive feedback, regardless of their actual perfor-mance, to control for differential effects. The other group received a midpoint control script (C2) that did not provide any SRL-specific information or feedback (see Figures 4a and 4b, respectively).
Figure 3 Pretraining Scripts (a) Pretraining Treatment Script (T1)
Welcome to this session where you will learn to design a website using Microsoft FrontPage. FrontPage is a user-friendly software, and you will find that designing a website is a simple and easy task. It is extremely important to learn the skills necessary to design a website because it is a critical skill that is highly valued by employers. By attending this session, you will learn the skills to set up your own personal website and show it to prospective employers.
Your goal in this session is to learn methods and concepts needed to design a website, and then apply these skills to develop a simple website. You will learn through the computerized e-learning system at∗∗∗∗∗called∗∗∗∗. Remember that you must pay close attention and
stay focused on the material presented on the screen. There will be many explanations, and the system will guide you in learning. You will find that it is easy to follow and understand. We are very confident that you will find that learning through this e-learning system is easy, and perhaps even more structured than learning from a classroom lecture. Several business students have used it before; they found it to be useful, and have given positive comments.
Most important, by being part of the business college and taking business courses, you are among a select set of students in the university who have met the criteria to be admitted to attend business courses. It means you have a relatively higher GPA and are capable and smart. We therefore believe that you are a very capable learner and will be able to focus, pay attention, and understand the material conveyed to you. You have shown that you can set your goals for this session and apply yourself to the task of learning to design a website. We think that you will be successful in learning to use FrontPage.
The information will be conveyed to you through eight simple, short, and consecutive modules. In the first module, you will understand what is meant by a website and learn about the different components in a website. Then, you will learn what the FrontPage screen looks like: i.e., the various menu options, the toolbars, and buttons on the screen. You will also learn how the application window is divided into different areas and the names/uses of these areas. In the subsequent modules, you will understand the notion of a template, create a webpage, and add more pages to the site. You will learn procedures to enter text on a webpage, format text, apply themes, and create hyperlinks to other webpages and websites. Each module is typically focused on one or two important aspects related to designing a website. Please pay attention to the material presented in each module and understand the main ideas.
Note that it is very important for you to pay close attention and stay focused on your task. Please be patient; read and follow the instructions correctly. After every module, you should reflect and try to recall the main elements learned in the module. If necessary, visualize what you learned, recall the main operations, and think how you would execute it. For example, recall the main procedures you would follow to change the color of a page. Hence, you must reflect on what you learned and monitor your learning progress. It is critical to be very focused and understand everything that is being described. Because you are capable learners, we are confident that you will be able pay deep attention, process the information provided by the system, and monitor your learning progress.
You can pace your own learning. Hence, use your own discretion and, if you need to, go back over the material. Please feel free to take notes as you would in any class and write down whatever you need. When you see a red arrow on the screen, it implies that you should point/choose the option indicated by this arrow. Because you are quite capable, we think you can manage your learning and that you will find the instructions fairly easy to follow. At the end of this session, you will have learned how to create a simple website using FrontPage.
Enjoy and have a successful learning experience! (b) Pretraining Control Script (C1)
Welcome to this session where you will learn some fundamental issues relating to designing a webpage. The software that you will learn is Microsoft FrontPage. Today, you will learn through the computerized online learning system at the∗∗∗∗∗ called∗∗∗∗∗. In this system, a
computer e-learning system will teach and guide you in the procedures required to design a webpage. The∗∗∗is a typical technology-based
training (TBT) software that is finding increased use in academic and business circles.
The American Society for Training and Development (ASTD) defines technology-based training (TBT) as “the delivery of content via Internet, LAN or WAN (intranet or extranet), satellite broadcast, audio- or videotape, interactive TV, or CD-ROM.” TBT in turn is considered to encompass the dimensions of Computer Based-Training (CBT) and Web-Based-Training (WBT). CBT refers to the use of computers in the instruction and management of the teaching and learning process and includes both Computer Assisted Instruction (CAI) and Computer Managed Instruction (CMI). WBT refers to the delivery of educational content through a Web browser over the public Internet, a private intranet, or an extranet. Typically, they provide links to other learning resources such as references, email, bulletin boards, and discussion groups.
A related component of TBT is the concept of Distance Learning or Distance Education. ASTD defines Distance Learning as the “educa-tional situation in which the instructor and students are separated by time, location, or both.” Courses are delivered to remote locations via synchronous or asynchronous means. The former refers to real-time, instructor-led learning with direct and simultaneous communication between participants. Associated technologies will include whiteboards, audio or videoconferencing, Internet telephony, or two-way live broadcasts. The latter has a time-lag component in the interaction between the instructor and the student. The popular self-paced courses taken via the Internet or CD-ROM, online discussion groups and email are examples of the asynchronous mode.
The use of technology in training spans three decades and can broadly be classified into three distinct phases. The initial phase was in the eighties where the primary driver was multimedia technology that provided rich audio, video, graphical, and animated content. The second phase in the nineties saw the development of the Internet as the primary vehicle for the electronic dissemination of training and learning. The development of efficient Web browsers, hyper-text markup languages, and media players propelled the Internet as a viable training medium. The third phase commenced at the turn of the century with the explosive increase in bandwidth access and availability.
Figure 3 (Cont’d.)
This, coupled with advances in Web-design technologies and high quality streaming media, enabled the development of real-time, low-cost, highly effective online training and learning environments.
This∗∗∗∗is a typical∗∗∗software and we expect that at the end of this session, you will have understood some of the elements that go into
creating a webpage using FrontPage. Please be patient, read the instructions, and follow them correctly. You can pace your own learning process. Hence, use your own discretion and go backwards if you need to. Please feel free to take notes as you would in any class and write down whatever you need to. When you see a red arrow on the screen, it implies that you should point/choose the option indicated by this arrow.
Dependent Variables. Learning outcomes on IT software training are measured with a declarative knowledge test (i.e., a written test to evaluate learn-ers’ conceptual understanding), and a hands-on task-performance test that evaluates their procedural knowledge and ability to use the software (Yi and Davis 2003). In this study, the declarative knowl-edge test included multiple choice and fill-in-the-blank questions that were pretested in the pilot study. The hands-on performance task required participants to develop a website. These tests were graded by a graduate student who was blind to the conditions and trained on this task (see Appendix 3).
Manipulation Checks on SRL. Researchers have noted both the lack of standard instruments to assess whether participants follow SRL strategies and the problems with developing such instruments (e.g., Winnie and Perry 2000). In developing our
Figure 4 Midpoint Scripts (a) Midpoint Treatment Script (T2)
Dear Student,
Excellent effort and progress!!! You are making good progress toward the goal of learning to develop a website using FrontPage. This midpoint evaluation shows that you have understood the fundamental concepts of what constitutes a website, what is meant by a hyperlink, and how to present information on a webpage. Your scores are excellent compared with other students who have learned through this e-learning system. Because you are learning on your own without any human instructor, this evaluation shows that you are quite capable of paying attention to the material, understanding, learning, and monitoring your learning progress. Please continue to do so, and focus on this learning task. In the next few modules, you will continue to learn other aspects of designing a website such as learning to create internal and external links from your page and to apply themes to your website. Please continue with the good progress you have made so far, and remember to reflect on the content learned in each module. Continue to pay attention and stay focused on the task and you will achieve the goal of learning how to design a website. GOOD JOB!!
(b) Midpoint Control Script (C2)
Dear Student,
Web-based training (WBT) is an innovative approach to distance learning in which computer-based training (CBT) is transformed by the technologies and methodologies of the World Wide Web, the Internet, and intranets. Web-based training presents live content, as fresh as the moment and modified at will, in a structure allowing self-directed, self-paced instruction in any topic. WBT is media-rich training fully capable of evaluation, adaptation, and remediation, all independent of computer platform. Web-based training is an ideal vehicle for delivering training to individuals anywhere in the world at any time. Web browsers that support 3-D virtual reality, animation, interactions, chat, and conferencing, and real-time audio and video will offer many training opportunities.
Instructional designers and training analysts are learning more about how to write and produce WBT. Therefore, in the future you will see a variety of WBT course offerings that will be distributed over the public Internet and private intranets. In the next few modules, you will learn more about webpage design.
instruments, we relied on Zimmerman’s (1994) iden-tification of four dimensions to SRL: motives, meth-ods, performance outcomes, and social-environmental resources. The motives dimension deals with learn-ers’ motivations to learn and complete the course. The methods dimension encompasses the application of SRL strategies. The performance outcomes dimen-sion deals with the monitoring of learning goals and closely corresponds to metacognitive activities. The social-environmental resources dimension deals with learners’ access to peers and teachers who can pro-vide guidance (this dimension was not applicable for our training context).
We conducted two types of manipulation checks, one with our participants and the other with a sepa-rate group. We did not want to ask our participants directly about their self-regulatory learning strate-gies (such as their methods for learning or moni-toring their learning goals) because doing so could
cue them to use these methods. Consequently, we measured only the motives dimension for our main study participants. Specifically, we twice measured their states of computer-learning self-efficacy (Zweig and Webster 2004) ( = 092) and motivation to learn (Hicks and Klimoski 1987) ( = 088): once at the beginning of training after the participants had read the pretraining scripts, and once during the training session (see Appendix 4A). We expected that these measures would indirectly indicate self-regulatory learning strategies because self-efficacy beliefs and motivation are described as being important compo-nents in the application of SRL strategies as per the social cognitive perspective. It is to be noted that these manipulation checks are state measures of computer learning self-efficacy and motivation to learn and may have quite different effects on learning performance than the corresponding trait measures that are con-sidered to be relatively stable dispositional variables (George 1991).
The second manipulation check was conducted on a separate group of participants. These were measured at the midpoint, before participants received the mid-point scripts. The results from this group were not included in the main study and, thus, we could assess the methods and performance dimensions without concern for cueing their behaviors. That is, these experimental sessions were conducted solely to per-form manipulation checks on whether participants who received the treatment script (T1) engaged in
rel-atively more SRL methods and performance strate-gies than those who received the pretraining control script (C1). For this separate group of participants, seven experimental sessions similar to the main study were conducted. After participants completed four modules, they were given the manipulation test ques-tionnaire (see Appendix 4).
Consistent with Schunk and Ertmer (1999), we developed this questionnaire based on Zimmerman’s work. Specifically, we chose methods found to be highly significant in SRL (Zimmerman and Martinez-Pons 1986, 1988): organizing and transforming (stu-dent-initiated overt or covert rearrangements of instructional material to improve learning), and rehearsing and memorizing (student-initiated efforts to memorize material by overt or covert practice). Based on these descriptions, we adapted three items
for each of these methods from Kanfer et al. (1994) and Pintrich and DeGroot (1990). To assess the perfor-mance outcomes dimension (student-initiated efforts to set specific goals and monitor progress), three items were adapted from Schunk and Ertmer (2000) and Schmidt and Ford (2003) (see Appendix 4B).
Results
Of the 134 participants who volunteered for the exper-iment, 10 were eliminated because of their prior Web-development experience and six because of technical problems (their computers froze or their audio did not work). For the remaining 118 participants, the average age was 21.6, the average GPA was 3.3, and they were almost equally divided between males and females (see Table 1a). All had experience with pop-ular software packages like Word and PowerPoint, but none had experience with Web-development soft-ware. As shown in Tables 1a and 1b, we found no significant demographic differences among the train-ing groups. A factor analysis and validity checks of individual learner traits showed a high correla-tion between learning orientacorrela-tion and self-efficacy for self-regulation, and a lack of discriminant valid-ity. Therefore, it was decided to drop the learning orientation measure and conduct the factor
analy-Table 1a Demographic Information Demographic measures T1-T2 T1-C2 C1-T2 C1-C2 Overall Mean age (SD) 215 (1.7) 218 (2.0) 218 (1.7) 214 (1.5) 216 (1.7) Mean GPA (SD) 33 (0.4) 33 (0.4) 32 (0.4) 33 (0.5) 33 (0.4) Number of males 14 15 13 15 57 Number of females 22 10 12 17 61
Note. Treatment-treatment (T1-T2), treatment-control (T1-C2),
control-treatment (C1-T2), control-control (C1-C2).
Table 1b Tests of Demographic Differences Between Conditions Experimental conditions Demographic measures T1-T2vs. C1-C2 T1-T2vs. T1-C2 T1-T2vs. C1-T2 p values of differences in means test Age 040 047 046 GPA 047 045 023
Note. Treatment-treatment (T1-T2), treatment-control (T1-C2),
Table 1c Factor Loadings
Items Factor loadings
SR7 0888 0133 0091 −0003 SR6 0871 0122 0139 0015 SR10 0859 0265 −0033 −0129 SR9 0820 0015 0160 0099 SR3 0774 0245 0211 −0154 SR2 0741 0090 0101 −0039 SR4 0599 0216 0244 0221 CSE6 0166 0875 0136 0041 CSE7 0227 0874 0114 0035 CSE5 0086 0870 −0058 −0238 CSE3 0295 0752 0203 0164 CSE2 0087 0659 0372 0258 SR5 0245 0459 0441 0303 SR1 0239 −0068 0760 −0152 CSE1 0055 0496 0720 −0004 SR11 0448 0271 0519 0013 CSE4 0165 0467 −0081 0646 SR8 0466 0281 0096 −0622
Notes. Trait variables: CSE, computer-learning self-efficacy; SR, self-efficacy for self regulated learning. Those items in bold indicate items used for sub-sequent analysis.
Table 1d Means, Standard Deviations, and Intercorrelations Cronbach’s
Mean SD alpha CSE SR HO DC
CSE 555 098 090 081
SR 544 108 092 0394∗∗ (0.80)
HO 2941 441 NA 0056 0022 NA
DC 644 262 NA −0016 0044 0446∗∗ NA
Notes. Trait variables: CSE, computer-learning self-efficacy; SR, self-efficacy for self regulated learning. Dependent variables: HO, hands-on performance; DC, declarative knowledge; NA, not applicable. Diagonal values enclosed in parentheses indicate the square root of the AVE.
∗Correlation is significant at the 0.05 level (two-tailed). ∗∗Correlation is significant at the 0.01 level (two-tailed).
sis again. In Table 1c, the factor analysis of individ-ual learner traits of computer-learning self-efficacy and self-efficacy for self-regulation are provided. It is seen that seven of the 10 items in the measure of self efficacy for self-regulation loaded together whereas five of the seven items in the computer-learning self-efficacy scale loaded together and they had acceptable factor loadings. These seven and five items, respectively, were used in our final analy-sis. As seen in Table 1d, the measures of individual learner traits of computer-learning self-efficacy and self-efficacy for self-regulation satisfy tests of conver-gent and discriminant validity, with the square root of the average variance extracted (AVE) of each
con-Table 2a Manipulation Check: Motives
Experimental conditions
Mean score (SD) Differences Manipulation check Cronbach’s Treatment Control in means measures alpha n = 61 (n = 57) (t-tests) After pretraining script
Motivation to learn 078 6.37 (0.57) 6.11 (0.81) p < 003 Computer learning 076 6.09 (0.59) 5.93 (0.77) p < 011
self-efficacy After midpoint script
Motivation to learn 084 6.24 (0.66) 5.96 (0.90) p < 003 Computer learning 085 6.06 (0.72) 5.90 (0.79) p < 013
self-efficacy
Table 2b Manipulation Check: Methods
Experimental conditions
Mean score (SD) Differences Manipulation check Cronbach’s Treatment Control in means measures alpha n = 16 (n = 17) (t-tests) Organizing and 070 5.85 (0.78) 5.35 (0.83) p < 004
transforming
Rehearsing and 071 5.48 (1.00) 4.84 (1.19) p < 005 memorizing
Note. Checks were conducted with a separate group of participants. Table 2c Manipulation Check: Performance Outcomes
Experimental conditions
Mean score (SD) Differences Manipulation check Cronbach’s Treatment Control in means measures alpha n = 16 (n = 17) (t-tests) Performance outcomes 083 5.94 (0.79) 5.33 (0.84) p < 003 Note. Checks were conducted with a separate group of participants.
struct being larger than the correlation between the constructs. The Cronbach alpha tests of reliability are also respectable.
The results of our manipulation checks are shown in Tables 2a–2c. As shown in Table 2a, participants who received the pretraining treatment scripts scored higher on a state measure of computer-learning self-efficacy beliefs and motivation to learn, suggesting that our pretraining treatment scripts affected the motive dimension of SRL. Tables 2b and 2c illustrate that those participants who received the pretrain-ing treatment script reported followpretrain-ing a significantly greater amount of SRL strategies (both methods and performance outcomes) compared with those who
received the pretraining control script. Note (as per theory) that all learners indicated following some level of SRL; however, it is the quality and extent of SRL that is important (and the participants in the treatment group indicated that they applied a rela-tively higher level of SRL).
When we conducted tests of hypotheses, we found that the dependent measures of declarative knowl-edge and hands-on task performance were correlated (r = 045, p < 00001). We first conducted an overall MANCOVA (in which the individual learner charac-teristics were entered as covariates). With a significant MANCOVA (Wilks’ Lambda = 0887, p < 005), we then conducted a Dunnet’s t-test, a recommended test for simultaneous comparison of treatment condition scores (see Table 3). The mean dependent measure scores and standard deviations for each of the four conditions are shown in Table 4 and graphed in Fig-ure 5. The scores for the treatment-treatment (T1-T2)
condition are the highest, followed by treatment-control (T1-C2), control-treatment (C1-T2), and
control-control (C1-C2) conditions.
For Hypotheses 1A and 1B concerning the train-ing conditions, the T1-T2 condition differed
signifi-cantly from the C1-C2condition. As shown in Tables 4 and 5a, the pretraining treatment script was effective in improving the learning outcomes. Hence, Hypoth-esis 1A was supported, suggesting that presession interventions would enhance learning outcomes. Fur-thermore, the midpoint scripts resulted in higher de-pendent measure scores for the treatment groups, sup-porting Hypothesis 1B as shown in Table 5b. It can be seen from Tables 5a and 5b that the differences in scores between the treatment and control groups are higher on the declarative knowledge test than the hands-on performance test, but both are significantly different.
Table 3 Differences in Learning Outcomes for Experimental Conditions
Experimental conditions T1-T2vs. T1-T2vs. T1-T2vs.
Learning outcomes C1-C2 T1-C2 C1-T2
Difference in means (p values)
Declarative knowledge 0002 0071 0006
Hands-on performance 0040 0395 0254
We dropped the learning-orientation measure be-cause of validity issues in relation to the other indi-vidual traits, so we could not test Hypothesis 2A. To test Hypotheses 2B and 2C concerning the effects of individual learner traits of computer learning self-efficacy and self-self-efficacy for self-regulation on learn-ing outcomes, we examined the covariates in the MANCOVA and in individual ANCOVAs. These indi-vidual learner traits were not significantly related to the learning outcomes.1 Thus, we did not find
sup-port for Hypotheses 2B and 2C concerning the effects of individual learner traits on learning outcomes.
Discussion
Based on Alavi and Leidner’s (2001) framework, we examined the interaction of features of informa-tion technology, psychological learning processes, and instructional strategies embedded in an e-learning-based IT training environment. Because learners may not be exercising the high levels of self-directed learn-ing strategies required in this trainlearn-ing set-up, we developed instructional strategies that included inter-ventions to induce learners to apply higher levels of self-directed learning strategies. Our tests showed that when the instructional strategy included such interventions that taught learners to self-regulate, learners applied more self-regulatory learning strate-gies, leading to enhanced learning outcomes. We also found that during the training session, providing positive feedback to learners on their use of SRL reinforces the use of SRL strategies, suggesting that learners must be reminded and motivated if they are to continue using SRL strategies. Results show that
1We also conducted a hierarchical regression analysis using the
T1-T2 and C1-C2 conditions. On each of the dependent
vari-ables (declarative knowledge and hands-on performance) we first regressed the treatment conditions, then the individual learner trait measures, and then the interaction terms between individ-ual learner trait measures and treatment conditions. On declara-tive knowledge, the addition of the individual learner traits did not account for significant additional variance (R2change = 004,
p < 085), nor did the addition of the interaction terms (R2change =
005, p < 019). Similarly, with hands-on performance, the addition of individual learner traits did not account for significant additional variance (R2 change = 001, p < 077), nor did the addition of the
interaction terms (R2change = 001, p < 080). Thus, the results do
not indicate interactions between the individual learner traits and training interventions.
Table 4 Learning Outcomes by Condition
Experimental conditions
Treatment-treatment Treatment-control Control-treatment Control-control Learning outcomes (T1-T2) n = 36 (T1-C2) n = 25 (C1-T2) n = 25 (C1-C2) n = 32
Mean outcome score (SD)
Declarative knowledge (Max = 15) 7.67 (2.50) 6.40b(2.18) 5.76a(2.31) 5.63a(2.87)
Hands-on performance (Max = 35) 30.56 (4.14) 29.56 (3.64) 29.16c(4.23) 28.12b(5.36)
Note. Means with superscripts a, b, and c differ from the corresponding treatment-treatment (T1-T2) condition at significance levels
of 0.01, 0.05, and 0.1 respectively.
the group instructed to follow SRL before and dur-ing traindur-ing achieved the highest learndur-ing outcomes, whereas the group that received no instructions to self-regulate learning either before or during train-ing received the lowest learntrain-ing scores. Our results showed that all learners apply some level of SRL, but e-learning necessitates a higher level of SRL. Interven-tions like those examined in this study could induce learners to apply the higher levels of SRL needed in e-learning-based training contexts, thereby leading to higher learning outcomes.
Based on past IS training research, we hypothe-sized that individual learner traits relating to SRL may influence learning outcomes. Our results did not support this premise. We found that the learning
ori-Figure 5 Graphs of Learning Outcomes
Hands-on performance 26.5 27.0 27.5 28.0 28.5 29.0 29.5 30.0 30.5 31.0 T1- T2 T1- C2 C1- T2 C1- C2 T1- T2 T1- C2 C1- T2 C1- C2 Declarative knowledge 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0
Note. T1-T2, Treatment-treatment; T1-C2, treatment-control; C1-T2,
control-treatment; C1-C2, control-control.
entation and self-efficacy for self-regulation measures were correlated, perhaps because each of them taps into different forms of regulation. Although learning orientation taps into regulation relating to achieve-ment motivation, and self-efficacy for self-regulation taps into academic motivation, they did not emerge as distinct measures and could not be tested simulta-neously. We tested self-efficacy for self-regulation and computer-learning efficacy; these did not have direct or interactive effects on learning performance. One explanation can be found in the theory itself, which states that SRL is very situation specific and has to be activated in any given context, pointing to the importance of interventions. Learners who had higher self-efficacy beliefs for self-regulation and computer learning may not all have activated their self-efficacy beliefs in the new e-learning context. Another expla-nation for the lack of significant effects for individual learner traits can be found in the debates over the
Table 5a Learning Outcomes for Pretraining Conditions Experimental conditions
Differences Outcome measures for Treatment Control in means pretraining conditions n = 61 n = 57 (t-tests) Mean outcome score (SD)
Declarative knowledge 715 (2.44) 568 (2.62) p < 0001 Hands-on performance 3014 (3.94) 2861 (4.87) p < 003 Table 5b Learning Outcomes for Midpoint Conditions
Experimental conditions
Differences Outcome measures for Treatment Control in means midpoint conditions n = 61 n = 57 (t-tests) Mean outcome score (SD)
Declarative knowledge 689 (2.59) 596 (2.60) p < 003 Hands-on performance 2998 (4.21) 2879 (4.69) p < 007
role of individual traits and their effects on learning performance. Researchers have argued that there is a marked difference in the effects of the distal trait vari-ables when compared to the proximal state varivari-ables of the same construct (George 1991). Our findings, along with other prior findings, signal conflicting find-ings on the relationship between trait-like individual differences and learning performance, suggesting that we must investigate more closely the influence of trait-like measures of individual differences relating to self-regulation and learning performance.
Our findings that individual traits relating to SRL did not have an influence on learning outcomes, but that interventions did, point to the value of encourag-ing learners to follow SRL strategies when takencourag-ing part in e-learning training. Results show that, unlike other IS training environments where individual learner traits play a relatively important role, in e-learning-based training environments, instructional strategies may play a more critical role than pre-existing differ-ences in individual learner traits.
Though the pretraining scripts were fairly straight-forward and simple to implement, we found strong learning effects, consistent with other research manip-ulating pretraining interventions (e.g., Webster and Martocchio 1995). According to the social cognitive perspective, people engage in some degree of SRL, but what truly matters is the quality and extent to which they apply these strategies in a specific con-text. Messages such as those provided in the pre-training scripts seem to persuade the learner to apply SRL strategies in the given training situation. From an organizational standpoint, once an e-learning tech-nology platform has been purchased and an IT course chosen as the target content, interventions in the instructional strategy such as those tested in this study may be the most suitable means to increase learning effectiveness. But it is also worthwhile to research whether persuasive messages such as those used in this study could be incorporated as part of the e-learning technology infrastructures so that learners can be encouraged to apply SRL strategies when they undergo e-learning courses.
Prior IS research has concluded that the use of TML-based methods for IT training may require the identi-fication of special skill sets to achieve success (Piccoli et al. 2001). In this study, we identify and demonstrate
that SRL is one such learner skill. This study advances the body of IT research on social cognitive theory (Compeau and Higgins 1995a, Johnson and Marakas 2000, Yi and Davis 2003) to encompass SRL. Therefore, our study helps contribute to a cumulative theoreti-cal foundation for IT training both from a TML and a social cognitive research perspective.
Implications for Research and Practice
This study introduces the concept of self-regulation to IS research and shows that it is relevant to train-ing programs. Our findtrain-ings are based on a strong experimental design, with data collected over four time periods and training conditions crossed at the midpoint. As with any experimental study, however, this research has several limitations. We used self-reports on the use of SRL as a manipulation check rather than interviews, observation, or verbal proto-cols, because self-reports provide a method by which to compare the use of SRL among the groups. Fur-thermore, although research suggests that training outcomes should be evaluated using several dimen-sions, we focused only on learning outcomes due to our primary interest in improving the effectiveness of e-learning. Consequently, this research should be extended to other outcome variables.
Another limitation is the use of student partici-pants. Although we used college students as partic-ipants, our findings can be considered to be fairly realistic: the students were learning the topic for the first time and website design is an important skill for this group. This research, however, should also be extended to employees in the workplace. SRL may be more relevant to employee training because of its capacity to counteract work-related distrac-tions; however, our study does not address desktop-based training, only training conducted in controlled environments. Reports indicate that many organiza-tions, including the American federal government, use e-learning as the primary method to deliver tech-nology training to employees (e.g., Hasson 2005) and research should be extended to this setting.
We applied paper-based interventions to influence self-regulatory strategies, but we believe that these interventions could be designed as an integral part of e-learning platforms. There are several major play-ers in the corporate IT e-learning market who offer
