Differential effects of problem-solving demands on individual
and collaborative learning outcomes
Femke Kirschner
a,b,*
, Fred Paas
a,b, Paul A. Kirschner
a, Jeroen Janssen
c aCentre for Learning Sciences and Technologies, Open University of The Netherlands, P.O. Box 2960, 6401 DL Heerlen, The Netherlands b
Institute of Psychology, Erasmus University Rotterdam, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands cResearch Centre Learning in Interaction, Utrecht University, P.O. Box 80140, 3508 TC Utrecht, The Netherlands
Received 19 January 2010; revised 9 January 2011; accepted 12 January 2011
Abstract
The effectiveness and efficiency of individual versus collaborative learning was investigated as a function of instructional format among 140 high school students in the domain of biology. The instructional format either emphasized worked examples, which needed to be studied or the equivalent problems, which needed to be solved. Because problem solving imposes a higher cognitive load for novices than does studying worked examples it was hypothesized that learning by solving problems would lead to better learning outcomes (effectiveness) and be more efficient for collaborative learners, whereas learning by studying worked examples would lead to better learning outcomes and be more efficient for individual learners. The results supported these crossover interaction hypothesis. Consequences of the findings for the design of individual and collaborative learning environments are discussed.
Ó2011 Elsevier Ltd. All rights reserved.
Keywords:Cognitive load theory; Worked-example study; Problem-solving; Collaborative learning; Learning efficiency; Learning outcomes
1. Introduction
Cognitive load theory (CLT:Kirschner, 2002; Paas, Renkl, & Sweller, 2003; Sweller, 1988; Sweller, Van Merrie¨nboer, & Paas, 1998) focuses on learning from complex cognitive tasks based on what is known about human cognitive architecture (Sweller, 1988, 2004). This architecture consists of an unlimited long-term memory (LTM), which interacts with a working memory (WM) that is limited in both capacity (Miller, 1956) and duration (Peterson & Peterson, 1959). For new information, the pro-cessing capacity is limited to 4 1 interacting information
elements which is lost if not rehearsed within 30 s (Cowan, 2001). LTM stores and organises knowledge in cognitive schemas that incorporate multiple elements of information into a single element (also referred to as chunking;Chase & Simon, 1973; Miller, 1956; Simon, 1974) with a specific function (i.e., learning). If learning has occurred over a long period of time, one’s schemas may consist of huge amounts of information. Because a schema can be treated by WM as a single element or even bypass WM if it has become sufficiently automated through long and consistent practice, WM limitations will disappear for more knowledgeable learners.
According to CLT, learning task complexity is determined by the number of new (i.e., to be learned) interacting infor-mation elements; the more new interacting elements, the more complex the task. Although highly interactive information elements can be processed in isolation, they can only be understood when all of them and their interactions are pro-cessed simultaneously. Given that WM capacity is limited * Corresponding author. Institute of Psychology, Erasmus University
Rot-terdam, P.O. Box 1738, 3000 DR RotRot-terdam, The Netherlands. Tel.:þ31 10 4088845; fax:þ31 10 408 9009.
E-mail addresses: [email protected] (F. Kirschner),[email protected]
(F. Paas), [email protected] (P.A. Kirschner), [email protected]
(J. Janssen).
www.elsevier.com/locate/learninstruc
0959-4752/$ - see front matterÓ2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.learninstruc.2011.01.001
with respect to new information, tasks that contain many new and interacting elements place high cognitive demands on WM. Within CLT, the load imposed on WM by the element interactivity or task complexity is called intrinsic cognitive load.
As basis for instructional design, CLT focuses on effec-tively dealing with individual WM limitations by creating instruction that is compatible with human cognitive architec-ture. Research, therefore, primarily has been on developing techniques for managing individual WM load and optimising information-processing in individuallearning settings (Ayres & Paas, 2009).Kirschner, Paas, and Kirschner (2009b, 2010)
have recently emphasised an alternative way of effectively dealing with individual WM limitations, namely making use of the multiple WMs of individuals in acollaborativelearning setting. From their perspective, groups of collaborating learners are considered to be information-processing systems (Hinsz, Tindale, & Vollrath, 1997; Ickes & Gonzalez, 1994; Tindale & Kameda, 2000), consisting of multiple limited WMs which can create a collective working space. Within these systems, valuable task-relevant information and knowl-edge held by each group member can be consciously and actively shared (i.e., retrieving and explicating information), discussed (i.e., encoding and elaborating information), and remembered (i.e., personalising and storing information) (Hinsz et al., 1997; Tindale & Kameda, 2000; Tindale & Sheffey, 2002). As long as the information is communicated between the group members and they coordinate their actions, not all group members needpossessall necessary knowledge, orprocessall available information aloneandat the same time (Johnson, Johnson, & Stanne, 2001; Langfred, 2000; Wegner, 1978, 1995). This, however, requires positive interdependence as described inJohnson and Johnson’s (1981)social interde-pendence theory. Positive interdeinterde-pendence reflects the extent to which group members must depend upon each other for effective group performance; each individual group member is responsible for the work of the group and the group as a whole is responsible for the learning of each individual group member. Group members are linked to each other such that each member cannot succeed unless the others succeed; each member’s work benefits the others and each member benefits from the others. Essential here is social cohesion and a heightened sense of ‘belonging’ to a group.
The perspective on shared memory systems in which communication plays a key role has also been investigated in the context of Wegner’s (1978) transactive memory theory. This theory is based on the idea that group members can serve as external memory aids for each other, and that they can benefit from each other’s knowledge and expertise if they develop a good, shared understanding and an awareness of ‘who knows what’ in the group. This could, for example, be achieved by training individuals on how to share their knowledge effectively (Deiglmayr & Spada, 2010; Prichard, Stratford, & Bizo, 2006). Findings indicate that transactive memory can facilitate group performance in groups whose members are aware of other group members’ knowledge and expertise as opposed to groups where the members are not
aware of fellow group members’ knowledge and expertise (Michinov & Michinov, 2009). A transactive memory system enables groups to better utilise the knowledge that their members possess, and to reach higher levels of performance than they would have reached without such a system (for a review, seeMoreland & Argote, 2003).
The CLT perspective, in which groups are considered to possess a shared memory system, has two conflicting conse-quences for individual group members. On the one hand, collaborating individuals can invest less cognitive effort compared to learners working alone, because the task’s inter-active information elements with its associated cognitive load (i.e., the intrinsic cognitive load) can be divided across a larger reservoir of cognitive capacity (Kirschner, Paas, & Kirschner, 2009a; Ohtsubo, 2005; Stasser, Stewart, & Wittenbaum, 1995). This is what is called thedistribution advantage. On the other hand, collaborating individuals need to invest cognitive effort in communicating information with each other and coordinating their actions, which individuals working alone do not have to exert. These, so called,transactional activities(Ciborra & Olson, 1988; Kirschner et al, 2009b; Yamane, 1996) can be beneficial for or deleterious to learning. CLT argues that, while a cognitive investment in beneficial transactional activities such as negoti-ating common ground should be stimulated, an investment in deleterious activities such as discussing ways to share informa-tion should be minimised.
The trade-off between the advantage of dividing informa-tion-processing among group members and the disadvantage in terms of having to cognitively invest in the associated transactional activities can be an indicator for the efficiency of group learning. This so called collective working memory effectwas demonstrated in a study byKirschner et al. (2010)
on the effects of low-complexity (i.e., low intrinsic load) and high-complexity (i.e., high intrinsic load) tasks on individual and group learning efficiency. Other than learning effective-ness, which is related primarily to learning outcomes (i.e., posttest performance), learning efficiency is related to the relationship between learning outcomes and the amount of mental effort learners invest to attain those outcomes; the higher the learning outcomes and the lower the effort, the higher the efficiency (Paas & Van Merrie¨nboer, 1993; Van Gog & Paas, 2008; Tuovinen & Paas, 2004). Mental effort in combination with learning outcome can indicate the quality of learning in terms of the efficiency of cognitive schema acquisition. By giving groups and individuals low and high-complexity learning tasks and then assessing their learning efficiency on an individual posttest, Kirschner et al (2010)
showed that group learning was superior to individual learning for high-complexity tasks, but inferior for low-complexity tasks.
For high-complexity tasks, Kirschner et al. (2010)argued that by sharing the task’s high intrinsic cognitive load among learners, the risk of exceeding the limits of the individual group members’ WMs was reduced (i.e., distribution advan-tage). Although the additional cognitive load imposed by communicating information and coordinating actions (i.e., transactional activities) has to be taken into account, this load
could be considered to be relatively low compared to the distribution advantage for complex tasks. Consequently, learning was more efficient for group members, allowing them to construct higher quality schemas in LTM than for individual learners who had to process all of the information individually (i.e., the collective working-memory effect: Kirschner et al., 2010).
For low-complexity tasks, they argued that individual learners had sufficient capacity to carry out the tasks alone and no advantage of learning together was expected. Sharing the information-processing among the group members required information communication and action coordination which for low-complexity tasks imposed a relatively high load (in relation to the benefits that will be accrued) thereby negating the distribution advantage. Consequently, learning was more efficient for individuals, allowing them to construct higher quality schemas in LTM than for group learners who had to engage in relatively high transaction activities.
In addition to the intrinsic load imposed by the complexity of the learning task, the cognitive load that learners experience is affected by a task’s instructional format. This can take two forms. When the instructional format imposes load that is not effective for learning, for example, when it provides redundant information (e.g., Chandler & Sweller, 1991), or requires students to mentally integrate spatially or temporally separated materials (e.g.,Sweller, Chandler, Tierney, & Cooper, 1990), it is called extraneous cognitive load. When it is beneficial to learning, for example when it presents students with high-variability materials (Paas & Van Merrie¨nboer, 1994), or encourages students to self-explain their actions (e.g.,Renkl, 1997), it is referred to as germane cognitive load (Paas, Renkl, & Sweller, 2004; Sweller, Van Merrie¨nboer, & Paas, 1998). Although extraneous load does not hamper learning for low in intrinsic load tasks (i.e., low-complexity), it does hamper learning for high intrinsic load tasks (i.e., high-complexity); hence, reducing extraneous load is imperative for high-complexity tasks (Van Merrie¨nboer & Sweller, 2005). A consistent finding of CLT research is that for tasks high in intrinsic load, instruction with high problem-solving demands imposes extraneous load on novice learners (Sweller et al., 1998). In addition to Kirschner et al.’s (2009a, 2010)
recent studies into the effects of different levels of intrinsic load on individual and collaborative learning efficiency, the present study kept intrinsic load constant and investigated the differential effects of extraneous load induced by the instructional format in terms of problem-solving demands. Specifically, individual and collaborative learning e by novices e from either tasks that heavily rely on problem-solving or tasks that heavily rely on example study were compared regarding their effects on learning efficiency and effectiveness.
Sweller (1988) has shown that problem-solving search when carrying out complex conventional tasks, places not only heavy intrinsic demands, but also heavy extraneous demands on WM. The strategy most commonly used by learners faced with novel problems for which they do not have
previously constructed schemas e a means-ends analysis e requires them to consider the current problem state, consider the desired goal state, extract differences between the two states, and find or choose a problem-solving operator that can be used to reduce or eliminate differences between the current problem state and the desired goal state. In addition, any sub-goals that have been established need to simultaneously be kept in mind. This problem-solving search strategy imposes high extraneous cognitive load on learners and, consequently, does not leave sufficient processing capacity for them to induce the generalised solutions or schemas that are prereq-uisite to learning.
An effective alternative to such instruction, and one that has been supported by multiple overlapping experiments using different instructional materials in a variety of populations, is instruction with low problem-solving demands relying on example study, either by using exampleeproblem pairs, prob-lemeexample pairs, or example study only (e.g.,Carroll, 1994; Hu¨bner, Nu¨ckles, & Renkl, 2010; Paas, 1992; Paas & Van Gog, 2006; Paas & Van Merrie¨nboer, 1994; Stark, Kopp, & Fischer, 2010; Sweller, 1988; Sweller, 2006; Sweller & Cooper, 1985; Trafton & Reiser, 1993; Van Gog, Kester, & Paas, 2010; for an overview see Atkinson, Derry, Renkl, & Wortham, 2000). Instruction that relies heavily on studying worked examples as a substitute for instruction involving problem-solving is argued to be beneficial because it decreases extraneous load by elimi-nating means-ends search. In contrast to instruction with a heavy reliance on problem-solving, worked examples focus attention on problem states and associated operators (i.e., solution steps), enabling learners to induce generalised solutions or schemas. The cognitive capacity that becomes available by reducing extraneous load can be devoted to activities beneficial to learning (i.e., germane load). This reasoning has led to the counterintuitive instructional guideline that for novices learning to solve problems, studying worked examples is a better strategy than solving the equivalent problems (Rourke & Sweller, 2009; Sweller, 1988; Sweller et al., 1998).
2. Research Question and Hypotheses
In the context of CLT, instruction with low problem-solving demands (i.e., studying worked examples) should impose lower cognitive load on novices than instruction with high problem-solving demands (i.e., equivalent problems that need to be solved). Combining this view on instructional formats with the results of the collective working memory effect (Kirschner et al., 2010), the question arises as to whether for students learning individually, instruction emphasising worked example study would be more effective and efficient than instruction emphasising solving problems. Analogous to this is the question of whether for students learning collaboratively, instruction emphasising problem-solving would be more effective and efficient than instruction emphasising worked example study. To this end, this study tested the following hypotheses:
For learning from instruction emphasising problem-solving, the load on the limited cognitive capacity of an
individual learner should be too high for effective learning. For learners in a group, the benefits of dividing information-processing (i.e., the cognitive load) among group members would be greater than the additional cognitive investment of inter-individual communication of information and coordina-tion of accoordina-tions in the learning phase. Group members would consequently be able to devote the freed-up cognitive capacity to activities that foster schema construction and automation (i.e., germane load), resulting in higher performance (i.e., learning outcome; Hypothesis 1a) and a more favourable relationship between performance and mental effort (i.e., higher learning efficiency; Hypothesis 1b) for learners who carried out the learning tasks in groups than for those who carried out the tasks individually.
With regard to learning from instruction emphasising worked examples, learners working individually or as members of a group should have sufficient cognitive capacity to process all information themselves. Hence, the transactional cognitive activities for learners working as members of a group would be high relative to the benefits of dividing the information-processing across group members in the learning phase. Consequently, qualitative differences in constructed schemas were expected between learners learning in a group and learners learning individually, resulting in higher learning outcomes (Hypothesis 2a) as well as higher learning efficiency (Hypothesis 2b) on an individual posttest for those who learned individually than for those who learned as members of a group.
3. Method
3.1. Participants and design
The effect of instructional format on the learning outcome and efficiency of individual versus collaborative learning was investigated using learning tasks emphasizing either problem-solving (further referred to as problem-problem-solving tasks) or worked-example study (further referred to as worked-example study tasks). In the learning phase, 140 Dutch high school sophomore students (71 boys, 69 girls) with an average age of 14.98 years (SD ¼0.96), were assigned to four conditions in such a way that 34 participants had to learn individually from problem-solving tasks, 34 had to learn individually from worked-example study tasks, 36 had to learn in 3-person groups (i.e., triads) from problem-solving tasks, and 36 had to learn in 3-person groups from worked-example study tasks. They participated in the study as part of their regular biology curriculum without any academic or financial compensation. They were told that the topic of heredity was part of their annual exam later that year. No differences in prior knowledge were expected since all participants had followed the same biology curriculum in the previous three years and the study topic was new. To further assure equivalence of conditions in all ways except for the treatment, participants were randomly assigned to the different experimental conditions. To deter-mine how much was learned, participants were required to individually take a posttest.
3.2. Materials
The materials were in the biology domain and were con-cerned with heredity; specifically genotypic and phenotypic transmission of biological traits from parents to offspring (e.g., eye colour in humans, fur length in dogs, leaf shape in plants). To this end, a general introduction and instruction on solving inheritance problems, worked example study tasks, problem-solving tasks, and transfer-test tasks were designed. All materials were approved by two biology teachers as being suitable for the learners. All materials were paper based. 3.2.1. Introduction
Relevant terminology, rules, and theory underlying heredity, as well as a worked example on solving heredity problems was discussed in the introduction. This introduction gave the definition of genes, genotype and phenotype, homo-zygosity or heterohomo-zygosity of dominant or recessive genes, the pedigree chart, and rules concerning Punnett squares (i.e., diagrams used to predict the outcome of a particular cross or breeding experiment). The worked example demonstrated how to solve a heredity problem by combining terminology, rules, and theory. Learners were required to use the definitions, rules, and theory underlying heredity problems when carrying out the learning tasks.
3.2.2. Learning tasks
Three learning tasks that were deemed complex for novices were presented as tasks emphasising either problem-solving (i.e., problem-solving task) or worked-example study (i.e., worked-example study task). These tasks consisted of nine information elements on a biological trait in a human family (e.g., ear shape, eye colour, hair colour) and a question about the proportion of possible genotypes and phenotypes of the offspring. Each individual information element was relevant but insufficient for successfully carrying out the task. A problem could only be solved by combining all nine infor-mation elements (JIGSAW;Aronson, Blaney, Stephan, Silkes, & Snapp, 1978; Moreno, 2009; Slavin, 1990). A simplified example of such a task would be that: information element 1 is the mother’s eye colour: blue; element 2 is the father’s eye colour: brown; and element 3 is the dominance of brown over blue for eye colour. Each element gives a certain amount of information, but to determine the eye colour of the offspring, the learner must combine all three information elements. The main difference between the two instructional formats (i.e., problem-solving or worked-example study) is the degree of example elaboration. In the problem-solving condition, learners received the solution to the problem (i.e., the correct answers to the questions), and were asked to use the infor-mation elements that they had received to determine how the solution was reached. Working from the givens to the solution, learners had to rely on problem-solving techniques. The problem solution presented to the learners could be used to verify the result of their problem-solving efforts. In the worked-example study condition, learners were in addition presented with worked-out solution steps, and asked to study
the way the solution was reached (see Fig. 1 for a task emphasising problem-solving andFig. 2for a task emphasis-ing worked-example study (b)). Workemphasis-ing from the givens to the final solution learners could study 11 worked-out solution steps and sub steps.
To prevent learners from incorrectly believing that they had arrived at the correct answer, the study did not use conven-tional problem-solving tasks (i.e., tasks with only a description of givens and a question without the final solution; Sweller, 1988). In previous studies using the same genetic problems as conventional problem-solving tasks, learners quickly decided that they had arrived at the correct problem solution without this actually being the case (Kirschner et al., 2009b, 2010). Providing the correct solution is one way of prevent-ing this and stimulatprevent-ing them to invest effort in the problem-solving steps towards the problem solution. It also provides learners with equal opportunities for learning how to correctly solve a genetics problem in both conditions (i.e., problem-solving and example study). In this way, participants become aware that they had arrived at a correct or incorrect solution, and in the latter case are stimulated to try again.
The tasks that had to be carried out in collaboration were structured such that there was high positive task interdepen-dence (Johnson, 1981; Saavedra, Early, &Van Dyne, 1993). Group members had to rely on each other and interact with each other to obtain resources and to effectively carry out the task. Positive interdependence reflects the degree to which group members are dependent upon each other for effective group performance (i.e., enhanced intra-group interaction). It holds that team members are linked to each other in such a way that each team member cannot succeed unless the others succeed; each member’s work benefits the others and vice versa (Kirschner, Strijbos, Kreijns, & Beers, 2004). This was ach-ieved by giving each group member a booklet containing only one third of the total number of information elements needed to
identify or understand the solution steps. To stimulate collab-oration, cognitive load distribution (i.e., the distribution advantage;Ciborra & Olson, 1988; Hinsz et al., 1997; Kirschner et al., 2009b; Yamane, 1996), and information exchange amongst the learners, there were no redundant information elements and the number of information elements was equal for all group members (i.e., three information elements per group member). In addition to the individual booklets, the whole Fig. 1. A learning task emphasizing problem-solving.
TASK 1: PIET AND LUCY’S FAMILY GIVEN
For humans, the gene for green eye color (G) is dominant over the gene for blue eyes (g).
Sandra has green eyes. Sandra’s mother has green eyes.
Sandra’s mother is homozygote for eye color. Sandra’s father has green eyes.
Sandra’s father is homozygote for eye color. Wim has blue eyes.
Lucy, a child of Sandra and Wim, marries Piet who has green eyes. Piet’s father has blue eyes.
QUESTIONS
What could be the genotypes and phenotypes of Piet and Lucy’s children? And what are their proportions?
SOLUTION STEPS
Below, the solution steps for answering the questions are given. You need the given information to study the solution steps and find out how the steps and final answers were reached.
STEP 1. Determine Piet’s genotype for eye color. Piet’s genotype is Gg.
STEP 2. Determine Lucy’s genotype for eye color.
It is not possible to know Lucy’s genotype at once; first it has to be determined:
STEP 2.1. What is Wim’s genotype for eye color? Wim’s genotype for eye color is gg. STEP 2.2. What is Sandra’s genotype for eye color?
It is not possible to know Sandra’s genotype at once; first it has to be determined:
STEP 2.2.1. What is Sandra’s father’s genotype for eye color? Sandra’s father’s genotype is GG.
STEP 2.2.2. What is Sandra’s mother’s genotype for eye color? Sandra’s mother’s genotype is GG.
STEP 2.2.3. Make a Punnett square between the genotypes of Sandra’s mother and father.
G G G GG GG
G GG GG
STEP 2.2.4. Determine Sandra’s genotype. Sandra’s genotype is GG.
STEP 2.3. Make a Punnett square between the genotypes of Sandra and Wim.
G G g Gg Gg
g Gg Gg
STEP 2.4. Determine Lucy’s genotype for eye color. Lucy’s genotype is Gg.
STEP 3. Make a Punnett square between the genotypes of Piet and Lucy.
G g
G GG Gg
g Gg gg
STEP 4. Determine the genotypes and their proportions of Piet and Lucy’s children. 25% GG – 25% gg – 50% Gg.
STEP 5. Determine the phenotype and their proportions of Piet and Lucy’s children. 75% green eyes – 25% blue eyes.
group received a booklet containing either a question and its solution (i.e., problem-solving condition) or a question, the solution steps, and the solution (i.e., worked-example study condition). This booklet was available to all group members at the same time. Learners working individually received a booklet containing all nine information elements plus the question, solution, and additional solution steps depending on the experimental condition.
3.2.3. Posttest
To determine how much was learned and to see if learners could apply the knowledge and skills that they were assumed to have acquired in the learning phase to a different kind of problem, four transfer-test tasks were used. These tasks required learners to use the same basic terminology, rules, and underlying theory, but they differed from the learning tasks with respect to families and traits used, kind of information elements given, task structure, and questions asked (e.g., genealogical tree, X-chromosome linked inheritance, dihybrid crossings; seeFig. 3for an example). The internal consistency of the posttest (Cronbach’s alpha) was 0.73.
The posttest consisted of two questions per task related to heredity characteristics of a certain trait in a family. With regard to performance, the questions could be scored on multiple elements, with 1 point for a correctly mentioned element and 0 points for an incorrectly or not mentioned element. A maximum score of 28 points could be earned for the four tasks. The minimum score for all tasks was 0. For the statistical analysis, the performance scores were transformed into proportions. In other words, a participant’s score on the four tasks was divided by the maximum score of tasks (i.e., 28). 3.2.4. Instruction
A non-content related instruction on the procedure, rules, and regulations concerning solving problems was presented to
the participants twice; once preceding the learning tasks and once preceding the posttest. The instruction preceding the learning tasks differed slightly between conditions. The indi-vidual learner had to read all information elements before solving the problem, while each group learner only had to read those information elements specifically allotted to her/him (i.e., one third of the total number of elements), but were also required to share the information elements with each other. In the problem-solving condition, participants had to find out how the solution to the problem was established, while in the worked-example study condition participants had to study the given solution steps. Because this study focuses on how instructional format influences collaborative learning, learners were not allowed to write things down during the learning phase. Using an external memory aid (i.e., writing things down) provides an external representation or visualisation of a problem and can, thus, seen as an ‘augmenting’ cognitive activity (Jonassen, Peck, & Wilson, 1999; Pea, 1993). As revealed by Duffy and Cunningham (1996) as well as Van Bruggen, Kirschner, and Jochems (2002), one impact of this is that it leads to ‘cognitive offloading’ allowing learners to devote more memory to other aspects of problem-solution. In this respect, allowing learners to use pencil and paper would skew the research since the basic premise of the research, as stated, was to determine the differential effects of instructional format (and the accompanying cognitive load) on individual and group learning when carrying out a learning task. Intro-ducing memory aids would make it impossible to determine whether the learning formats and contexts increased or decreased cognitive load or whether the results were due to the cognitive offloading.
The instruction preceding the posttest was the same for all participants: First they had to read all information elements thoroughly, then read the questions, and finally try to answer the questions as correctly and quickly as possible using all information elements using pen or pencil and paper to write down the solution steps.
3.2.5. Cognitive load measurement
To measure learner cognitive load after each learning task and test task, the subjective 9-point mental-effort rating scale developed byPaas (1992)was used. Participants were asked to rate the level of effort required to solve a problem on a scale ranging fromvery, very low effort(1) tovery, very high effort (9). This measure provides an overall indication of the cognitive load (i.e., the total of intrinsic, extraneous, and germane cognitive load), has been used in numerous studies dealing with cognitive load, and has proven to be non intru-sive, valid, and reliable (Paas, Van Merrie¨nboer, & Adam, 1994).
3.2.6. Efficiency measurement
The combination of performance and cognitive load measures can provide a reliable estimate of the relative effi-ciency of instructional methods, both in terms of the learning process and learning outcomes. Paas and Van Merrie¨nboer’s (1993)computational approach (see Van Gog & Paas, 2008) Fig. 3. A transfer-test task used in the posttest.
was used to calculate learning efficiency as a function of instructional format. The basic idea underlying this approach is that the combination of performance and mental-effort data collected during a test phase is indicative of the quality of the cognitive schemas constructed during learning. Instructional conditions with higher performance on the posttest in combination with lower or equal invested mental effort (i.e., a lower level of reported cognitive load) are more efficient than lower performance on the posttest in combi-nation with higher or equal invested mental effort; cognitive schemas have been more efficiently acquired. Learning effi-ciency was calculated by standardising each learner’s scores on the posttest and the cognitive load invested working on these tests. For this purpose, the grand mean was subtracted from each score and the result was divided by the overall standard deviation, yielding z-scores for effort (R) and performance (P). Finally, each participant’s performance efficiency score, E, was computed using the formula: E¼[(PR)/21/2]. High learning efficiency was indicated by relatively high posttest performance combined with relatively low mental effort. In contrast, low learning efficiency was indicated by relatively low posttest performance combined with relatively high mental effort.
3.3. Procedure
All participants first individually studied a paper-based general introduction to heredity-related concepts along with a worked example and returned the introduction after 15 min. During the whole experiment, the time (i.e., study and testing) was fixed and was managed by a proctor. They were then randomly assigned to one of the four experimental conditions (i.e., individual problem-solving, individual worked-example study, group problem-solving, and group worked-example study) to carry out the first series of tasks in 7 min. After each task, independent of whether in a group or in an individual condition, all participants rated the amount of invested cognitive load on the 9-point mental effort rating scale. The second and third series of learning tasks followed the same procedure. The instruction for all participants preceding the series of learning tasks consisted of advising them to read all information elements thoroughly, read the questions carefully, and finally try to identify the solution steps to the problem in the case of the problem-solving condition or understand the solution steps in the case of the worked-example study condition, using all information elements. For learners in the group conditions, it was also stressed that working together was necessary. However, to keep the load that is ineffective for learning to a minimum, learners were only permitted to communicate about task-related topics. To prevent them from offloading their WM, they were not allowed to write anything down. After the learning phase, the test phase required participants to individually work on four transfer tasks for 5 min each. Invested mental effort was measured after each transfer-test task with the rating scale. Use of pen or pencil and paper was allowed and stimulated in this phase.
3.4. Data analyses
When analysing the effects of social context (individual vs. group) and instructional format (problem-solving vs. worked-example study), the data-analytical problem of nonindepen-dence had to be taken into account (Cress, 2008; Kenny, Mannetti, Pierro, Livi, & Kashy, 2002). Because students in the group condition worked in triads, they influenced each other through their shared experiences and collaborative discussions. This might cause group members to experience similar amounts of mental effort invested in the learning tasks. Furthermore, because they discussed how to solve the learning task, their performance on the posttest may be correlated. For example, when group members collaborated effectively and found efficient solutions to the problem, they may all perform well on the posttest. On the other hand, when collaboration was not effective and they failed to find solutions to the problem, all group members may perform poorly on the posttest. This violates the assumption of nonindependence of observations of individuals, making the results of traditional analytical techniques, such as ANOVA or MANOVA unreli-able (Kenny, 1995). Multilevel analysis (MLA) can cope with this nonindependence and is therefore a more appropriate technique (Bonito, 2002; Snijders & Bosker, 1999). When investigating the effects of social context and instructional format, MLA will therefore be used.
4. Results
The results for the learning and test phases are described separately. A significance level of 0.05 was used for all analyses. Due to registration problems there were incomplete data from 4 participants in the learning phase and an addi-tional 5 in the test phase. Case-wise deletion of those participants was carried out in the analyses. Table 1 shows the resulting number of participants as well as the means and standard deviations for mental effort as a function of social context and instructional format in the learning phase, and performance, mental effort, and learning efficiency as a function of social context and instructional format in the test phase.
4.1. Learning phase
The data from the learning phase were analysed using a random intercept multilevel model (Snijders & Bosker, 1999) that included three predictor variables: social context (dummy coded with 0¼individual and 1 ¼group), instruc-tional format (dummy coded with 0 ¼ problem-solving and 1 ¼ worked-example study), and the interaction between social context and instructional format. The dependent vari-able was the perceived amount of mental effort invested in studying. The random intercept regression equation is given in Eq. (1), where g00 is the intercept, g10 is the regression coefficient of the group level variable social context,g20is the regression coefficient for the group level variable instructional format, g30 is the regression coefficient for the interaction
between social context and instructional format, U0jis group level variance, andRijis individual level variance.1
Mental effortij ¼g00 þg01Contextjþg02Formatj
þg03ContextjFormatj þU0jþRij ð1Þ
This model was estimated with MLwiN version 2.18.Table 2
shows the estimated model parameters. The MLA shows a significant effect of social context on perceived mental effort, p<.01. The positive sign ofg01shows that group members rated the mean mental effort higher than individuals, independent of instructional format. No effect of instructional format was found, nor was there an interaction effect between social context and instructional format in the learning phase. The deviance reported inTable 2can be used as a test for the goodness-of-fit of the multilevel model. By comparing the deviance value shown in
Table 2to the deviance of a multilevel model without predictor variables (i.e., empty model, labelled Model 1 inTable 2), one can test whether the estimated model fits the data better than the
empty model. In this case, the decrease in deviance was signif-icant,c2(3)¼7.444,p <.05. As seen inTable 2, the model including social context and instructional format was able to explain 12% of the group level variance and 8% of the individual level variance.
4.2. Test phase
The data from the test phase were analysed using a similar multilevel model as the one in Eq. (1). Three different multilevel models were estimated with posttest performance, mental effort, and learning efficiency as dependent variables. For posttest performance, the MLA results inTable 3show a significant effect of social context,p<.01 and instructional format, p < .01. These main effects were qualified by a crossover interaction effect of social context and instruc-tional format,p<.01. Inspection of regression coefficientg03 shows that participants who had learned individually per-formed better on the posttest when they learned from solving problems than when they learned from studying worked examples. The opposite effect was found for participants who Table 1
Means and standard deviations of the dependent variables in the learning and test phase as a function of social context and instructional format. Instructional format Social context
Individual Group
M SD M SD
Learning Phasea Mental Effort (1e9) Problem-solving 4.14 1.29 4.91 0.94
Example study 4.24 1.20 4.75 1.19
Test Phaseb Performance (0e1)c Problem-solving 0.57 0.22 0.76 0.13
Example study 0.69 0.19 0.66 0.23
Mental effort (1e9) Problem-solving 4.90 1.45 4.88 1.29
Example study 4.59 1.30 4.75 1.56
Learning efficiencyd Problem-solving 0.37 1.24 0.26 0.80
Example study 0.16 1.16 0.02 1.45
a
n¼34 for each condition. b
n¼32 for all individual conditions,n¼34 for theegroup-problem-solving conditions,n¼33 for the groupeworked-example study conditions. c
Performance is the proportion of correct answers on the posttest. d Based onz-scores of mental effort and performance in the test phase.
Table 2
Estimates for random intercept model for perceived amount of mental effort in the learning phase.
b SE
g00¼Intercept 4.09 0.20
b1¼Social context 0.821** 0.34
b2¼Instructional format 0.11 0.29
b3¼Social context * Instructional format 0.31 0.49 Variance Group level 0.71 0.20 Individual level 0.60 0.13 Deviance 385.91 Decrease in deviance 7.45* *p<.05 **p<.01. 1
Note: In our study we have a multilevel structure with students nested in groups only as a subset of our sample. For students who worked individually (I-condition), there is no hierarchical clustering in groups, whereas there was clustering for students who worked in triads (T-condition). The research design, thus, can be considered to be a partially nested design (cf.,Bauer, Sterba & Hallfors, 2008). The analyses deal with this by considering students in the I-condition to be members of different 1-person groups. This means that in the ML analyses a nested structure was forced on students in both the I- and the T-condition. We are aware that is not fully consistent with our partially nested design. This for example means that for students in both the I- and T-conditions variance in the dependent variable is decomposed into individual level variance as well as group level variance, while there is actually no group level variance in the I-condition. Bauer et al. proposed a method that better matches such a partially nested dataset where students in the I- condition are still considered to be sole members of a group. However, by using a fixed intercept random slopes multilevel model, variance is decomposed into group-and individual level variance only in the T-condition, which is consistent with the partially nested design. When using Bauer et al.’s approach, highly similar results were obtained. This leads to the conclusion that the initial MLA approach was appropriate. Accordingly, the results presented hereafter refer to ML analyses that do not take the partial nesting into account.
had learned in groups. They performed better on the posttest when they learned from studying worked examples than when they learned from solving problems (seeFig. 4). These results confirm Hypotheses 1a and 2a. Because of this interaction effect, it is important that the effects of social context and instructional format on posttest performance are not consid-ered separately, but rather that they are interpreted together. The goodness-of-fit of the model was adequate as indicated by a significant decrease in deviance compared to the empty model, c2(3) ¼ 12.272, p < .01. Furthermore, the model explained 22% of the variance of the group level variance and 12% of the individual level variance.
The results of the MLA of the effect of social context and instructional format on the perceived mental effort invested in solving test-problems are shown inTable 4. As can be seen, the effects of social context and instructional format, as well as the interaction between social context and instructional format were not significant. The goodness-of-fit of the model was poor,c2(3)¼0.480,ns. Compared to the empty model (Model 1 in Table 4), the model including social context and instructional format explained only a small part of the variance at the group- and individual level (2% and 1% respectively).
Finally,Table 5shows the results of MLA on the effect of social context and instructional format on learning efficiency. A significant effect was found for social context, p <.05, and
instructional format,p<.05. These main effects were qualified by the crossover interaction effect of social context and instructional format,p<.05. The negative sign ofg03should be interpreted as follows: participants who had learned individu-ally carried out the posttest more efficientlyeas indicated by a more favourable relationship between posttest effort and posttest performanceewhen they learned from solving prob-lems than when they learned from studying worked examples. The opposite effect was found for participants who had learned in groups; they carried out the posttest more efficiently when they learned from studying worked examples than when they learned from solving problems. This resulted in an interaction pattern which was similar to the pattern for the posttest performance. Therefore, Hypotheses 1b and 2b were both confirmed. Inspection of the decrease in deviance compared to the empty model revealed that the goodness-of-fit of the model was only marginally significant,c2(3)¼5.572,p¼.05, and that the model explained 5% of the group level and 8% of the indi-vidual level variance.
5. Discussion
This study examined the effects of instructional format and social context along with their interaction on learning outcomes, cognitive load, and learning efficiency. From the performance interaction graph (Fig. 4) it is apparent that
Fig. 4. Performance scores in the test phase as a function of social context and instructional format.
Table 4
Estimates for random intercept model for perceived amount of mental effort in the test phase.
b SE
g00¼Intercept 4.91 0.24
b1¼Social context 0.03 0.36
b2¼Instructional format 0.35 0.35
b3¼Social ContextInstructional format 0.24 0.51 Variance Group level 0.27 Individual level 1.64 Deviance 451.72 Decrease in deviance 0.48 Table 3
Estimates for random intercept model for performance in the test phase.
b SE
g00¼Intercept 0.58 0.03
b1¼Social context 0.180** 0.05
b2¼Instructional format 0.12** 0.05
b3¼Social contextInstructional format 0.22** 0.07 Variance Group level 0.01 Individual level 0.03 Deviance 63.17 Decrease in deviance 12.27** **p<.01. Table 5
Estimates for random intercept model for learning efficiency in the test phase.
b SE
g00¼Intercept 0.37 0.20
b1¼Social context 0.64* 0.31
b2¼Instructional format 0.59* 0.29
b3¼Social context * Instructional format 0.88* 0.44 Variance Group level 0.26 Individual level 1.08 Deviance 404.87 Decrease in deviance 6.57* *p<.05.
although learning from solving problems in a group led to higher learning outcomes than learning from studying worked examples, the reverse relationship between learning from solving problems and studying worked examples was found for individual learners. As there were no differences in the amount of mental effort invested during the posttest, the similar interaction between context and format that was found for learning efficiency seems to be caused by learning outcomes only. These test phase results provide an affirmative and clear answer to the research question, namely that for students learning individually, instruction emphasising worked example study is more effective and efficient than instruction emphasising solving problems while for students learning collaboratively, instruction emphasising problem-solving is more effective and efficient than instruction emphasising worked example study.
Learning outcome and learning efficiency were determined by the crossover interaction between learning individually or in a group from tasks imposing high or low total cognitive load. For problem-solving tasks, though the total cognitive load was high for the individual learner, within groups it could be distributed across group members. Whereas individual learners can be expected to have difficulties processing all information and constructing schemas, as indicated by the lower learning outcome and learning efficiency scores, group members could devote freed-up capacity to activities that fostered schema construction and learning, as indicated by the higher learning outcome and (Hypothesis 1a) learning effi-ciency scores (Hypothesis 1b). Those results make clear that the cognitive investment of inter-individual integration and coordination of information were lower within groups than the benefits of dividing the processing of information across individuals. With regard to studying equivalent worked examples, it can be argued that the decrease in extraneous load left individual learners with enough processing capacity to successfully deal with the information, negating the distribu-tion advantage of groups. Although both individual and group learners could successfully process the information, only group learners had to invest in transactional activities. This disadvantage for group learners resulted in group learners constructing lower quality schemas, as indicated by a lower learning outcome (Hypothesis 2a) and efficiency (Hypothesis 2b) scores than individual learners.
These results were supported by the differential effect of social context on mental effort investment in the learning phase where learners in the group condition invested more mental effort than learners in the individual condition, regardless of instructional format. For learning by studying worked examples, this result supports our explanation that group members, as opposed to individual learners, need to invest additional mental effort for inter-individual integration and coordination of infor-mation and that this additional effort is not directly effective for learning. While group members invested more mental effort than individual learners, this did not result in the construction of cognitive schemas; they learned less efficiently than individual learners. For learning by solving problems, the results support our explanation that the freed-up WM capacity of group
members which resulted from the distribution advantage was devoted to activities that foster learning. The greater invested mental effort of group members in the learning phase resulted in higher performance as well as a more favourable relationship between mental effort and performance on the posttest. This explanation should be further investigated by, for example, observing and analysing group communication and coordination processes to determine what the precise nature of the invested mental effort was.
A closer look at the absolute scores of invested mental effort in the learning phase reveals fairly low average scores (i.e., groupM ¼4.83; individual M ¼4.19), and subtle but significant differences between the experimental conditions. These results could be explained by the consistently found effect that, although the scale ranges from 1 to 9, participants tend to stay in the middle range of the scale (Paas, Tuovinen, Tabbers, & Van Gerven, 2003), and that the subtle differences are reliably indicative for substantial differences in experi-enced cognitive load (Van Gog & Paas, 2008).
Although the cognitive load experienced by individual learners learning from tasks emphasising problem-solving was not high enough for it to be considered cognitive overload (i.e., group M ¼ 4.83; individual M ¼ 4.19), their posttest performance was relatively low. An alternative explanation for this might be that these learners were not able to profit from the discussion, argumentation, and reflection in the group, rather than from not having the advantage of an expanded cognitive capacity. In line with this, the finding that individ-uals had relatively high posttest performance when learning from tasks emphasising worked-example study could be explained by the absence of having to engage in these collaboration processes. The provided solution steps in these learning tasks may have made elaborate problem-solving activities unnecessary. Consequently, individual learners may have performed better than group learners because the latter could not profit from positive interdependence, but more likely suffered from working together with other group members (i.e., collaborative inhibition;Weldon & Bellinger, 1997). To investigate this, combined qualitative process-oriented research and quantitative cognitive load theory research, would lead to better understanding of the coordinative and communicative processes that contribute to the cognitive investment made by learners in collaborative learning (see
Janssen, Kirschner, Erkens, Kirschner, & Paas, 2010). The crossover interaction found for learning individually or collaboratively by solving problems or studying worked examples is similar to the collective working memory effect found by Kirschner et al. (2010) for learning individually or collaboratively on low and high-complexity problems. Combining the findings of both studies, one can argue that the efficiency of individual versus collaborative learning is determined by the complexity of the learning task (i.e., intrinsic load) and the instructional format of the learning tasks (i.e., extraneous and germane load). At a more general level, one can argue that the total cognitive load imposed by the learning tasks, that is, the sum of intrinsic, extraneous, and germane cognitive load (Paas, Tuovinen et al., 2003),
determines the efficiency of individual versus collaborative learning. Learning from tasks that impose high cognitive load can be expected to be more efficient for groups than for individuals, while the opposite can be expected for learning from tasks that impose low cognitive load. An interesting topic for future research would be to investigate the level of cognitive load at which it becomes more effective and/or efficient to learn in a group as opposed to learning individually.
The positive interdependence that was stimulated in this experiment by dividing the information elements over the group members might have influenced the effects of social context. Each of the group members received three informa-tion elements which had to be exchanged to solve the problem or to understand the problem solution. Individual learners, in contrast, were confronted with all nine information elements which might have initiated qualitatively different mental representations for group and individual learners. To deter-mine the magnitude of this effect, future studies should include an experimental condition where, similar to the indi-vidual learning condition, each learner in a group is confronted with all information elements from the outset.
It should be noted that the learning conditions in this study were designed to optimise collaboration. This creates tension between ecological validity and experimental validity. The learning environment differed from settings encountered in ‘real’ education in that all collaborating participants received only part of the unique information elements and, conse-quently, were required to exchange information to solve the problems or study the worked examples. Also, participants were not allowed to offload their WMs by using pencil and/or pen and paper while learning, which also might have stimu-lated them to collaborate. Finally, the learning setting was highly structured and scripted causing a minimal cognitive investment with respect to transactional activities. In this sense, it is not clear to what extent the results obtained in this study can be generalised to real classroom settings. It can be assumed that when all individuals have access to all infor-mation, where offloading WM is possible, and where there are normal transactional activities, different results might be obtained. Future research should investigate the contribution of the ‘artificial’ aspects of the study to the effects of the load imposed by learning tasks on the effectiveness and efficiency of learning in a group.
Finally, a theoretical implication of the results is that the limited processing capacity of an individual learner can be expanded by learning in collaboration with other learners. From this perspective, collaborative learning can be consid-ered an instructional technique for managing individual working memory load. When groups of collaborating learners are considered as information-processing systems (see e.g.,
Hinsz et al., 1997), the information necessary for carrying out a learning task and its associated cognitive load can be distributed across multiple collaborating working memories. The freed-up cognitive capacity of group members can consequently be devoted to activities that foster learning. In other words, through good instructional design, a collective
working memory effect can be achieved. This view has implications for CLT, one of which is that it seems that the functional properties of the learner’s cognitive architecture change when collaborating with others. The expanded limited processing capacity can only be used effectively by inter-individual information communication and coordination processes. It is clear that these processes not only lead to a cognitive investment, but also to affective investments. Until now, CLT has focused on the alignment of instruction with cognitive processes, without recognising the role of affective ones. This research on group learning might stimulate cogni-tive load theorists to address affeccogni-tive issues in their research. Practically, the results suggest that wholesale adoption of collaborative learning is not a sensible educational practice. The challenges that a learning task poses to the learner’s cognitive capacity and/or the amount of cognitive load a task imposes, should be determining factors when deciding whether to employ a learning model or environment based upon an individual or a collaborative learning paradigm. The higher the load imposed by the learning tasks, the more likely that collaborative learning will lead to better learning outcomes, in terms of effectiveness, efficiency, or both. This means that when choosing collaborative learning as an educational approach, educational designersemost often the teachers themselves e must assure themselves that the learning tasks given to the groups (e.g., problems, projects, et cetera) are complex enough that they cannot be carried out easily by an individual. This also suggests that it would be better for practitioners not to make an exclusive choice for individual or collaborative learning, but rather vary their approach depending on the complexity of the learning tasks to be carried out and the goals of the instruction.
References
Aronson, E., Blaney, N., Stephan, G., Silkes, J., & Snapp, M. (1978).The jigsaw classroom. Beverly Hills, CA: Sage Publications.
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. W. (2000). Learning from examples: instructional principles from the worked-examples research. Review of Educational Research, 70, 181e214. doi:10.3102/ 00346543070002181.
Ayres, P., & Paas, F. (2009). Interdisciplinary perspectives inspiring a new generation of cognitive load research.Educational Psychology Review, 21, 1e9. doi:10.1007/s10648-008-9090-7.
Bauer, D. J., Sterba, S. K., & Hallfors, D. D. (2008). Evaluating group-based interventions when control participants are ungrouped. Multivariate Behavioral Research, 43, 210e236. doi:10.1080/00273170802034810.
Bonito, J. A. (2002). The analysis of participation in small groups: method-ological and conceptual issues related to interdependence.Small Group Research, 33, 412e438. doi:10.1177/104649640203300402.
Carroll, J. (1994). Relating complexity to practical performance in parsing with wide-coverage unification grammars. In Proceedings of the 32nd annual meeting of the association for computational linguistics, NMSU, Las Cruces, NM, USA, 287e294. doi:10.3115/981732.981772.
Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction.Cognition and Instruction, 8, 293e332.
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55e81. doi:10.1016/0010-0285(73)90004-2.
Ciborra, C., & Olson, M. H. (1988). Encountering electronic work groups: a transaction costs perspective. In Proceedings of the 1988 ACM
conference on computer-supported cooperative work. doi:10.1145/62266. 62274. Portland, OR, USA.
Cowan, N. (2001). The magical number 4 in short-term memory: a reconsid-eration of mental storage capacity. Behavioral and Brain Sciences, 24, 87e114. doi:10.1017/S0140525X01003922.
Cress, U. (2008). The need for considering multilevel analysis in CSCL research: an appeal for the use of more advanced statistical methods.
International Journal of Computer-Supported Collaborative Learning, 3, 69e84. doi:10.1007/s11412-007-9032-2.
Deiglmayr, A., & Spada, H. (2010). Training for fostering knowledge co-construction from collaborative inference-drawing.Learning and Instruc-tion.Advance Online Publication. doi:10.1016/j.learninstruc.2010.06.004. Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: implications for
the design and delivery of instruction. In D. H. Jonassen (Ed.),Handbook of research for educational communications and technology (pp. 170e198). New York: Macmillan Library Reference.
Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups as information processors. Psychological Bulletin, 121, 43e64.
Hu¨bner, S., Nu¨ckles, M., & Renkl, A. (2010). Writing learning journals: instructional support to overcome learning-strategy deficits.Learning and Instruction, 20, 18e29. doi:10.1016/j.learninstruc.2008.12.001.
Ickes, W., & Gonzalez, R. (1994). “Social” cognition and social cognition: from the subjective to the intersubjective. Small Group Research, 25, 294e315. doi:10.1177/1046496494252008.
Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010). Making the black box of collaborative learning transparant: combining process-oriented and cognitive-load approaches.Educational Psychology Review, 22, 139e154. doi:10.1007/s10648-010-9131-x.
Johnson, D. W. (1981). Student-student interaction: the neglected variable in education.Educational Research, 10, 5e10, Retrieved from.http://www. jstor.org/stable/1175627.
Johnson, D. W., & Johnson, R. T. (1981). Effects of cooperative and indi-vidualistic learning experiences on interethnic interaction. Journal of Educational Psychology, 73, 444e449. doi:10.1037/0022-0663.73.3.
444.
Johnson, D. W., Johnson, R. T., & Stanne, M. B. (2001). Impact of goal and resource interdependence on problem-solving success. The Journal of Social Psychology, 129, 621e629.
Jonassen, D. H., Peck, K. L., & Wilson, B. G. (1999).Learning with tech-nology: A constructivist perspective. Upper Saddle River, NJ: Prentice Hall.
Kenny, D. A. (1995). The effect of nonindependence on significance testing in dyadic research. Personal Relationships, 2, 67e75. doi:10.1111/j.1475-6811.1995.tb00078.x.
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups.Journal of Personality and Social Psychology, 83, 126e137. doi:10.1037//0022-3514.83.1.126. Kirschner, F., Paas, F., & Kirschner, P. A. (2009a). A cognitive load approach
to collaborative learning: united brains for complex tasks.Educational Psychology Review, 21, 31e42. doi:10.1007/s10648-008-9095-2.
Kirschner, F., Paas, F., & Kirschner, P. A. (2009b). Individual and group-based learning from complex cognitive tasks: effects on retention and transfer efficiency. Computers in Human Behavior, 25, 306e314. doi:10.1016/j. chb.2008.12.008.
Kirschner, F., Paas, F., & Kirschner, P. A. (2010). Task complexity as a driver for collaborative learning efficiency: the collective working-memory effect.Applied Cognitive Psychology.Advance Online Publication. doi:10. 1002/acp.1730.
Kirschner, P. A. (2002). Cognitive load theory: implications of cognitive load theory on the design of learning.Learning and Instruction, 12, 1e10. doi: 10.1016/S0959-4752(01)00014-7.
Kirschner, P. A., Strijbos, J. W., Kreijns, K., & Beers, P. J. (2004). Designing electronic collaborative learning environments. Educational Technology Research and Development, 52, 47e66. doi:10.1007/BF02504675. Langfred, C. (2000). Work-group design and autonomy: a field study of the
interaction between task interdependence and group autonomy. Small Group Research, 31, 54e70. doi:10.1177/104649640003100103.
Michinov, N., & Michinov, E. (2009). Investigating the relationship between transactive memory and performance in collaborative learning.Learning and Instruction, 19, 43e54. doi:10.1016/j.learninstruc.2008.01.003. Miller, G. (1956). The magical number seven, plus or minus two: some limits
on our capacity for processing information. Psychological Review, 63, 81e97. doi:10.1037/h0043158.
Moreland, R. L., & Argote, L. (2003). Transactive memory in dynamic organizations. In R. Peterson, & E. Mannix (Eds.), Understanding the dynamic organization(pp. 135e162). Mahwah, NJ: Erlbaum.
Moreno, R. (2009). Constructing knowledge with an agent-based instructional program: a comparison of cooperative and individual meaning making.
Learning and Instruction, 19, 433e444. doi:10.1016/j.learninstruc.2009. 02.018.
Ohtsubo, Y. (2005). Should information be redundantly distributed among group members? Effective use of group memory in collaborative problem solving.
Applied Cognitive Psychology, 19, 1219e1233. doi:10.1002/acp.1162.
Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive load approach. Journal of Educational Psychology, 84, 429e434.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instruc-tional design: recent developments. Educational Psychologist, 38, 1e4. doi:10.1007/s10648-005-3951-0.
Paas, F., Renkl, A., & Sweller, J. (2004). Cognitive load theory: instructional implications of the interaction between information structures and cogni-tive architecture. Instructional Science, 32, 1e8. doi:10.1023/B:TRUC. 0000021806.17516.d0.
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory.
Educational Psychologist, 38, 63e71.
Paas, F., & Van Gog, T. (2006). Optimising worked example instruction: different ways to increase germane cognitive load.Learning and Instruc-tion, 16, 87e91. doi:10.1016/j.learninstruc.2006.02.004.
Paas, F., & Van Merrie¨nboer, J. J. G. (1993). The efficiency of instructional conditions: an approach to combine mental-effort and performance measures.Human Factors, 35, 737e743.
Paas, F., & Van Merrie¨nboer, J. J. G. (1994). Variability of worked-examples and transfer of geometrical problem-solving skills: a cognitive-load approach. Journal of Educational Psychology, 86, 122e133, Retrieved from.http://doc.utwente.nl/26427/1/Paas94variability.pdf.
Paas, F., Van Merrie¨nboer, J. J. G., & Adam, J. J. (1994). Measurement of cognitive load in instructional research.Perceptual and Motor Skills, 79, 419e430.
Pea, R. D. (1993). Practices of distributed intelligence and designs for education. In G. Salomon (Ed.),Distributed cognition: Psychological and educational considerations(pp. 47e87). Cambridge: Cambridge University Press. Peterson, L., & Peterson, M. (1959). Short-term retention of individual verbal
items. Journal of Experimental Psychology, 58, 193e198. doi:10.1037/ h0049234.
Prichard, J. S., Stratford, R. J., & Bizo, L. A. (2006). Team-skills training enhances collaborative learning.Learning and Instruction, 16, 256e265.
doi:10.1016/j.learninstruc.2006.03.005.
Renkl, A. (1997). Learning from worked-out examples: a study on individual differences.Cognitive Science, 21, 1e29. doi:10.1207/s15516709cog2101_1.
Rourke, A., & Sweller, J. (2009). The worked-example effect using ill-defined problems: learning to recognise designers’ styles.Learning and Instruc-tion, 19, 185e199. doi:10.1016/j.learninstruc.2008.03.006.
Saavedra, R., Early, P. C., & Van Dyne, L. (1993). Complex interdependence in task-performing groups.Journal of Applied Psychology, 78, 61e72. Simon, H. A. (1974). How big is a chunk?Science, 183, 482e488. doi:10.
1126/science.183.4124.482.
Slavin, R. E. (1990). Cooperative learning.Review of Educational Research, 50, 315e342. doi:10.3102/00346543050002315.
Snijders, T. A. B., & Bosker, R. J. (1999).Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage Publications. Stark, R., Kopp, V., & Fischer, M. R. (2010). Case-based learning with worked
examples in complex domains: two experimental studies in undergraduate medical education.Learning and Instruction.Advance Online Publication. doi:10.1016/j.learninstruc.2009.10.001.
Stasser, G., Stewart, D. D., & Wittenbaum, G. M. (1995). Experts roles and information exchange during discussion: the importance of knowing who knows what.Journal of Experimental Social Psychology, 31, 244e265. doi:10.1006/jesp.1995.1012.
Sweller, J. (1988). Cognitive load during problem solving: effects on learning. Cognitive Science, 12, 257e285. doi:10.1016/0364-0213(88) 90023-7.
Sweller, J. (2004). Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture. Instruc-tional Science, 32, 9e31. doi:10.1023/B:TRUC.0000021808.72598.4d. Sweller, J. (2006). The worked example effect and human cognition.Learning
and Instruction, 16, 165e169. doi:10.1016/j.learninstruc.2006.02.005. Sweller, J., Chandler, P., Tierney, P., & Cooper, M. (1990). Cognitive load as
a factor in the structuring of technical material.Journal of Experimental Psychology: General, 119, 176e192.
Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 58e89.
Sweller, J., Van Merrie¨nboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design.Educational Psychology Review, 10, 251e295. doi:10.1023/A:1022193728205.
Tindale, R. S., & Kameda, T. (2000). Social sharedness as a unifying theme for information processing in groups.Group Processes and Intergroup Rela-tions, 3, 123e140. doi:10.1177/1368430200003002002.
Tindale, R. S., & Sheffey, S. (2002). Shared information, cognitive load and group memory.Group Processes and Intergroup Relations, 5, 5e18. doi: 10.1177/1368430202005001535.
Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying examples and solving problems to skill acquisition. In M. Polson (Ed.),Proceedings
of the fifteenth annual conference of the cognitive science society (pp. 1017e1022). Hillsdale, NJ: Erlbaum.
Tuovinen, J. E., & Paas, F. (2004). Exploring multidimensional approaches to the efficiency of instructional conditions. Instructional Science, 32, 133e152. doi:10.1023/B:TRUC.0000021813.24669.62.
Van Bruggen, J., Kirschner, P. A., & Jochems, W. (2002). External representation of argumentation in CSCL and the management of cognitive load.Learning and Instruction, 12, 121e138. doi:10.1016/S0959-4752(01)00019-6. Van Gog, T., Kester, L., & Paas, F. (2010). Effects of worked examples,
example-problem, and problem-example pairs on novices’ learning. Contemporary Educational Psychology. Advanced Online Publication.doi: 10.1016/j.cedpsych.2010.10.004.
Van Gog, T., & Paas, F. (2008). Instructional efficiency: revisiting the original construct in educational research. Educational Psychologist, 43, 1e11. doi:10.1080/00461520701756248.
Van Merrie¨nboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: recent developments and future directions.Educational Psychology Review, 17, 147e177. doi:10.1007/s10648-005-3951-0. Wegner, D. M. (1978). Transactive memory: a contemporary analysis of the
group mind. In B. Mullen, & G. R. Goethals (Eds.),Theories of group behavior(pp. 185e208). New York: Springer-Verlag.
Wegner, D. M. (1995). A computer network model of human transactive memory.Social Cognition, 13, 319e339.
Weldon, M. S., & Bellinger, K. D. (1997). Collective memory: collaborative and individual processes in remembering. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 1160e1175. Yamane, D. (1996). Collaboration and its discontents: steps toward
over-coming barriers to successful group projects. Teaching Sociology, 24, 378e383, Retrieved from.http://www.jstor.org/stable/1318875.
