Process data and their measurement

The data for the social and cognitive processes were collected during the collaborative learning phase. The data source for the process measures is the discourse of one of the three web-based discussion boards. Social modes of co-construction and epistemic activities have been analyzed with a cod- ing system for multi-level analysis of knowledge co-construction processes in discourse (Weinberger, Fischer, & Mandl, 2002), which is based on a coding system by Bruhn, Gräsel, Fischer, and Mandl (1997) that has been applied by Fischer et al. (2002).

Unit of analysis

In the quantitative analyses, the unit of analysis of the process meas- ures is the relationship between two subsequent conceptual components of the discourse of the students (cf. Hofer & Pikowsky, 1993). For instance, the sentence “The cue that no one in the family is witty anyway is equiva- lent to an attribution on talent” consists of the case information “that no one in the family is witty” on the one hand and the concept “attribution on tal- ent” on the other hand. The relation between the conceptual components is defined by the words “is equivalent.” Conceptual components may be spread over syntactical units such as sentences. For instance, “My first im- pression is that the pupil stresses his lack of talent. This is an internal attri- bution.” One conceptual component is “that the pupil stresses his lack of talent” which is related to another conceptual component, namely “internal attribution.” The utterance that relates two conceptual components may be more relevant for collaborative knowledge construction because the coarser granularity of this unit of analysis may indicate components of mental mod- els better than syntactical units (cf. Chi, 1997; Chi, DeLeeuw, Chiu, & LaVancher, 1994; Hofer & Pikowsky, 1993).

Rater training

Two raters have been trained to identify these units of analysis by processing ca. 1000 units together. After that, inter-rater reliability was de- termined on the grounds of roughly 500 units more that the raters analyzed independently from each other. Discrepancies were not resolved in accor- dance with (Chi, 1997), “because resolving them can actually bias the inter- pretation of subsequent codings” (p. 307). Coder-correspondence for identi- fying units was 87%, determined on the grounds of the ca. 500 units, which is an acceptable ratio (cf. Chi, 1997).

Overall raw data

In total, the discourses of all groups in one of the three web-based discussion boards consisted of 17863 words (not including the prompts) distributed over 412 messages. On average, 184 words (SD = 133.18) were posted in 4 messages (SD = 3.66) per person in one of the three web-based discussion boards. These discourses have been segmented into a total of 1763 codeable units (6 utterances were incomplete and could not be coded). On average, each learner produced 18 codeable units for this one problem case (SD = 14.34). There are no substantial differences regarding the num- ber of units of analysis between the experimental groups. For a clearer rep- resentation and better comparability, the process measures of social modes of co-construction and epistemic activities will be additionally reported in percentages. Effects of the scripts will be presented with z-scores.

Categorization

With the help of the ‘Coding System for a Multi-Level Analysis of Knowledge Co-Construction’ (Weinberger et al., 2002), each unit can be analyzed simultaneously on levels of social processes (social modes of co- construction) and cognitive processes (epistemic activities).

(1) Social modes of co-construction. Any unit of analysis in the dis- course was coded with respect to social modes of co-construction that indi-

cate an increasing degree of transactivity, namely, externalisation, elicita- tion, quick consensus building, integration-oriented consensus building, and conflict-oriented consensus building. The inter-rater reliability regarding social modes of co-construction, which was measured with Cohen’s Kappa, ZDV

(a) Externalization. Units have been coded as externalization if they were neither prompted by learning partners or constructed as a response to any other contribution of learning partners. Thus, new initiating messages typically contained externalization, as for instance in “My first impression is that the pupil ascribes his bad performance in mathematics to talent.”

(b) Elicitation. A unit is coded as elicitation if a learner aims to di- rectly trigger a specific reaction from the learning partners. This is typically done by question asking (e.g., “Why do you believe that Michael ascribes his deficits to an internal, stable cause?”), but syntactically not restricted to questions. Equally, directive discourse moves, such as “You should mention the attribution of the parents,” aim to elicit a specific reaction by a learning partner and are accordingly coded as elicitation.

(c) Quick consensus building has been coded when learners agree without further comments (e.g., “Yes”). Agreement may be expressed by short sign of approvals or by literal repetition of what has already been said (A: “Michael does not feel like making an effort in math;” B: “He simply does not learn for this subject”) or by juxtaposition (“Both are somehow correct”).

(d) Integration-oriented consensus building. Segments in which con- tributions by a learning partner are adopted into one’s own considerations, but were not previously considered, have been coded as integration-oriented consensus building. This includes sublation of perspectives and explicit adoption of perspectives (e.g., A: “Michael attributes to internal, stable causes;” B: “The parents attribute to talent;” C: “Both Michael and his par- ents have a detrimental attribution pattern”), but does not refer to mere agreement.

(e) Conflict-oriented consensus building. Utterances that dismiss, modify, or devaluate contributions of learning partners have been coded as conflict-oriented consensus building. In this way, not only explicit, but also implicit rejections indicated by slight repairs or modifications of contribu- tions of partners are regarded to as conflict-oriented. Thus, individual con- flict-oriented segments are indicated by explicit rejections (“I don’t think so”), replacements (A: “The attribution of the teacher is de-motivating;” B: “The attribution of the teacher is beneficial”), modification (A: “The attri- bution of the parents is positive because it liberates Michael of his feelings of guilt;” B: “It is positive in the sense that the parents do not put pressure on Michael, but accept him principally”), or critical endorsement (A: “The teacher motivates Michael by ascribing his bad performance to laziness;” B: “The teacher motivates Michael by also evaluating the attributions of his parents”).

(2) Epistemic and non-epistemic activities. Raters had to differenti- ate the various epistemic activities and if a unit of analysis was epistemic or non-epistemic. The inter-rater reliability of the individual epistemic and QRQHSLVWHPLF DFWLYLWLHV WKDW ZHUH PHDVXUHGZLWK &RKHQ¶V .DSSD ZDV .90.

Contributions have been classified as non-epistemic, when the con- tent of the contributions was neither about theoretical concepts, nor case information. Thus, typical for non-epistemic activities were students’ di- gressions off topic, e.g., “The weather could be better today.” Coordinating activities of learners have been also coded as non-epistemic, e.g., “Where is everybody?”. The epistemic activities describe the tasks of learners to con- struct problem space, conceptual space, and interrelations between both spaces.

(a) Construction of problem space. Relations between two concep- tual components have been coded as construction of problem space when both components are case information. In other words, participants con- structed problem space whenever they repeated or paraphrased case infor- mation in order to acquire a better model of the situation. One characteristic of the construction of problem space is that no theoretical concepts are re-

ported as in, for example “Michael’s mother says that no one in the family is actually witty anyway; this she said with a smile.”

(b) Construction of conceptual space. If participants built relations between two theoretical concepts, this relation was coded as construction of conceptual space. Thus, the repetition of concepts of the theoretical text has been coded as construction of theoretical conceptual space. In the example “An attribution towards talent is an internal, stable attribution,” a relation between two theoretical concepts is constructed (“attribution towards talent” and “internal, stable attribution”).

(c) Construction of relations between conceptual and problem space. When the participants related theoretical concepts and case information, these utterances were classified as constructions of relations between con- ceptual and problem space. In other words, this category describes how learners applied concepts to problems. For example, in the utterance “That Michael said, ‘I am just not talented’ points to an internal attribution,” the theoretical concept “internal attribution” was applied to the case information “Michael said ‘I am just not talented.’”