Control variables were assessed with the help of pen and paper tests and questionnaires that included multiple choice items and a baseline case. The data for the control variables were collected prior to the experiment except for computer-specific attitudes, which were measured after the col- laborative phase in order to avoid priming effects. The individual control variables included:
(1) Demographic data. Several demographic variables, namely gen- der, age, and first language (and years speaking German as a foreign lan- guage) have been assessed with a questionnaire.
(2) Prior knowledge. Prior knowledge has been assessed with respect to applicable knowledge. Prior applicable knowledge has been assessed with a baseline case, comparable to the problem cases of the collaborative phase and the transfer case. For this reason, the ‘coding system for a multi-level analysis of knowledge co-construction’ has been applied to the pre-test case analyses. Raters were again blind with respect to experimental condition. 629 units of analysis have been segmented and categorized regarding appli- cable prior knowledge. With the students’ analysis of the problem case, fo- cused and multi-perspective applicable prior knowledge was measured. However, the 1st-year-students disposed of extremely little applicable prior knowledge. Thus, 90% of the students were unable to construct a relation that was attributable to focused applicable knowledge, and 76% of the stu- dents could not produce relations attributable to multi-perspective applica-
ble knowledge. Three coding units regarding focused as well as multi- perspective applicable knowledge were shown as the empirical maximum of individuals’ prior knowledge. Due to this floor effect, possible differences UHJDUGLQJ IRFXVHG &URQEDFK¶V = .49) and multi-perspective applicable NQRZOHGJH&URQEDFK¶V = .33) could not be reliably measured.
(3) Learning strategies. The learning strategies were measured with the help of Wild and Schiefele’s (1994) scale prior to the experiment. Reli- DELOLW\ZDVPHDVXUHGZLWK&URQEDFK¶V$OSKD = .64).
(4) Social anxiety. Social anxiety was measured with the help of Rost and Schermer’s (1997) scale prior to the experiment. Reliability was measXUHGZLWK&URQEDFK¶V$OSKD = .92).
(5) Uncertainty orientation. Uncertainty orientation has been as- VHVVHG SULRU WR WKH H[SHULPHQW DV ZHOO ZLWK &URQEDFK¶V = .72 (Dalbert, 1996).
(6) Interest. The interest towards the learning environment has been measured with a scale of some self-developed items in a questionnaire prior to the experiment based on Prenzel, Eitel, Holzbach, Schoenhein, and Schweiberer (1993; see also Krapp, 1999), e.g., “Please mark the statements that apply to you with an × (from ‘does not apply’ to ‘does apply’): ‘I am interested in getting to know new pedagogical theories and concepts’” &URQEDFK¶V = .74).
(7) Computer-specific attitudes. In order to not prime participants with respect to learning with computers, participants were asked after the experiment to report their computer-specific attitudes regarding their belief of how computers may de-personalize society, based on a questionnaire by Richter et al. (2001)&URQEDFKV = .77.
6.7 Treatment Check
It has been checked if the treatments were realized by the partici- pants in the intended way. Both SCOS- and ECOS-prompts should have been answered according to the intention of the individual prompt. For in- stance, the SCOS-prompt “WE HAVE NOT REACHED CONSENSUS CONCERNING THESE ASPECTS:” should have been followed by differ- ence of opinions between the learning partners. In other words, the learning partners were supposed to engage in conflict-oriented consensus building. If the learners engaged in other social modes, e.g., if the response to this prompt signaled quick consensus building, the prompt has been coded as ‘not answered in the intended way’ regardless of possible reasons for not responding in the intended way, e.g., lack of knowledge. Therefore, the treatment check consisted of the assessment of responses to the prompt that diverged from the intention of the prompt. Additionally to unintended re- sponses, missing responses to prompts were counted and entered the treat- ment check. The results of the treatment checks are calculated in relation to the number of prompts of the individual conditions.
Additionally, SCOS guided learners through the individual discus- sion boards of the problem cases and pre-structured the number of the mes- sages that the participants should contribute. This number of messages was the same for all participants (eight messages in total). Therefore, the number of messages and the heterogeneity of the number of messages will be ana- lyzed as additional treatment checks of the SCOS. As an indicator for het- erogeneity of the number of messages within the groups, dissimilarity scores based on standard deviations of the number of messages will be analyzed (cf. Cooke, Salas, Cannon-Bowers, & Stout, 2000; Fischer & Mandl, 2001).
6.8 Statistic Procedures
All main and interaction effects will be tested with univariate ANOVA for statistical significance. Significant findings of an univariate
ANOVA will be reported with regard to their effect sizes of explained vari- ance ( 2). Post-hoc group comparisons will be calculated with unpaired t- tests. For correlation of the single process dimensions with the outcomes of collaborative knowledge construction, multiple linear regressions will be conducted. Effect sizes will be reported with reference to the explained vari- ance with the adjusted R2-value. Relations between outcomes as co- construct and as individual acquisition will be calculated with Pearson’s bivariate correlation.
As the individual learners influence each other in the learning groups, all tests will be calculated based on groups of three, as unit of analy- sis. In order to explore relations between processes and individually ac- quired knowledge an exception to this group level procedure will be made because aggregation bias can be expected (Sellin, 1990). A hierarchical lin- ear model may not be applied due to high requirements of this statistic pro- cedure, e.g., large number of aggregated units (Ditton, 1998; Mok, 1996). Relations between processes and knowledge as co-construct will not be cal- culated, as variables of both dimensions feed from the same data in the col- laborative phase.
As the total amount of analysis segments of the data sources are not identical, processes and learning outcomes will be graphically presented as z-scores to improve comparability. The z-transformations have been calcu- ODWHGRYHUWKHZKROHVDPSOH$Q OHYHORIZDVXVHG IRUDOOVWDWLVWLFDO tests, except differences between learning prerequisites. In order to warrant, WKDWWKHJURXSVGLGQRWGLIIHUUHJDUGLQJOHDUQLQJSUHUHTXLVLWHVDQ OHYHORI .10 was used instead.