Prior to performing statistical analyses for the alignment data, principal components analysis (PCA), a variable-reduction technique, was conducted to arrange variables into separate components (factors) based on how strongly correlated variables are with each other. For the current study, results of PCA determined if the four subtests included in the LLAMA language aptitude tests (i.e., LLAMA B, D, E, and F) should be collapsed into larger variables or principal components, and, if so, to provide weighted scores that could then be used as independent variables in statistical models. The same PCA procedures were used to examine if the three subcomponents of the perception questionnaire (participant’s perception of the interlocutor’s language proficiency in terms of fluency, vocabulary competence, and grammatical competence) could be grouped into separate components. Weighted scores for the component(s) were to be used as fixed effect(s) in the subsequent statistical analyses.
In addition to PCA, three reliability analyses were conducted to measure internal consistency reliability of questionnaire items for each of the following three questionnaires: experiential cognitive style, rational cognitive style, and perception of task experience. If the questionnaire items are found to be strongly associated with one another, representing the relevant construct (experiential cognitive style, rational cognitive style, and perception of task experience), an average score for each participant can be used as an independent variable.
Statistical analysis of data was carried out in SAS 9.4 (SAS Institute Inc., Cary, North Carolina, USA) In order to systematically answer the research questions, generalized linear mixed-effects models (GLMM) with a logit link function (i.e., logit mixed-effects models) were developed to analyze binary categorical data. These logit mixed-effects models (henceforth, logit mixed models) offer methodological advantages over the use of traditional regression and
analysis of variance (ANOVA) analyses (Baayen, Davidson, & Bates, 2008; Jaeger, 2008). First, mixed-effects models allow for the inclusion of participant-level and item-level factors in a single unified analysis and therefore, the analysis does not require averaging over participants or items (Boyd & Goldberg, 2012). Mixed-effects models additionally allow for non-independence of data, which means that one participant or item can contribute more than one data point per condition, allowing for an analysis of raw data points rather than a single mean score per participant, as would occur in traditional ANOVA methods. Furthermore, the fixed effects component of a mixed-effects model can contain multiple independent variables of interest to the researcher, including categorical predictors (e.g., aligned vs. misaligned), continuous predictors (e.g., age), or a mixture of the two. Mixed models also include random effects in order to account for variation attributed to individual differences amongst participants and items. Finally, mixed- effects models are robust in the presence of several problems known to affect ANOVA validity including unequal N sizes, missing data, non-normality, and heteroscedasticity (Quene & van den Bergh, 2008). As such, mixed-effects models allow for more accurate interpretations of the influence of specific effects when attempting to measure the influence of a predictor variable on an outcome variable (Baayen et al., 2008).
Several logit mixed models were constructed to systematically answer the research questions. The first logit mixed model was constructed to answer RQ 1, which concerns the
linguistic alignment effects. RQ 1 examined the extent to which linguistic alignment occurred during L2 peer interaction in FTF and SMMC mode and the mediating effects of modality (RQ 1-1), social factors (RQ 1-2), and individual differences in cognitive abilities (RQ 1-3). In total, two logit mixed models were developed for the two types of alignment: structural alignment and lexical alignment. The amount of structural and lexical alignment was measured separately to address RQ 1 for each of the respective constructs. Following previous research (e.g., Jung et al., 2017; McDonough et al., 2015; Trofimovich et al., 2014), successful alignment effects refer to learners’ production of the target linguistic features after hearing the interlocutor’s production of
the identical grammatical structure (for structural alignment) or the same word (for lexical alignment).
The dependent variable for research question (RQ 1) and its three sub-questions concerning the effects of moderating factors on the magnitude of alignment was the degree of alignment operationalized as the amount of aligned production of the target linguistic feature in the alignment activities. The fixed effects included modality (categorical; FTF vs. SMMC), social factors (continuous; participants’ perceptions of their interlocutor with respect to proficiency, comprehensibility, and task experience), and cognitive factors (continuous; cognitive style, aptitude for explicit and implicit language learning, language analysis ability, and English proficiency) as well as participants’ demographic information (i.e., sex, age, length of English study). Prime type (prime vs. non-prime; if alignment occurred following a prime or a non-prime) was also entered as a fixed effect only in the structural alignment model in order to account for instances where participants used a stranded preposition in RC after hearing a filler sentence without the target structure. Cognitive style was subdivided into the rational-analytical (rationality) and the experiential-intuitive cognitive styles (experientiality). Individual
participants’ rationality and experientiality scores were entered into the model to examine the
relationship between cognitive style and structural alignment effects. Based on the results of PCA, the aptitude variable had two components including aptitude_explicit and
aptitude_implicit, which were included as separate factors representing the participants’ explicit language aptitude and implicit language aptitude, respectively (Saito, Suzukida, & Sun, 2018). Participant and item were included as random intercepts. A random slope of item was added to the random effect of participant.
To address the research questions concerning the role of alignment activities on L2 development (RQ 2) and the moderating role of modality (RQ 2-1), social factors (RQ 2-2), and individual differences in cognitive abilities (RQ 2-3) on the learning effects, four logit mixed models were fitted to the measurement data from sentence production task, GJT, word
production test, and word translation test. For the second RQ and its sub-questions concerning the learning effects of the alignment activities, the dependent variable was subsequent learning effect of the alignment activities measured by learners’ performance in the pretest, immediate posttest, and delayed posttest of the measurement tests (i.e., sentence production test, GJT, word production test, and word translation test). For all the mixed models, time (pretest, immediate posttest, and delayed posttest) and group (experimental vs. control) were included as fixed effects in addition to the variables used in the alignment models. Additionally, a two-way interaction between time and group and a three-way interaction between time, group, and modality were also included in the analysis. The random effects includedrandom intercepts by participants and items (i.e., test items of the measurement test). A random slope of item was added to the random effect of participant.
To present results, the solution for fixed effects and type III tests of fixed effects were used to infer the statistical significance of fixed effects and interactions on each dependent variable. For significant interactions between fixed effects of particular interest to the study, a
posteriori pairwise comparisons of least-square mean values were conducted. Results of the
pairwise comparisons tested whether any learning effects carried over from the treatment sessions.
4 RESULTS