Measures and Instruments

Word Naming and Word Meaning Explanation

To examine the students’ learning of the five focal new words taught in each story, two tasks—word naming and word meaning explanation—were given to all students six days after experiencing each condition. In the task of word naming, pronouncing both two characters in a word correctly earned one point. In scoring students’ word naming, students were not penalized for not using the correct tones because they often used rising intonation when not certain about the pronunciation of the characters. Therefore, it was challenging to decide whether they had known the tones of the characters. Accurately explaining each word received one point. Their responses were

to explain a word by using the word in a sentence, translating the word into English, or using gestures to point to an object. Their oral responses were recorded on answer sheets by the trained experimenters.

According to Beck, McKeoun, and Kucan’s (2002) definition of word tier, these 20 focal new words were categorized into Tier 1 words (n = 8) and Tier 2 words (n = 12). Tier 1 are more basic and concrete in meaning, such as 玻璃/bo1li2 (glass), 铁锤

/tie3chui2 (metal hammer), or 轮胎/lun2tai1 (tire); Tier 2 words represent more precise or mature ways of referring to more complex or abstract ideas and meaning concepts 独 孤/gu1du2 (lonely), 骄傲/jiao1ao4 (proud), or 捣乱/dao3luan4 (to make a mess).

Chinese Word Reading

To tap into students’ general Chinese literacy knowledge previously acquired in their program, the measure of Chinese word reading was tested at the end of the study. The Chinese word reading task included 20 single-character and 60 two-character words. These words were chosen from children’s Chinese textbooks and storybooks. Children were asked to name each word presented on a paper. Scores were based on the number of correctly named characters. The total possible score for this task was 80. The task was administered individually with trained experimenters. If a child failed to name 10 item words consecutively, the testing was stopped. The internal reliability (Cronbach’s α) of the task in the present study is .92.

4.2.7 Data Analysis

Descriptive analysis, contrast analysis, and two-level multilevel analysis were used to compare the learning of Chinese words in the four conditions. Descriptive analysis provided raw scores for correctly naming or explaining each word in each condition. Three sets of contrasts were set up to compare the learning of sounds and meanings of the Chinese words between conditions: Full Pinyin and Partial Pinyin conditions, No Pinyin and Pinyin conditions (i.e., Full and Partial Pinyin combined), and No Instruction and the three learning conditions combined (see Table 4.2). Finally, multilevel modelling was used to compare the learning in the Pinyin and No Pinyin conditions by taking account of students’ prior Chinese literacy knowledge and word tier. Multilevel modelling is more appropriate than single-level regression analysis or fixed- effects ANOVA in the present study because this approach takes into account of the fact that there are multiple observations from each participant, which violates the assumption of independence of the observations in single level models. Multilevel regression analysis provides information on fixed and random effects of the between-students variable at Level 2. Because the scores in the tasks of Word Naming and Word Meaning

Explanation were either 0 or 1, multilevel logistic regression models were created. The significance threshold of all the analyses was set at .05.

In the multilevel models, the response variables are the correct scores for each word in the tasks of Word Naming and Word Meaning Explanation. The three levels of learning conditions are within-students variable at Level 1, the categorical variable of word tier (WT) is a within-students variable at Level 1, and the continuous variable of Chinese word reading (CWR) is a between-students variable at Level 2. The fixed effects were modelled at level 1 for learning conditions and WT. The random effect was

modelled at Level 2 for CWR. The formula is expressed as follows:

Yij = β0j + β1jconditionij + β2jWTij + rij (Level 1 model)

β0j = γ00 + γ01 CWRj + u0j (Level 2 model)

β1j = γ10

β2j = γ20

Yij = γ00 + γ01 CWRj + u0j + γ10conditionij + γ20WTij + rij (Multilevel model)

Yij is the observed score in word i for student j.

Table 4.2 Comparisons Weights for Each Conditions in Contrast Analysis

P vs. F N vs. P + F NI vs. N + P + F

NI 0 0 1

N 0 1 -1/3

P 1 -1/2 -1/3

F -1 -1/2 -1/3

Note. NI = No instruction, N = No Pinyin condition, P = Partial Pinyin condition, and F = Full Pinyin condition.

γ00 is the mean score in the No Pinyin condition.

γ01 is the coefficient for the Level 2 factor of Chinese word reading.

u0j is the residual for student j at level 2.

γ10 is the coefficient for the Level 1 predictor of condition.

γ20 is the coefficient for the Level 1 predictor of word tier.

rij is the residual in word i for student j.

Therefore, the multilevel logistic regression model for the task of Word Naming (WN) and for the task of Word Meaning Explanation (WME) as the response variable are expressed as follows:

Logit(WNij) = γ00 + γ01 CWRj + u0j + γ10conditionij + + γ20WTij + rij (Multilevel model)

4.3 Results

RQ1. Does the use of Pinyin captioning in reading materials can facilitate the learning of Chinese words in meaningful reading activities?

4.3.1 Descriptive Analyses

Descriptive statistics for the correctness of word naming and word meaning explanation in four conditions are displayed in Tables 4.3 and 4.4. The mean scores of Word Naming and Word Meaning Explanation were very low in the No Instruction condition, indicating that the focal characters were mostly unfamiliar to the students or the students were not able to guess the sound or meaning of the characters. In both tasks, the mean scores in the No Pinyin condition were noticeably higher than those in Partial Pinyin and Full Pinyin conditions. The mean scores between Full and Partial Pinyin conditions were very similar.

Table 4. 3 Descriptive Results in the Task of Word Naming

Condition M SD Min Max

Full Pinyin (n = 71) 1.06 1.11 0 5

Partial Pinyin (n = 69) 1.06 1.36 0 5

No Pinyin (n = 70) 1.27 1.51 0 5

No Instruction (n = 76) 0.09 0.37 0 2

Table 4. 4 Descriptive Results in the Task of Word Meaning Explanation

Condition M SD Min. Max.

Full Pinyin (n = 71) _{1.23 1.24 0 5}

Partial Pinyin (n = 69) _{1.35 1.50 0 5}

No Pinyin (n = 70) _{1.51 1.60 0 5}

No Instruction (n = 76) 0.09 0.41 0 3

Note. The possible total score in each task in each story was 5.