Instrument and data collocation procedure

The main instrument of the current study is one-on-one interview with all

participants. The following section provides a description of the format and procedure of the interviews and other data collection methods.

4.2.4.1. Instruments

In the present study, spoken data were elicited through one-on-one interviews with all 30 participants. Each 1-hour long interview consists of two parts: 30 minutes of conversation on daily topics and 30 minutes of discussion on academic topics. The description of these two parts of the interview is presented below.

One part of the interview is a casual conversation of common non-academic daily topics between me and the participant. Each participant was asked a number of questions regarding their personal background, language learning experience, daily activities and hobbies in a casual and spontaneous way. The list of questions used in this part of the interview can be found in Appendix B. Although the same list of questions was used as guiding topics for each interview, this part of the conversation is still very open and includes a variety of different topics which were brought up by the participants during the interview.

The other part of the interview is structured around the discussion and narration of abstract academic topics. Two types of prompts (open-ended question prompt and

cartoon interpretation prompt) are employed in this part to elicit participants’ spoken response to abstract topics. First, because all participants in this study are enrolled in graduate-level coursework and are trained to perform academic research in their own fields, each participant was asked to introduce in detail an academic research project they were involved in or were planning to carry out.

After being asked about their own research projects, participants were further asked to comment on two political cartoons that the author chose for this study from major news agencies. The two political cartoons concern two widely recognized social and cultural issues: China’s environmental pollution and China’s educational system reform (Appendix C). Each participant was given 2 minutes to read each cartoon carefully and then was simply prompted to describe the cartoon and offer their analysis and opinions on the issue reflected by the cartoon. The participant then gave a narration largely in the form of a monologue. In some cases, I asked simple follow-up questions to elicit more responses.

4.2.4.2. Interview Procedure

Each participant had one-on-one interview with me in a common office setting. Because each interview consists of two parts, the study employs a counterbalanced design to minimize the order effect of sequencing the two types of topic. For each group of 10 participants, I would begin 5 interviews with the topic of daily conversation and begin the other 5 interviews with the topic of academic discussion.

The interview process with the three groups is basically the same, except that both CFL learners and CHL learners were asked to fill in a simple language background questionnaire at the beginning of interview whereas native speaker did not fill in the questionnaire. The questionnaire was used to collect general information about learners’ language learning including native language, years of formal Chinese learning, study- abroad experience and other related information. The language questionnaire used in this study could be found in Appendix A.

For each interview, I would start by introducing the topic and purpose of the current study and asking the participant to sign the consent form. I would then move on to one of the two types of topics.

For the daily conversation topics, the interview is carried out in casual question- and-answer format and I tried the best to create relaxed environment for the participants. For example, the following questions were asked during the interview:

 “我很想听听你是怎么开始对中文/英文有兴趣的。（I really want to hear about how you first became interested in learning Chinese/English.）”

 “你好像很喜欢旅游，可以聊聊你最近一次旅行的经历吗？（It looks like you like travelling a lot, could we talk about one of your recent travel experiences?”

While interviewing on more abstract academic topics, I tried to elicit more formal and extended spoken language. For one elicitation prompt, I raised the following open- ended question:

 “请向我具体地描述一下你读研究生时做过的一个研究项目，你认为这个研

究项目有什么重要性呢？(Can you describe to me in details about a research project you have done for your graduate work and discuss why you think it is important.)”

And for the cartoon picture prompt, the spoken instruction was given as follows:

 “请你仔细看一下这幅漫画，然后谈谈漫画的内容和意义。(Please read this

cartoon carefully and comment on the content and meaning of it.”

In instances when a participant only had very little to say about the political cartoon, simple follow-up questions would be asked to extend the discussion:

 “很好的想法，还有别的方面吗？(Good thoughts, any other aspects?)”

 “你觉得中国的教育体系需要怎样的变化呢？(What kind of change do you

Since the current study is about collocational use, I also tried to minimize the use of V-N collocations in my own questions and prompts. This was done through careful planning of questions and prompts in advance.

Both parts of the interview were recorded using two audio-recording devices. For each part of the interview, I tried to control the length to be as close to 30 minutes as possible, so the total length of each interview is about 1 hour.

After all interviews were carried out, I transcribed all interviews and organized all transcriptions into three datasets: CFL learners, CHL learners and CNSs. All instances of V-N collocations were then manually identified and classified in the three datasets.

It is also important to note the limitations regarding the multiple identities of me, the researcher, as both the language instructor of the participants and the investigator of the research study. On the one hand, the familiarity between myself and most of the learners in the study could to a certain extent facilitate the interview process. Learners may feel more relaxed in answering questions and thus can produce longer discourses. On the other hand, such familiarity may also present a problem for examining the influence of different topics. This is because most one-on-one interactions between learners and the language instructor tend to be casual and conversational, thus learners might not feel the need to use formal and academic language even in discussing abstract topics. Also, instead of being a natural conversation, learners may perceive the interview more as a language practice with the instructor and this can also influence their overall language fluency and accuracy.

4.2.4.3. Examine the degree of acceptability and communicativeness of V-N collocations produced by learners

After all instances of V-N collocations were identified in the data, I then try to decide whether the collocations produced by learners are commonly used in the target language and whether the collocations are communicative in expressing meanings.

The first step is to determine the degree of conventionality. Since the current study involves collocation usage, it would be ideal to check every combination against huge spoken corpora and/or judged by large numbers of native speakers. However, there currently is no extensive Chinese spoken corpus and it is also beyond the scope of this study to have every combination judged by large numbers of native speakers. Instead, a more practical approach is adopted following the methods of Nesselhauf (2004) and Xin (2014).

In the present study, three types of sources are adopted to judge the overall degree of conventionality of each collocation produced by learners in the data: dictionaries, written corpora and experts’ judgements.

First, collocations were judged conventional if they occurred in identical form and with the same (or similar) meaning in one of the following three widely used Chinese

dictionaries: 《现代汉语词典》(Contemporary Chinese Dictionary published in

mainland China), 《国语辞典》(Chinese Dictionary published in Taiwan), and《现代

汉语搭配词典》(Dictionary of Modern Chinese collocations published in mainland China).

Second, collocations were also judged conventional if they occurred in similar form and meaning for at least three times in the Peking University Contemporary Chinese Language Corpus (CCL). Up to now, there is no large scale and easily accessible Chinese spoken language corpus available to researchers. However, most of the major

contemporary Chinese language corpus includes a wide variety of written language samples such as literature, newspaper, academic, blogs and micro blogs. And the inclusion of many informal sources and on-line media would to some degree reflect the common and everyday use of Chinese language under a spoken context.

All the V-N collocations that could not be judged as conventional on the basis of dictionaries and the CCL were then presented to expert raters for acceptability

judgements. The two primary expert raters were native speakers of mandarin Chinese from mainland China and both were Chinese instructors teaching at the college level in the U.S. The V-N collocations were presented in context (highlighted in the original sentence produced by the learners) to the native speaker experts, so that the collocations could be judged in relation to their intended meaning in context. The raters were asked to judge the collocations on a five-point scale from completely unacceptable (1)

unacceptable (2) not sure (3) acceptable (4) to completely acceptable (5). Whenever a combination was judged “not sure”, “unacceptable” or “completely unacceptable”, the raters were also asked to provide an acceptable or better option to express the intended meaning.

Initially, two native speaker raters were asked to judge and rate all the non- conventional V-N collocations used by the learners. If their ratings were the same, their

judgement constituted the final score for the collocation. If their ratings had a difference of 1 point, an average score would be calculated to be the final score. If the two raters’ scores had a difference of two or more points, one additional native speaker rater was asked to provide a judgement. After a third score was given for the collocation, an average score would be calculated using all three scores.

This method of judging the acceptability of collocations produced by learners is limited and remains an approximation. As pointed out by Nesselhauf (2004), “there is not necessarily a one-to-one relation between what native speakers find acceptable or

unacceptable when explicitly asked about a certain language phenomenon and what they themselves produce frequently” (p. 53) However, a number of studies have argued that there is good correlation between corpus data and native speaker judgement on lexical co-occurrence (e.g., Lapata et al, 1999; Hoffmann & Lehmann 2000; Shei 1999,

Neselhauf, 2004). For the present study, the number of native speaker raters for judging the acceptability of collocations is relatively small and therefore the acceptability score for each collocation should be used only as an approximate number to show the overall trend and characteristics.

The second step is to determine the communicativeness of all unconventional collocations. The term “communicativeness” is adopted in the current study to mean if a speaker can convey his/her intended meaning to a listener through the use of certain expressions. In previous literature, most researchers have chosen to study only

acceptability of collocations. However, as communicative competence has increasingly been emphasized by language educators at an important goal of learning English and as

more researchers start to view learner language as “asset” rather than “deficit”, it seems equally important to find out how listeners perceive the spoken language produced by learners.

As an attempt to determine the communicativeness of learner language, all unconventional collocations were again presented to the two expert raters for judgments and their judgments are based on three scales: (1) not communicative (2) not sure and (3) communicative. The two raters’ judgments would then be compared and analyzed.