Data collection

After the first version was created and translated, a small group of English speaking Chinese teachers of the Bachelor of Business, graduating Chinese B Bus students and Chinese employers were the pilot subjects. The purpose was to confirm if the questions conveyed the intended meaning. This process ensured familiarity with the research content and influenced the research design. Thus the outcomes of the pilot, as stated by Yin (2003, p. 79), ―help you to refine your data collection plans with respect to both the content of the data and the procedures to be followed‖ (discussed further in 6.5.4). Where pilot questions were still confusing, the question was either adapted and retested or abandoned and jargon such as ―graduate capabilities‖ was avoided. The translations (see 6.5.2) were extensively reviewed, individually and then collaboratively, by three bilingual teachers for their clarity, cultural and educational meaning. A Likert scale was deemed cross culturally appropriate, as the less confusing option and hence the more robust source of data collection; discussed further in 6.5.5.1).

To address Singh‘s (1995) focus on functional and conceptual equivalence and thus enable

appropriate and meaningful data to be collected from graduating students, lecturers and employers, the precise wording of the survey questions was tailored for each group. The same invitation letter and explanatory document accompanied each survey and preceded each interview (see Appendix J). Confidentiality and privacy were assured and the Head of School‘s phone number was included as a contact point if respondents had queries. Care was taken to ensure that the survey was professionally presented and the interview conducted within the University‘s Ethics guidelines.

All surveys contained the following four sections:

A. Questions asking for background information, unique to each group

B. Questions about the importance for graduating students to have certain personal characteristics

C. Questions about the importance for graduating students to have certain broad attributes D. Questions about the importance for graduating students to have certain workplace skills

The ‗Lecturer‘ and ‗Student‘ questionnaires also contained the following two sections: E.. Questions about the teaching and learning program

155

F. Questions about the frequency of encouraging students to acquire certain skills

The ‗Student‘ survey contained a seventh section:

G. Question about the number of semesters spent studying in Australia

The ‗Employer‘ questionnaire contained a fifth section:

H. Questions seeking information about behaviour and attitudes of recent employed graduates

Exploratory interviews were conducted to inform the survey questions and more formal interviews after the analysis of the data to further validate the findings. The interview process is discussed in 6.6.

6.5.4.1 Pilot study

A pilot study was conducted to detect any design weaknesses. According to Baumgartner and Strong (1998) the objectives of a pilot study are:

1. To determine whether the questions provide appropriate and needed data; 2. To ensure that the respondents understand the context of questions; and 3. To rectify any ambiguity in both the instructions and questions.

Fink (1995) argues that at least ten people for each population are required for an effective pilot and that the draft questionnaire should be piloted with a similar group to the final sample population. In this research, twelve surveys were distributed to representatives of the sample population: four to graduating students, four to visiting Chinese ―joint program‖ lecturers and four to Chinese employers. Participation was voluntary but due to existing relationships any respondent who was unable to

complete the pilot handed it on to a colleague. Within one week six completed surveys were received. While this was less than the desired ten, the feedback from each was clear, could be cross-checked and was incorporated if appropriate. The findings and subsequent modifications to the questionnaire are presented below:

1. Several ideas that participants found confusing were changed. For instance, the lecturers‘ question ―What are your qualifications?‖ was changed to ―What is your highest

qualification?‖ This avoided unnecessary data and the expense of paying for more translation than necessary.

2. One student complained about the questionnaire length: ―it will take a long time to answer so many questions‖. The researcher then raised this issue with the other five respondents but none considered the length an issue. She also asked what questions could be omitted to which

156

there was no consensus. However the length may explain the 50 per cent return rate. Even so, the researcher did not change the length of the questionnaire.

3. The presentation of the survey was amended, and divided clearly into sections and subsections. But there was a further request for the font size to be slightly larger and

important words bolded. Therefore, font size was increased to 11 points and, in consultation with informants, key words agreed on and bolded.

4. Employers and lecturers raised several questions about professional development, often regarded as training conducted by the Chinese Ministry of Education or professional industry bodies. In order to achieve a wider understanding the words ―…further courses or…‖ were added.

5. The piloted questionnaire contained a five-point Not Important to Highly Important scale. On reviewing the spread of answers and a marked tendency to be neutral, and following further review of relevant literature, this centre-point was removed (see 6.5.5.1) (Chang 1994; Hu & Korllos 1995; Lee et al. 2002).

The next iteration of the draft questionnaire was electronically sent to four colleagues in China with an email explaining the nature and objectives of the research. A follow-up internet phone

conversation with each colleague led to two instructional changes and the inclusion of the section G question. Apart from these changes, the questionnaire was now acceptable and likely to produce reliable and valid data. Combined feedback on the two drafts meant that ten people who resemble the sample population had piloted the survey.