Data Collection Issues

This section focuses specifically on the issues that arose from my data collection methods and gives a detailed analysis on each of the methods that provided data for this research.

3.10.1. Focus Groups

Zorn, Roper, Broadfoot and Weaver (2006) warned of group conformity and dominant group members sometimes present in discussions. However, based on my history as a Kurashiki junior high school employee and my familiarity with one member of each focus group I felt the sessions were uchi or inside/in-group encounters based on trust, where participants spoke their mind freely and honestly. I decreased the threat of showing conformity in my results by recording the sessions and transcribing the discussion verbatim. By doing so I could identify negotiations that took place on key issues before opinions converged. Morgan (1996, p. 146) concluded that smaller groups were more appropriate when dealing with emotionally charged topics, therefore the small size of my focus groups in this case was apposite.

3.10.2. The Q-sort Process

Unfortunately, I could not complete the Q-sort process, as described by Nahm et al., perfectly (see 3.5.6). Whether my compromise of increasing judge numbers while only completing the process once was an effective solution is debateable, however, a good level of inter-judge agreement was reached in all three questionnaires. Nahm et al. emphasized that this procedure leans towards qualitative analysis rather than being a thorough

3.10.3. Back-translation

In his paper on questionnaire translation Griffee (1999) stated that documents become invalid through translation. This is because some words have no equivalent in other languages, or may have a looser or more figurative meaning. Though the Q-sort was in English and the administered questionnaires were in Japanese the questionnaires were subjected to back-translation. Questionnaire items were initially written in English and translated into Japanese, and then back again into English; each by a separate translator. When the initial English item list and the final, back-translated English list were compared, side-by-side, mistakes were easily identified and corrected by another translator.

Furthermore, after the back-translation process the Japanese versions of the questionnaires were piloted by the groups that it would actually survey to check for any ambiguous or irrelevant items.

3.10.4. The JTE Questionnaire

Unfortunately questionnaire administration for this research was carried out only once. During data analysis I became aware of some shortcomings present in the

questionnaire. One deficiency was the absence of in-depth questions on some central themes. Question 13, for example, could have been expanded upon. Though JTEs indicated whether their lesson content is affected by MEXT policy it would have been beneficial to know how they were affected. A question examining the level of JTE awareness on MEXT policy should have also been included. Such points were examined in the one-on-one interviews, yet this information can only be considered as the opinions of two JTEs and may not be applicable to JTEs in general. Another area requiring further research was looked at in questions 17 and 18, while general attitudes were received it is apparent that information on what elements of pre-service training were missing or what parts of in- service training are useful/not useful could have been pursued.

It may also have been wise to examine which student grade the respondent was currently teaching as this may have an influence on the amount of English use in class. In

my experience third grade EFL lessons featured far less spoken English because the JTEs focused on preparing their students for the senior high school entrance examination.

The official sanctioning and promotion of the questionnaire by the local Board of Education may have influenced JTE responses in certain sections. JTEs might have been concerned about confidentiality breaches resulting in answers being passed on to the Kurashiki Board of Education. However, responses to questions 13 to 16 showed policy makers, who are directly linked to the Kurashiki Board of Education, in a very poor light. These responses would indicate that JTEs were not overly concerned about this possibility.

Despite the issues detailed I felt a satisfactory response rate was achieved. 57 of the 90 possible JTE participants responded which is a response rate of just over 63%.

3.10.5. The Student Questionnaire

The student questionnaire only related to Research Question One so a long, in-depth questionnaire was unnecessary. Due to budget constraints and a desire for simplicity the questionnaire was only one A4 paper sheet long. In hindsight a more spacious format to allow for the addition of a comments section after each question may have provided some rich information to supplement the figures. An item examining why students felt they would or would not need to speak English in the future would have provided a useful insight into their motivation toward EFL.

3.10.6. The Parent Questionnaire

The point about a more spacious format alluded to in 3.9.5 is also relevant for the parent questionnaire. While there was no space designed for comments on this

questionnaire some parents wrote comments beside and in between questions. One can only imagine how much rich data could have been gained with the addition of comment sections. This questionnaire may feature more bias than the other two questionnaires because, in all probability, respondents may have only been the parents positively active in their child gaining a good rounded education. Possibly the more highly educated parents made up

most of the respondents as they are generally the ones that care most about education, since they realise the benefits. They are also more likely to recognise the benefits of their

children learning English.

A problem relating to response numbers was the number of hands that this

questionnaire had to pass through before data analysis. It is impossible to know how many parents never saw the questionnaire, but it is probable that some parents failed to receive them from their children which obviously had a negative bearing on response rates. Despite this issue, of the 337 questionnaires given to the students for their parents 267 were

completed and returned, a response rate of just over 79%.

3.10.7. General Questionnaire Issues

A problem inherent in all questionnaires is that the information gathered may only be applicable for a short time. Furthermore, in regards to the JTE questionnaire, there is no way to gauge if an answer was based on practice rather than beliefs, which is not to say that respondents are dishonest. Yet perception and reality are two different things.

Questionnaires merely indicate what respondents state they are doing or provide an interpretation of ‘their world’. In question 10 of the JTE questionnaire JTEs may have given answers more indicative of how much in-class English they would like to think of themselves using or how much they thought they should be using rather than how much they use in reality.

Another problem with categorical data like that gained from my likert scales is that there is no indicator of the degree of conviction in respondent answers. Though two respondents may have given the same answer one may have had far stronger feelings in their answer than the other. I tried to minimise this problem by having comment areas after questions in the JTE questionnaire.

3.10.8. One-On-One interviews

While useful in filling gaps, it must be remembered that these interviews can only realistically be considered as one person’s opinion. However, more positively they did provide the rare chance to listen to teachers talk about their experiences, or “lived reality”, in a Japanese public junior high school EFL classroom.

3.11. Summary

The focus for this research is the junior high school EFL environment in Japan. The research participants are those intimately involved– the JTEs, the students, and the parents. There were numerous stages undertaken in developing the questionnaires, particularly the JTE questionnaire, before they were administered. The method used to fill information gaps left by the JTE questionnaire was in the form of two, one-on-one interviews. There were numerous data collection issues in my research. The most prominent of these were the absence of in-depth questions in the questionnaire, general questionnaire issues such as inadequate indication of participant conviction in answers, and the bias potential of the one- on-one interviews. I was often completely dependent on my Japanese informants. While, without their assistance, I could not have undertaken this research, when one relies so heavily on others it can prove to be difficult, especially when communication breaks down. A fine example of such a time was when the JTE helping with the student and parent questionnaires did not show the principal copies of the questionnaires before 500 copies were printed. After the principal saw them, and rejected them, I possessed nothing but paper for recycling.