Phase Two – The Interview Phase

During the time students were completing the last of the questionnaires the students’ New Zealand Qualifications Authority (NZQA) consent forms were sent to a designated staff member at NZQA. The staff member subsequently emailed me each participant’s level 1 and level 2 NCEA English achievement standards results. These results were one of three criteria used to determine which 16 of the 107 students would be selected for an interview. The other two criteria used to decide who would be interviewed were gender, and those who had indicated they would be willing to receive information about the possibility of being interviewed. Students results and gender were the key criteria in determining who was selected to be part of a stratified purposive sample.

First, students’ NCEA level 2 English results for all the English achieved standards against which they were assessed were checked. Students received one of four results for each achievement standard: Not Achieved, Achieved, Merit, or Excellence.

Students were grouped into five groups. Those who gained mainly Excellences were placed in one group. This pattern was repeated to form a mainly Merits group, a mainly Achieved group, and a mainly Not Achieved group. The fifth group comprised of

students who had a mixture of results (e.g., one Excellence, one Merit, two Achieved, and one Not Achieved). This last group was discarded from the pool of potential interviewees, because of the variation in their results.

Second, two females and two males were selected from each of the four categories (e.g., two females and two males who got mainly Excellences), with preference being given to those who had the greatest number of results all in the same category. Third, these students’ names were checked against a master list, which indicated whether individual students had initially agreed to receive information about possibly being interviewed on the consent form they signed when initially agreeing to complete questionnaires (see Appendix A7).

Once the 16 students had been selected, each was sent an information sheet and a consent form (see Appendices A12 and A13). All agreed to be individually interviewed and a suitable time was negotiated with each participant in May 2011. The interviews were all scheduled during school hours, but outside timetabled classes (e.g., during a lunch break or a study period). A small office was used in each school to conduct the interviews. As these offices were not normally occupied by anyone associated with senior management, it was assumed the offices had no negative connotations for the interviewees.

Prior to beginning the formal interview, efforts were made to put the students at ease. Establishing a positive, professional rapport was assisted by the fact that students had met with me over several lunch breaks. Furthermore, the students had completed 11 questionnaires, so were aware that the focus was their motivation in NCEA English and they were familiar with the type of questions I might be asking. In addition to explaining the interview process, students were reminded that they did not need to answer any questions they were not comfortable answering and that there were no right or wrong answers.

Each student was asked questions from the interview schedule that were specifically written for them. Additional questions were asked during the interview when further information was required to gain a fuller understanding. Non-directive questions were used in such instances, such as, “Could you tell me more about …?” Students were encouraged to take time to think about each question, rather than rush to give a response. Care was taken with my body language, and my comments, so as not to unduly influence the interviewee’s responses. However, efforts were made to show genuine interest in what the interviewee had to say. Each interview was digitally recorded with the permission of the students. At the end of the interviews students were thanked for their willingness to be interviewed and for sharing their thoughts about their motivation.

The transcriber transcribed the interviews verbatim into Word documents. Transcripts were checked for accuracy and then sent to the 16 participants. They were asked to read the transcription of their interview, and add or delete anything that they believed more accurately reflected what they wanted to say. Two students added additional points, but no deletions were made. All participants returned the document, along with their signed copy of the Authority for the Release of Transcripts form (see Appendix A14).

The flow diagram in Figure 4.1 is designed to explicate the data collection procedures for Phase One and Phase Two of this study. In particular, the diagram highlights when specific questionnaires were administered in Phase One and when interviews occurred in Phase Two.

Figure 4.1: Data collection procedures

*While this is the typical sequence and timing for the administration of Pre- and Post-Achievement Standards Questionnaires for 2.1 and 2.2, in a couple of cases teachers choose to complete work on 2.2 before focusing on 2.1. In these instances the questionnaires for 2.2 were administered first.

School year began in February. Participants sought at the beginning of March. Phase One: Students completed Initial Questionnaire in March/April.

Students completed a unit of work for 2.1, during which time they undertook trial assessments and received teacher feedback. For their summative assessment for 2.1 they received general teacher feedback on their draft piece of creative writing, reworked their writing and then submitted their final piece for marking. Students then completed Pre-Achievement Standard

Questionnaire 2.1 in April.*

Students received their grades for 2.1 and then completed Post- Achievement Standard Questionnaire 2.1 in June.*

Students completed a unit of work for 2.2, during which time they undertook trial assessments and received teacher feedback. For

students’ summative assessment for 2.2 they received general teacher feedback on their draft pieces of formal writing, reworked their writing and then submitted their final piece for marking. Students then completed Pre- Achievement Standard Questionnaire 2.2 in July.* Students received

their grades for 2.2. and then completed Post- Achievement Standard Questionnaire 2.2 in August.*

Students completed units of work for 2.3, 2.4 and 2.6 (in addition to work for remaining level 2 English achievement standards), during which time they undertook trial assessments and received teacher feedback. In September students completed mock exams for 2.3/2.4 and 2.6. They then received their mock exam results.

Students completed Pre- Achievement Standard Questionnaire 2.6 in October. Within ten days they completed Pre-Achievement Standard Questionnaire 2.3 or 2.4 and the Outside Class Activities Questionnaire.

In mid- November students were assessed for 2.3/2.4 and 2.6 in external national exams. Students’ exams were

marked. They received their NCEA level 2 results and their marked exam scripts later in January 2011. They could then request that their exam answers be re-marked. Students completed Post-

Achievement Standard Questionnaires 2.3/2.4 and 2.6 in March. Within ten days they completed the Final

Questionnaire.

Phase Two: NCEA level 2 English results, requested by the researcher and received from NZQA in March, were used to help identify16 students for interviews.

Sixteen students individually interviewed in May.

4.7. Data Analysis

To analyse the quantitative data from the 11 questionnaires, students’ responses were initially coded and entered into the computer software program IBM SPSS Statistics. SPSS was used primarily to produce frequency tables. Inferential statistical analyses were also undertaken to examine gender differences using the Mann-Whitney U test and the chi-square test for independence. Spearman’s rho test was also employed to examine the strength of relationships where appropriate.

The qualitative data in this study were analysed by me with the aid of the qualitative computer software program NVivo (www.qsrinternational.com). NVivo enabled the large amount of qualitative data generated in this study to be effectively and efficiently coded. The coding process would have been more difficult, problematic and time- consuming if done without the assistance of such software. For example, NVivo

enabled all the relevant data to be easily collated for closer inspection and analysis (e.g., all the answers to a particular question coded to one node). Categories and subcategories could be readily set up at any point in the coding process and data could be easily copied into these different categories or subcategories as themes became evident. NVivo also enabled data within categories to be readily checked for

consistency, while at the same time enabling data to be revisited in its original context. Furthermore, word or phrase searches could be undertaken and memos added as ideas emerged during the analysis and coding and process.

For Phase One, folders were set up in Nvivo for each of the 11 questionnaires, and the 1177 Word files containing each student’s open-ended responses for each

questionnaire were imported into NVivo. All the students’ responses to each open- ended question within each questionnaire were then collated together at a node, which is where NVivo stores a category (Richards, 2009).

Once data were collated for each question, specific questions were selected for analysis. The students’ responses for that particular question were read in conjunction with relevant quantitative results. For example, Question 20a in the Initial

Questionnaire asked students to indicate whether they would have enrolled in Year 12 English if it had not been a compulsory subject. InQuestion 20b students were asked to explain the reason for their response in part ‘a’. The quantitative results for part ‘a’ assisted with the interpretation and coding of responses for part ‘b’ of the same question, thus enhancing coding consistency and the validity of the interpretations.

The collated qualitative data for each question were examined to identify what was significant, and to gain a sense of what patterns, themes, ideas and concepts were prevalent in the data. Sub-nodes were created to capture theoretical constructs from Ryan and Deci’s (2000a) taxonomy of human motivation (e.g., introjected regulation). Following the examination of the data, additional sub-nodes were also created for other relevant social and contextual themes (e.g., peers or teachers).

The data within each sub-node were checked for intra-coder reliability. When

inconsistencies were detected data were re-assigned to the correct sub-nodes or new sub-nodes were created to deal with themes not initially captured. Existing sub-nodes were collapsed where they were found to be addressing the same theme, or relabelled and redefined to better capture the meaning of the data. For example, there were initially two separate categories for the theme difficulties with or dislike of aspects of English; one focused on the difficulties while another focused on the dislike. However, it became apparent that a comment about disliking was often accompanied by a comment about difficulties, or vice versa. Given that students often disliked something because they found it difficult, or found something difficult because they disliked it, a decision was made to combine these two categories into one. Internal homogeneity and external heterogeneity were continually sought through this refinement process (Patton, 2002). The small quantity of data that was unable to be coded into any meaningful categories has been reported in the next chapter.

Coding sets of data that answered the same question asked in four different questionnaires, and further analysing data coded to the same theme across the different questionnaires, provided repeated opportunities to confirm, further refine, or challenge initial coding decisions. This iterative process not only enhanced coding consistency, but also assisted in gaining greater clarity and understanding of the data. Memos were also made during the coding process to capture important ideas, question unexpected responses, and draw tentative inferences (Johnson & Christensen, 2012).

An inter-coder reliability check was undertaken for 20% of students’ responses (42 of 212 responses) to Question 11b in the Final Questionnaire. This question was pivotal in the study, because students were asked to identify the factors that most influenced their motivation overall. The 42 responses selected were initially coded as examples of different types of motivation from Ryan and Deci’s (2000a) taxonomy of human

motivation. The other coder was very familiar with this taxonomy. A master list of the 42 responses was created. Each response was coded as it had been in NVivo. The other

coder was sent a copy of the list, minus the original coding and any identifying

information about the participants. The other coder asked to code the responses using Ryan and Deci’s (2000a) taxonomy of human motivation (e.g., as examples of

amotivation or introjection). This coding was checked against the original coding on the master list. There was 100% agreement.

For the data analysis in Phase Two, each of the 16 students’ 11 questionnaire responses and their interview transcripts were subsequently imported into a folder in

NVivo; 192 files in total. The same qualitative data analysis processes and intra-rater reliability checks were employed for these data as were employed for the qualitative questionnaire data analysis outlined above.

Once data were coded they were re-examined and evaluated for their degree of significance, alongside the relevant quantitative results. When determining the degree of significance, links to theory, and the weight, coherence, and consistency of the data were all taken into consideration. In doing so, patterns, nuances within those patterns, and anomalies were considered. For example, peers were found to play a role in some students’ motivation to achieve. In deciding the significance of peers on students’ motivation to achieve across the 107 participants, it was important to consider in what ways peers influenced students’ motivation; what weight students placed on the influence of peers (e.g., did they list peers as very important influences and rate them as very influential in different questionnaires); whether it was classmates and/or friends who were influential; whether the influence was positive or negative; literature on the influence of peers; and the degree to which the data supported or challenged

theoretical understandings about motivation.

To ensure that more salient themes were clearly evident when making inferences from the qualitative data, some qualitative data were transformed into quantitative data (Tashakkori & Teddlie, 1998). To provide transparency around this process and thus enable judgments to be made about conversion validity, additional details were provided about what was being counted (e.g., number of responses, or number of students). Additional qualitative and/or quantitative evidence was also sought across and within questionnaires to confirm or disconfirm tentative conclusions. This process was aided by text searches in NVivo. Once both types of data were analysed, the inferences from the quantitative and qualitative strands of this study were brought together to address the research questions.