Chapter Three
3.2 Mixed Method Research:
3.5.3 Step Three
Step three required the establishment of a small group of teacher volunteers willing to participate in the AR. The AR phases can be visualised in Figure 3.1, shown in the shaded circular section. While there was a possibility of sufficient numbers of teachers volunteering to form two small groups, this did not eventuate. On reflection, this was a positive outcome for this study as the time needed to co‐ ordinate and gather data from both groups would have been unmanageable on my
part. The group who did volunteer participated in AR based around the use of a professional teaching portfolio and the mandatory requirements of the MOE, especially that of observation (Ministry of Education, 1997). The aim behind approach was to develop an appraisal system that was teacher driven with a professional development focus. The portfolios were structured around the NZTC’s RTC which came into effect in 2010. To facilitate this, discussions of how the RTC could be interpreted to reflect teachers’ practices and their place of work were undertaken. Professional discussion is the way the RTC are intended to be utilised (Thomas, 2009) and this became quite a strong focus with group sessions. The discussions led to quite different outcomes for individual teachers in the group as portfolio development strongly reflected each teacher’s personality and individuality and their understandings of the RTC specific to their subject area. While a shared understanding of the RTC themselves was possible, how these were evidenced became quite individualised. The cycles for the AR were conducted over the course of a year. It was anticipated that during the first school term only one or two cycles would be completed. In the remaining three terms, the aim was to have 3‐4 cycles completed in each. This was to ensure a wide variety of actions were explored and evaluated with a variety of data collected. Each cycle was planned to build on the previous one allowing for the embedding of learning alongside the further development of ideas. The researcher’s role in these groups was that of the expert facilitator in helping to provide direction but also as participant, contributing to the process.
The next step was the planning and implementing of a course of action that was based around professional learning and which demonstrated how the RTC were being met. This course of action was to be followed by evaluation of the process to date and planning further steps. Evaluation and planning were conducted through group sessions to enhance professional discussions and provide both support and guidance as necessary. Each teacher included with his or her portfolio a reflective journal. The timelines and meeting schedule for the group was decided in the initial meeting, and although it was envisaged that these would be 3‐4 weekly, this
was not always possible within the busy school term, and times had to be carefully negotiated. Arrangements could not always be made too far in advance due to the repeated occurrence of ‘unexpected’ events.
While the plan was for repeated AR cycles, what eventuated was quite different. The flow chart representation (Figure 3.1) became more accurate than I initially anticipated. The study developed one main AR cycle, facilitated by me, that focused on the appraisal cycle and the processes involved. Within this cycle each participant, including me, undertook his or her own AR based on their professional development needs at the time. My role then became more of an expert for these smaller cycles rather than participant. The over‐arching AR cycle was a process that involved all participants, who brought their understandings and ideas based on outcomes of their smaller, more specific AR cycles. Within this larger cycle, a school‐wide focus of the processes of appraisal and the documentation necessary began to develop.
3.5.4 Step Four
Step four involved final interviews (See Interview Schedule, Appendix 9.7) with each participant who discussed their views on their ownership of the appraisal process and the research process undertaken the previous year. Further future directions of the process, including any necessary modifications to documents and strategies developed, were also established at this point. The principal of the institution was also interviewed to give his views on the appraisal process and provide a possible point of difference to comments made by the teacher volunteers. During this meeting time, a brief review of the research process and outcomes were discussed to provide initial feedback.
Also during this step, the second confirmatory survey (Appendix 9.8) was conducted which gathered information about how perceptions of the whole staff had changed toward performance management in the school. The survey responses also sought future directions for professional development, an area of
need identified in the initial data analysis. This survey helped to establish where the main ‘gaps’ were, particularly with regard to improving teachers’ knowledge of opportunities available for their professional learning and access to these opportunities.
3.5.5 Step Five
Step five was critical for two reasons: first it involved the embedding of changes into the school and further adjustments to the PMS, and second, the final analysis of all data and the writing up of results were undertaken. A final report of the findings was presented to the institution at the completion of this stage of the research programme. This stage also saw the beginning of preparation of a paper with the intention of presenting this at a future education‐focused conference.
3.6
Data Analysis
The analysis of the data needs to be structured in such a way that it answers the research questions. While this may seem a logical statement, it is very easy to have large data sources and be side‐tracked into findings not related to the focus of the research. A systematic approach to analysis helped to maintain the focus of the research and to make the task more manageable.
The surveys were analysed using both ‘themes’ and statistical analysis. Descriptive statistics, used to describe and organise data, were used to summarise choices made by participants and the relationship of those choices to age. Inferential statistics were also used on this dataset to make inferences about the information gathered. As the group surveyed, 100 teachers, is a sample of a larger population some degree of inference can be made to the whole population. The analysis explored frequency of responses to get an indication of potential changes that would benefit the most people. Nominal data highlighted patterns. Ordinal data allowed for the ranking in order of preference of related data, again assisting in the identification of possible changes to the appraisal system (Salkind, 2008).
ANOVA data analysis was used between the datasets to see if there were any statistically significant correlations between the findings. Had this research been conducted over more than one educational site the differences between groups could have been significant. However, this was not possible at this point in time due to the scope of the study. Even though the statistical analysis was only a minor focus of this study, the computerised analysis programme SPSS was utilised for the interpretation and ordering of the data. The qualitative data was much more time consuming and difficult to manage given the nature and quantity of data collected. With AR, making sense of the data is an on‐going process as reflection on the data forms part of the cycle of the research process (Cresswell, 2002). The analysis of qualitative data was carried out on a regular basis as patterns and themes began to emerge. This progressive analysis helped to alter the overall structure of the study from its initial plan of numerous AR cycles, changing to one main AR cycle with individually focused AR cycles under this umbrella. Writing memos and notes while coding was helpful and made checking information later on much more straightforward (Gratton & Jones, 2003). It was the patterns that occurred that were of importance, and descriptive narratives have been used to interpret these within the school’s context. This process, done manually, is time consuming. An alternative is the use of a computerised programme such as NVivo, as this allows the researcher to do all the manual manipulation with much more speed and potential accuracy. However, after some training on this programme and its initial use in analysing results, I returned to a manual method, arranging folders within Microsoft Word for each of the themes. Into these folders I not only put the relevant sections of transcripts from group sessions and interviews, but also quotes from literature to support each theme. While this may seem time consuming, I found that I had a much better understanding of the data collected and could begin organising the folders at an early stage in the analysis process. The use of the printed version of each theme and highlighters were invaluable in making connections across themes and ideas. I am aware that NViVo allows for this within its programme, but for me it was more
time expedient to use a method I knew well, and that seemed to work for me, rather than spending time I did not have trying to learn the intricacies of a programme that would add little value to the processes I was currently employing.