Use of Methods
4.4.4 The Lesson Observations and Feedback
The lesson observations were another central source for my evidence, and were considered to be of equal significance to the interviews. The importance of observations has been highlighted by Ernest (1989); it has been shown that there can be ‘a great disparity between a teacher's espoused and enacted models of teaching mathematics’ (1989: 2).
Depending on differing circumstances, the individual participants were observed a varying number of times. The record of observed lessons is shown in Table 4.6 below.
Lessons observed (LO) in 2013-2014:
Lessons observed(LO) in 2014-2015:
Lessons observed (LO) in 2015-2016:
Lessons observed (LO) in 2016-2017:
Harvey
LO2:
Table 4.6 Record of Observed Lessons
The initial intent was to simply observe lessons, with reference to the
‘Framework to identify effective mathematics teaching’ (discussed in Chapter 3), to see whether any differences in subject knowledge, behaviours, attitude or efficacy could be detected over time, and whether any perceived changes appeared to correlate with the teachers’ espoused views.
In reality, the need to master the crafts of lesson observations and mentoring, and to be ethically responsible soon became apparent. Using the framework – and relying only on the high inference judgments – appeared not wholly
appropriate. Opting to use the critical events technique, promoted by Wragg (2012) - with the idea that the observer looks for ‘specific instances of
classroom behavior which are judged to be instances of some salient aspects of
the teacher’s style or strategies’ (2012: 64) - a narrative of the lesson was recorded. Aiming to stay as neutral as possible, I documented what preceded an event, what happened and what came next.
Electing to be a non-participant observer, my decision was driven by the desire to capture the essence of the lesson by recording as much input, from both teachers and students, as possible. This approach was commented upon by Katy:
‘The observation itself was slightly different [to previous experiences of being observed] in that you didn’t get involved with the student’s working but rather observed what was going on in the room. I think this yielded different, more detailed information for feedback and gave us specific points to discuss.’
(Feedback-on-feedback1 (from LO3))
On occasions I did blend the participant and non-participant approaches to observation and moved around the classroom to view students work in an attempt to see whether the student responses mirrored the teachers’
expectations - and to capture on camera examples of student work.
The actual observation documents were written as a non-judgmental
commentary of what occurred in the lessons. These were used as a stimulus to promote reflections and informal discussion immediately following a lesson.
Later each written, real time commentary, was polished and sections were added commenting on the ‘strengths’ and ‘limitations’ of the lesson with reference to the ‘Framework to identify effective mathematics teaching’
(Appendix N); more detail regarding this process is discussed later.
It is common for teachers to feel some trepidation toward being observed and I wanted to minimize anxiety. I wanted the experience to feel, and be, worthwhile.
The purpose of the observations was therefore made explicitly clear to the participants from the outset: this is an emerging study to explore the impact, or otherwise, of this retraining programme on teacher subject knowledge,
attitudes, beliefs and practice over a period of time. Wragg (2012) describes the importance of having won ‘the confidence and respect of those observed’
(Wragg 2012: 142) and the relationships forged through the retraining face-to-face sessions and the preceding interviews (described in detail above) helped enormously in this regard. Teachers welcomed me into their classrooms and appeared to trust and value my presence and the feedback offered – as
documented by the overwhelmingly positive comments to myself and to Hedger, the independent evaluator. Relationships based on openness, trust, and
honesty were carefully nurtured.
An observer in a classroom will inevitably create impact (Wragg 2012): the dynamics within the classroom and the lesson itself will be altered and this presents further ethical considerations. To what extent a lesson is distorted is impossible to ascertain, but these considerations could at least be explored during the post lesson feedback sessions. And in an attempt to limit my intrusive impact, I determined a typical routine for lesson observations, as shown in Table 4.7.
• Arrive before the class to find a discreet place to sit in the classroom.
• Teacher to decide how, and if, they choose to introduce me.
• Limit interaction with the students; respond only if they are curious.
• Reiterate to teacher that the scribbling of copious notes, throughout the lesson, is part of the process in an effort to record as much of the lesson as possible, focusing on ‘critical events’ (see above).
• Offer verbal constructive feedback (only if it appears the teacher would welcome it) immediately after lesson.
Table 4.7 Illustration of Lesson Observation Etiquette
A danger of the observer-observee relationship is that the observer can be cast in the more authoritative role; participants can be ‘inescapably locked into a superior- subordinate relationship’ (Wragg 2012: 59). I was keen to downplay this potential dynamic. In an attempt to both diminish this scenario and be sensitive to the observed teachers’ receptivity, I was overly grateful (for their time, and the opportunity to observe); I emphasized the positives of the lesson and tended towards flattery in order to reward and thank the teacher.
Other ethical issues arose when it became obvious teachers sought instant verbal feedback. There are issues with providing feedback, not least that there is no absolute certainty regarding what constitutes teaching effectiveness (this, discussed in detail in Chapter 2). Reflecting on the nature of effectiveness in terms of teaching, has been a recurring and constant theme throughout the entire research project and recognizing there are often several opinions about quality, not (of course!) just one’s own, was of particular relevance in terms of lesson feedback. Other reservations in providing feedback concerned the dangers of appearing judgmental with potential negative consequences for the teachers who had, after all, voluntarily chosen to take part in this research and give up their own free time. These ethical implications certainly suggested it could be ‘easy to settle for saying little’ (Wragg 2012: 62). And yet teachers (as perhaps all performers) anticipate some feedback and some form of evaluation;
as Wragg (2012) says: ‘Anyone teaching new subject matter or breaking fresh ground, may be desperate to know what they can do to improve their
professional skill’ (2012: 63). Wragg (2012) also suggests that if lessons are
‘worth observing then they are also worth analyzing properly’, and that ‘little purpose is served if, after a lesson, observers simply exude goodwill, mumble
vaguely or appear to be uncertain why they are there, or what they should talk about’ (2012: 2).
Following the initial lesson observations, and building on the established rapport, the decision was made to offer each case study teacher individual themes on which to focus. The second set of observations, shortly after the completion of the course, were conducted as developmental lesson
observations focusing on these themes. With a focus on development, I was aware that the stakes could potentially feel higher for the teachers. Acutely aware that lesson observations can create anxiety for many teachers
(discussed in Chapter 2), I attempted to preempt any possible concerns by way of an email missive, conveying the spirit of lesson observation as envisaged by Wragg (2012):
‘…if teaching is to develop to the point where it can display its talents with pride and its frailties without fear, a tall order in a climate where teachers have often been criticised unfairly and blamed for society's ills, then positive steps must be taken...to make lesson observation a high priority, and that may mean ingenuity, and capitalising on what is available like the presence of student teachers and their tutors.’ (2012: 131)
Informal developmental verbal feedback was once again offered; and again the participants reported (in interviews, questionnaires and dialogue) on the value of having ongoing support and advice. Being observed, without the spectre of
‘performance management’ or Ofsted, was considered as very liberating.
Although this informal feedback was appreciated, I felt lacking of a firm framework for formal feedback to properly and rigorously address the many misconceptions and limitations being observed within the lessons. I was ‘feeling’
my way in terms of feedback, as I didn’t want to risk damaging the developing relationships; ultimately without my participants I had no project. However in the spirit of ‘capitalising on what is available’ (Wragg 2012) - and with the support
from my supervisor and consent from my participants - I decided to experiment with a more robust form of feedback. Exploring this emerging theory, that more robust developmental lesson observation and feedback is needed for newly retrained teachers, became the next stage of my study.
The lesson observation document continued to be written in real time, and then used as a vehicle for discussion immediately following the lesson. At this
juncture, the strengths and limitations of the lesson were discussed and co-constructed by the teacher and myself, with reference to the high inference items from the ‘Framework of effective mathematics teaching’ (see Appendix N). The teacher was always invited to reflect and feedback first, reflecting the ethos of the Japanese Lesson Study model (APEC 2013).
These focused mentoring and coaching sessions became a key feature of the lesson observation process. The difference between mentoring and coaching can be subtle and I use them virtually synonymously (and there is more
description of these in Chapter 2) as a continuation to the support given during the retraining programme. Mutual trust and respect were essential, and
comments from the participants (outlined in Table 4.8) highlight the significance of these considerations.
Anna: . I feel part of the process and not judged…It allowed me time to reflect on my lesson and the discussion with Naomi was along the ‘coaching’ type and therefore I was able to notice areas to improve myself with her guidance. (Feedback-on-feedback 1)
Bea: . I value your opinion - as lecturer/academic - more than that of HT (Interview 3)
Cath: . With you…it is non-judgmental…I don’t know how others are managing without this [feedback and mentoring] (Interview 3)
Darcy: . I really didn’t enjoy that; I found him very harsh [supervisor visit to
triangulate results]. It made you seem lovely …(Interview 3)
Euan: . It was forward thinking and constructive whilst also being an honest appraisal of the teaching seen. (Feedback-on-feedback 1)
Harvey: . There are not many opportunities to get feedback that is non-judgmental and this is an opportunity to get [that] feedback; The non-judgmental side is very powerful to me (discussion following Lesson Observation 6) Janet: . It is important to be observed by someone who is a good teacher. The
observer needs to be credible. I’ve had some observers… where I think:
Like to see you do this!! (discussion following Lesson Observation 1) Katy: . Thank YOU! I wish you could come more often. That really was one of
the most useful pedagogic discussions I've been able to have in years! I wish lesson observations were more like that (constructive). This is the best observational process - it is developmental and non-judgmental (email: 15/102015)
Table 4.8 Comments from Participants Highlighting Mutual Trust and Respect
Within a few days of each lesson observation, the participants now received the polished written report, identifying all the strengths of the lesson and with all the observed limitations and misconceptions clearly addressed (see Appendix G for an exemplar lesson observation document). Photographs were incorporated within this document, providing visual cues to the lesson, including board work, pages from students exercise books, activities and exercises. The teachers were always given a right to reply by way of being invited to respond and to edit.
None ever did. Two weeks after each observation (when the teacher had had a chance to digest, absorb and reflect on the feedback) a ‘feedback-on-feedback’
form was sent to the teacher, to explore their thoughts and reactions to this more robust feedback process (see Appendix H). These responses added to the data.
I conducted over 50 lesson observations with each of the eight participants experiencing somewhere between 5 and 9 observations. Some other planned observations were cancelled at short notice by the teachers and the reasons for this varied: tests or examinations, scheduled after the appointment arranged, were considered by the teacher unworthy to observe; sometimes a teacher
reconsidered and cancelled if they subsequently felt the observation had, after all, been arranged with too short notice; one teacher felt that, at times, there was just ‘too much on’; family issues impacted on one teacher; illness and absenteeism; and one case study teacher, in his late 40’s, simply stopped teaching - resigning from the profession after 14 years of teaching. During the months of May and June, it was particularly difficult to arrange visits as teachers were very focused on the upcoming examinations and associated revision;
perhaps also suggesting teachers felt these lessons unworthy of observation.
All of this may suggest that even though great efforts were made to put
teachers at ease and to minimize the intrusive nature of the observation, for the teacher an observation may still be a matter of some significance or stress - and that a heightened level of performativity is required - only to be embraced when conditions are favourable. Alternatively it could be, as some teachers
commented, that they didn’t want to ‘waste’ an opportunity for feedback and for mentoring when they were not set to gain the most.
To summarise, from 2015 to 2017 participants received (with their consent) rigorous verbal and written feedback highlighting developmental aspects. My role as observer therefore evolved into one of mentor and coach. Exploring this more definitive support role, and considering how this affected teachers both personally and professionally, became another emerging theme from my research.
In addition to capturing feedback, by way of ‘feedback-on-feedback forms’, various other questionnaires were employed throughout the study; these are considered next.
4.4.5 Questionnaires
The questionnaires employed throughout this study were used mainly to service the major instruments of interviewing and observation. All teachers, enrolled on the Plymouth SKE+ course, completed the standard Mathematics Attitude Questionnaire (Appendix I) and Subject Questionnaire for Mathematics (Appendix J) both pre and post the retraining. These questionnaires explored their confidence in subject knowledge as well as their views on the teaching of mathematics and asked teachers to gauge their expertise level (from 1 to 5) on 20 different topics of mathematics; these questionnaires were repeated at the end of the course so the teachers’ own perceptions could be compared. The questionnaires also ascertained a few basic teacher beliefs, which provided an ideal entry point for the first interviews with the research participants and a direction in which to probe much deeper into their beliefs, ideas and
motivations. Other questionnaires were designed throughout the course of the study for theoretical sampling, to triangulate, to capture shifts in teacher
perceptions over time, and to prime teachers prior to an ensuing interview.
These questionnaires, influenced by Garet et al (2001) and Boyle et al (2005), are included in Appendices K, L and M.
Along with these questionnaires, and the lesson observations and interviews previously discussed, reflections, thoughts, observations and ideas were all noted and explored by way of memos; these a key tool for grounded theory methodologists.
4.4.6 Memos
Memos or note-writing are a tool to keep track of ideas, thoughts, connections, queries, hunches and sense-making (Creswell 2012, Watt 2007). Used
throughout the entire research process, memos can help shape emerging ideas and ‘prevent paralysis from mountains of data’ (Creswell 2012: 439).
Throughout the data collection cycles and throughout the whole research study, I was conscious of being as reflexive as possible. Reflexivity, a concept that comes from anthropology, is a process of reflecting on oneself as the
researcher, to consciously acknowledge and examine experiences,
assumptions and preconceptions which preempt the research and that therefore inevitably influence the process and outcome. By thinking reflexively, my
intention was to reduce the risk of being misled by my own assumptions. And any insights or underlying (and revealed) motivation and purpose for the study, can be used to add depth and clarity to the research. Renshaw (2009) refers to this reflexivity as the quality of a person’s ‘inner listening, attention and
awareness’ (2009: 6). Learning to be reflexive, and to reflect on one’s thoughts and behaviours, ‘creates a means for continuously becoming a better
researcher’ (Watt 2007: 82). Becoming a better researcher, whilst
simultaneously aiming to contribute to the field in question, can be considered core aims for this PhD study.
Experience as a teacher and educator, and also as a mentor and manager – will have undoubtedly coloured the lens through which I see. Two decades ago, as a young and relatively inexperienced new head of a mathematics
department, I inherited teachers of mathematics who were non-specialists. This experience is still clearly embedded in my memory and will influence my
thoughts, ideas and interpretations. As too, will be the constant struggle to recruit effective teachers to a rural comprehensive, questioning even then what
it was we really wanted (and needed) teachers of mathematics to be able to do.
This study deliberately focuses on one retraining scenario – that of teachers, taught by me under the umbrella of Plymouth University. Nevertheless, visiting and observing other retraining models (for example, in Newcastle), funded similarly by the government, has helped me reflect on my own practice and to think more widely about the process of retraining. Observing and talking to mathematics teachers from other countries and examining other education systems and aspirations, has afforded me the opportunity to step outside the box of England’s education system, with its shortage of mathematics
specialists, and look from the outside, in – and to explore assumptions, expectations and cultural differences. Meetings, arranged with other training specialists – for example with Murphy from TAM (Teaching A level
Mathematics) – have also helped hone and examine my thoughts and emerging themes.
In the style of an online journal, I kept eclectic memos in one document
spanning from early 2013 until the end of 2017. I started writing these memos without knowing what to write or think, taking inspiration from Laurel Richardson (2010) and her article: Getting Personal: Writing stories. For Richardson,
‘Writing is a method of discovery…I write because I want to find something out.
I write in order to learn something that I did not know before I wrote it’(2010:
35). Embracing this sentiment, I used memos to help to me think and understand reflexively. Richardson (2010) recognises that we are in our research; by engaging with internal dialogue and documenting it by way of memos, the ‘process of knowledge construction’ can be revealed (Watt 2007:
84). Charmaz and Belgrave (2012) echo this sentiment, believing it is the writing of memos which ‘links coding to writing the first draft of the analyses’ and that
memos keep interpretations close to data and ‘avoids forcing the data into extant theories’ (Charmaz and Belgrave 2012: 357).
In addition to these writings, I also compiled extensive field notes with numerous notebooks taken into the field to capture incidents and events.