Data collection methods

A diverse range of data collection methods were used to address the research questions including: (1) a series of in-depth, one-to-one interviews (baseline, 6-, and 12-month follow-up schedules); (2) self-report questionnaires; and a classroom observation tool. The materials used in this study are detailed in the following chapter subsections.

6.5.3.1 Interview schedules

The researcher developed a series of interview schedules (see Appendices 1 to 6), for teachers, principals, programme facilitators, and the development manager respectively. Interview content was informed by a review of the literature and by the research questions outlined earlier in chapter 1. With regard to the teacher participants, baseline interviews related to current management practices, prior knowledge of the TCM programme, and expectations surrounding training participation. Initial open-ended questions enquired about challenges in classroom

Principal ID Gender School DEIS band

P1 Male Non-DEIS

P2 Female DEIS Band 1

P3 Male DEIS Band 1

P4 Female DEIS Band 1

P5 Male DEIS Band 2

management and perceptions of support. Where necessary, prompts were given and based upon participant response the order of questions was altered. Follow-up interview schedules related to teacher experiences of undergoing TCM training (at 6- month follow-up), and their perceptions regarding post-intervention classroom management (at both 6- and 12-month follow-up).

The Principal Interview Schedule was constructed to measure broader issues relating to behaviour management within schools, and to contextualise the experiences of teachers. This included questions relating to the types of behavioural problems typically encountered, their school’s ‘code of behaviour’, the perceived cause of difficult classroom behaviour, and prior knowledge of the TCM programme.

Facilitator interviews, conducted post-intervention, addressed experiences with TCM delivery, and perceptions regarding the influence of group-dynamic and extraneous factors on the training process. The ways in which facilitators dealt with emergent problems, and the extent to which such issues affected their ability to remain within TCM protocol, were also examined. The interview with the Archways development manager addressed issues pertaining to school and teacher recruitment/retention. The process, challenges, and successes of this activity were discussed, along with issues relating to the NUIM-Archways partnership. Such community-university collaboration involves an active bridging of ‘town’ and ‘gown’, a beneficial (although often difficult) partnership noted by McWilliams, Desai, and Greig (1997). Suarez- Balcazar, Harper, and Lewis (2005) proposed that such collaborations involve a range of interrelated benefits and challenges, which can manifest from both community and university perspectives (Norris et al., 2007; Ross, 2010).

6.5.3.2 Quantitative measures

All intervention group participants were invited to complete self-report questionnaires in order to elicit experiences of workshop participation (one questionnaire per individual workshop) and overall post-training programme satisfaction. Structured classroom observations were also conducted to ascertain the classroom dynamic, pre- and post-TCM training. Each of these is described briefly in the following subsection.

Teacher Workshop Evaluation Questionnaire

The Teacher Workshop Evaluation Questionnaire (TWEQ), developed by The Parenting Clinic13, is a self-report measure, completed by teachers on a monthly basis after each of the five training sessions. This questionnaire addresses teacher experience of individual TCM workshops (see Appendix 9). It consists of four open- ended questions to identify aspects of the workshop that were liked most and least, the most helpful component, and recommendations for workshop improvement. It also contains six 4-point Likert scale items ranging from 1 (very poor/unhelpful) to 4 (above average/very helpful), which evaluate the workshop leaders and group interaction (3 items), and the training strategies used (3 items). The extent to which the workshop evaluation is favourable is indicated by the mean score of these six items.

Teacher Satisfaction Questionnaire (revised)

The Teacher Satisfaction Questionnaire (TSQ) also developed by The Parenting Clinic, was abbreviated by the researcher for use in the current study. The original 37-item measure displayed overlap with the TWEQ (outlined above), and was considered to be excessive in length, thus it was revised for use in the current study. The revised version comprises five 5-point Likert scale items ranging from 1 (a negative response such as ‘not at all’) to 5 (a positive response such as ‘very much so’) and 4 open-ended questions. This questionnaire contains items regarding participant satisfaction with training, expectations prior to TCM participation, perceived appropriateness of the training programme, whether the participant would recommend the training programme to others, post-training classroom management confidence, and overall impression of the programme. The satisfaction score is calculated by obtaining the mean score from these items with the exception of item 1,

13

The University of Washington’s Parenting Clinic develops, delivers, and evaluates programmes to promote social competence and prevent conduct problems in children. Co- directed by Dr. Carolyn Webster-Stratton, (developer of Incredible Years curricula), one of the aims of the Parenting Clinic is to provide publicly-accessible measures for programme evaluation. The TWE and TSQ are available at: http://www.son.washington.edu/centers/ parenting-clinic/documents/Teacherworkshopeval.pdf

which addresses expectations prior to TCM training. In the current study, the internal consistency of this revised measure was .942.

Classroom observations: The Teacher-Pupil Observation Tool

The T-POT (Martin, 2005) was used in this study to assess teacher behaviour14 in the classroom (Appendix 7). This tool was developed at Bangor University and is based upon contemporary teaching and child behaviour research. It incorporates elements of the Dyadic Parent Child Interaction Coding System (DPICS; Robinson & Eyberg, 1982) and the Multiple Option Observation System for Experimental Studies (MOOSES; Tapp, Wehby, & Ellis, 1995). The T-POT has been shown to have promising reliability (inter-rater reliability = 0.78) and validity (Martin et al., 2010). Its ease of administration within a busy classroom made it ideal for use within the current study.

Several possible teacher and child behaviours relating to both verbal and non-verbal communication are coded on an itemised checklist. The focus in the current study is on ‘Teacher Positives’, ‘Teacher Negatives’, and ‘Teacher Praise’. The category ‘Teacher Positives’ includes such behaviours as ‘positive affect’ (e.g., smiling at a child), ‘physical positive’ (e.g., touching a child in a positive manner), and ‘encouragement’ (for example, where a teacher tells a child “I know that you can do a good job”). The category ‘Teacher Praise’ comprises both labelled and unlabelled praise. Both are regarded as favourable behaviours; because labelled praise is specific (e.g., “I like how you are taking your books out of your bag”) it is preferable to unlabelled praise (e.g., “Good job!). The categories ‘Teacher Positives’ and ‘Teacher Praise’ may be contrasted with ‘Teacher Negatives’, which includes ‘negative commands’ (e.g., “Stop” or “Don’t do that”), ‘physical negatives’ (e.g., physically restraining a child), ‘criticisms’ (e.g., “You’ve been very naughty”), and ‘warnings’ (e.g., “If you don’t sit down, you’ll be sent to the naughty corner”). T-POT scores are comprised of frequency counts for all relevant behaviours. These are summed to give

14 _{Although not a focus in this process evaluation sub-study, child classroom behaviour was observed}

alongside teacher behaviour, as part of the larger RCT - see McGilloway et al. (2011) for further details and for RCT findings pertaining to child classroom behaviour.

an overall score for teacher and child (positive and negative) behaviours. As several different researchers typically code simultaneously, it is important to ensure consistently high inter-rater reliability- this will be discussed in more detail later in this chapter.