Reliability and Validity

The British Psychological Society (BPS) defines a psychological test or assessment as “any procedure on the basis of which inferences are made concerning a person’s capacity, propensity or liability to act, react, experience, or to structure or order thought or behaviour in particular ways”. According to the BPS and the Standards for Educational and Psychological Tests (AERA, APA, and NCME, 1999), psychological assessments (or psychometrics) need to strive to achieve a number of standards in their construction, evaluation, and supporting documentation. In addition to this, professionals using psychological assessments need to consider fairness in testing, and the responsibilities that testing involves. Assessments need to demonstrate that they are both reliable and valid: that their scores are as accurate and precise as they can be whilst taking into account degrees of error, and that their scores are a meaningful measurement of what the assessment has set out to measure whilst taking in account contextual factors.

The CRI-Y dedicates a number of pages in its professional manual in order to detail its psychometric characteristics and how the test’s development has been tailored to achieve these. A recently updated annotated bibliography summarises research conducted using the Coping Responses Inventory (both youth and adult forms). This includes research conducted by the authors themselves as well as that conducted by other researchers, which consists of published studies and unpublished Doctoral dissertations or theses. The CRI-Y and adapted versions has been applied in research across cultures, including Australian (Eyles & Bates, 2005), Iranian (Aguilar-Vafaie,

171

youth. Findings from these studies, particularly those concerning the psychometric properties of the CRI-Y, provide further questions and suggestions for further validation. This is discussed in more detail below.

The Cronbach’s alpha coefficient is considered by some to be the best index of reliability in terms of internal consistency (Kline, 1999). Internal consistency is a measurement based on correlations between scale items, and measures the extent to which items assess the same construct: in this case, coping styles. The internal consistency estimate of reliability of each scale in the CRI-Y was assessed using Cronbach’s alpha, and was found in the majority of cases to be “moderate” (Moos, 1993). This was measured across the first two Waves of field trials during the development of the assessment. A Cronbach’s alpha value of 0.7 or above is considered to be necessary for testing with individuals, and anything below is considered to be questionable or poor (Kline, 1999). Internal consistencies for only four scales out of eight have a coefficient value of 0.7 or above. This is highlighted in the professional manual. It is important to remember that internal consistency and reliability can be affected by context and the construct that they are measuring. Moos (1993) tries to justify the moderate internal consistency for some scale items by hypothesising that some coping responses may reduce stress, and therefore reduce the use of alternative coping strategies from within the same category of responses (i.e. approach or avoidant coping responses). This is thought by the author to place an upper limit on the internal consistencies of the CRI-Y scales. Psychometric qualities in terms of reliability and factorial validity were again assessed by Hamdan-Mansour et al. (2008). This study recruited a large number of adolescents (248 males and 376 females), which is a larger sample

172

The sample used in this study, however, was recruited from rural high schools in the US, and does not encompass several different populations like Ebata and Moos (1991) and Moos (1993) in their research and tests concerning psychometric properties. The eight subscales of the CRI-Y were found to have moderate reliability and consistency (ranging from .47 to .70). Other studies, including Forns et al. (2005) with Spanish adolescents, replicate these results using factor analysis and find similar ranges of reliability coefficients. Moos (1993) reports that the wider domains of the CRI-Y, including approach and avoidance (.81 and .87 respectively) and cognitive and behavioural (.84 and .91 respectively) domains, were noted as showing good reliability coefficients. It was also noted that scores on the approach coping styles domain were positively related to the adolescents’ perceived levels of social support, and that scores on the avoidance coping styles domain were positively related to the adolescents’ use of alcohol and their experiences of depressive disorders. This finding echoes results found in studies exploring coping styles in adult samples using the adult form of the Coping Responses Inventory (Moos, 1990) (Avants, Warburton &

Margolin, 2001; Billings & Moos, 1981; Evans & Dunn, 1995), and hints at least moderate predictive validity. Some studies finding support via factor analysis for the Moos (1993) theoretical components of approach and avoidant coping response factors (e.g.

Eyles & Bates, 2005; Forns et al., 2005; Griffith, Dubow & Ippolito, 2000), however in some cases the distribution of specific coping scales did not fit the original CRI-Y model (Zanini et al., 2010).

The test-retest correlations and reliability of the CRI-Y were assessed using 254 out of the 315 adolescents who were recruited during the first wave of the field trial for the assessment, and are reported in the professional manual (Moos, 1993). Subscale scores

173

(Burgess & Haaga, 1998; Moos, 1993). The average correlation between the eight scales was rs = 0.29 for males and rs = 0.34 for female adolescents, values which are considered to be moderate to low. However, although the interval of time between applications of the assessment was beyond the minimum three month period recommended (Kline, 1999), it is possible that results may have been altered due to the fact that it was administered to adolescents.

Childhood and adolescence are noted to be periods of rapid developmental and cognitive change, as well as a period of changes in life context (Smith, Cowie & Blades, 2003; Carr, 2006).

Adolescents also vary in their rates of development into adulthood and in the way that they adapt to the stresses and challenges of this (Ebata & Moos, 1991). In addition to this, reporting in the CRI-Y professional manual does not detail whether or not the focal stressor described by the adolescents was the same when the test was re-administered. Coping styles may vary depending on the source of the stressor, or if the stressor develops or changes.

Therefore interpretation of these values is difficult. The stability of the coping responses that were identified in adolescents is noted to be lower than that identified in healthy and depressed adults over a similar timeframe (Holahan & Moos, 1987; Moos, 1993), perhaps reflecting adolescent cognitive and developmental change. Other studies have highlighted that CRI-Y responses are stable over time at a group level, but with notable within-subject differences between males and females (Kirchner et al., 2010). It is also important to consider conceptual context and the inherent variability in coping with a continuous stressor. High test-retest reliability could be seen as a contradiction to this (Schwarzer &

Schwarzer, 1996).

174

CRI-Y, and describes how the authors feel that face and content validity has been achieved. In order to be retained for inclusion in the final version of the CRI-Y, items needed to have several qualities. In addition to having good face and content validity, items required a varied response distribution (i.e. items on which each point of the four-point scale were selected by different adolescents), a moderate level of internal consistency, and a minimisation of dimension overlap. This was obtained by combining dimensions that were conceptually similar. Due to these criteria, outlined in the professional manual, two initial coping dimensions or scales (Affective Regulation and Behavioural Withdrawal) were excluded, and the remaining eight retained to compose the final version of the CRI-Y, each consisting of six items. These coping dimensions are described above, in Table One. Inter-item correlations are not reported. However, inter-scale correlations are reported for both male and female adolescents, and are controlled for the type, severity, and appraisal of the focal stressor (Moos, 1993).

Correlations between scales are low to moderate. The average correlation between approach and avoidance coping dimensions are r = .31 for male and r = .26 for female adolescents assessed. The inter-scale correlations between some scales, particularly between Logical Analysis and Problem Solving (r = .61), are rather high. This could indicate that there is possibly a different structure of dimensions or factors, and that some of the coping scales should be combined or excluded from the assessment. These correlations are highlighted in the CRI-Y professional manual by Moos (1993), who justify the retained eight-dimension factor structure by highlighting the benefits of differentiating between cognitive (Logical Analysis) and behavioural (Problem Solving) coping styles. Moos (1993) also highlights that coping styles are likely to differ among groups of adolescents, using differences in inter-scale correlations between

175

plausible defences; however it is worth noting that several independent studies have failed to replicate the coping dimensions outlined in the CRI-Y (for example, Zanini et al., 2010).