3. Results
4.5 Evaluation of the Current Research Design
4.5.2 Methodological limitations
4.5.2.3 Measurements
allowed for a large amount of data to be collected at one time, and produced data that can be easily quantified, allowing for more objective analysis compared with other methods of research (Ackroyd & Hughes, 1981; Popper, 1959). All the measures used a Likert scale (Likert, 1932), which is recommended for use with children and
adolescents, over and above other measures such as visual analogue scales, due to being the easiest to complete (van Laerhoven, van der Zaag-Loonen & Derkx, 2007). Additionally, and promisingly, it has been shown that children are able to accurately report on their depressed mood and symptoms from the age of 9, and able to
recognise and identify different emotions (Harter, 1999; Kazdin, 1994). Providing support for this, within this study all questionnaire measures were shown to have high internal reliability and findings were shown to be fairly typical of an adolescent sample, as discussed within section 3.3.
However it is important to consider the shortfalls of self-report measures, such as being sensitive to bias and problems with validity. Indeed the Likert scale has received criticism due to people having different patterns of responding (Clark-Carter, 2010). For example, if two people score the same on an item this does not necessarily reflect true similarity, as one person may be exaggerating their symptoms and the other person may be minimising theirs. Additionally, there is a tendency for
to social desirability bias, simply not knowing how to respond to an answer, or the question not being applicable to the respondent. However, although this has been found to be the case (Garland, 1991), research has shown that this does not significantly affect the validity or reliability of a measure (Kulas, Stachowski & Haynes, 2008). Additionally, self-report measures have been shown to be the most effective form of measuring internal experiences, such as thoughts and emotions, as often these cannot be measured using observable behaviour (Chan, 2009; Korb, 2011).
It is important to consider the impact of such biases, for example the social desirability bias, which arises from people responding in a way that is ‘expected’ of them, as determined by social norms. From a review of research employing a social desirability measure, close to half of the studies reviewed found social desirability influenced their results (van de Mortel, 2008). With regard to the current research, the measures involved are not considered to be particularly socially sensitive, for example the SCS has been shown to have no correlation with social desirability (Neff, 2003b). However, research has found that children scoring highly on a social
desirability scale are less likely to report symptoms of psychological distress such as depression (Logan, Claar & Scharff, 2008). Therefore, given the current study measured for depressive symptoms, it may have been beneficial to include a social desirability measure to detect and control for such bias. Within this study it was felt important not to add to the pre-existing demands of completing the battery of questionnaires, but this may be something to consider for future research.
Further criticisms of self-report measures include the lack of a mechanism to adequately measure and understand emotions, feelings and behaviour, and the difficulty in knowing both how truthful a respondent is and how much thought they have invested into their responses (Ackroyd & Hughes, 1981; Popper, 1959). It may
therefore be useful for future studies to include a multimethod or multirater approach, for example participant or parent interviews, in order to more accurately assess self- compassion, depressive symptoms, rumination and avoidance.
The measures used within this study are discussed in further detail below. Details regarding appropriate use with adolescents are mentioned, however this has been previously discussed in greater detail in section 2.4.
There were some problems identified with the measures selected for this study. For example, although the CES-D has been deemed appropriate for use with
adolescents and has been shown to have good internal consistency both in this study and within the wider literature (Dierker et al., 2001), one of its main limitations is its sensitivity. The CES-D has been shown to lead to a high number of false-positives (Myers & Winters, 2002; Roberts et al., 1991) and the proportion of true cases among those exceeding the cut-off has been shown to be between 10-30% (Garrison et al., 1991; Roberts et al., 1991), suggesting that it is over-sensitive. Indeed this appeared to have been the case within the current study, with almost half the sample (40%)
exceeding the cut-off score for clinical depression on this measure. Similar percentages of adolescents exceeding the cut-off have been shown within a non- clinical community sample (49% amongst 11-15 year olds, 53% amongst 15-18 year olds; Radloff, 1991). Radloff also used a convenience sample and, as with the current study, this may not be representative of the adolescent population. In response to the high percentages Radloff (1991) raised the cut-off to 28, a value designed to be more clinically meaningful, and found significant lowering in the percentage of adolescents exceeding the cut-off (13% amongst aged 11-15 year olds; 18% amongst adolescents aged 15-18 years). Similarly, Garrison and colleagues (1991) found a cut-off point of 12 for males and 22 for females, whilst Roberts and colleagues (1991) found a cut-off
point of 22 for males and 24 for females, to be the most effective to screen for depression. Within the current study 12 (13%) of adolescents exceeded Radloff’s (1991) raised cut-off, which is comparable to the percentage within his study. It may have been useful to have employed a higher cut-off score within this study; especially as parents were informed at the lower cut off and may have become unduly concerned about their children. However, from a risk management point of view it is preferable to have more false positives than more false negatives. Considering the implications of the cut-off score level for the data analyses this was not problematic, as the data collected were continuous and not categorical.
It is also possible that the scores on the CES-D were inflated by transient symptoms arising from influences such as mood swings, school stress, or
interpersonal factors. Indeed, when persistence of symptoms has been measured and controlled for, those who exceed the cut-off have been shown to significantly reduce (e.g. Radloff, 1991; Wells, Klerman & Deykin, 1987). Measuring the persistence of symptoms and other extraneous factors, such as current or recent stressors that the participants may have been exposed to, would be relevant within future research. This would allow a more accurate understanding of depressive symptomatology.
The problems identified with the CES-D probably stem from the fact that it was designed as a measure for epidemiologic research and was not intended for clinical diagnosis. Therefore caution needs to be taken when interpreting the scores, and it may be beneficial for future research to utilise a more comprehensive
diagnostic measure or procedure.
Within the current study the measure of compassion used was the SCS. As shown in Neff and McGehee’s (2010) study, the measure was demonstrated to have high internal consistency, suggesting the scale is appropriate for use with adolescents.
However it needs to be considered that self-compassion is a construct proposed by Neff (2003a), which is based on a Buddhist perspective and not strictly a
psychological perspective. Therefore it may be more clinically relevant to use a scale that is more rooted in psychological theory, such as Gilbert’s model of compassion (Gilbert, 2005) based on attachment and evolutionary perspectives. At present no such scale has been developed for use; the development of such a measure might be
clinically relevant, and warrants further consideration. Additionally, it may be useful to measure self-compassion using a range of different methods. Most studies
researching self-compassion, including the present one, use a cross-sectional survey design and it may be that using experimental and qualitative research designs would broaden and add further dimensions to our understanding of self-compassion.
The RRS was shown to have high internal consistency amongst the current adolescent sample, both for the total and subscale scores, reflecting previous reports within the literature for the reliability of the measure within adolescent samples (e.g. Cox et al., 2012). Additionally, previous studies have reported acceptable convergent and predictive validity for the measure (Butler & Nolen-Hoeksema, 1994; Nolen- Hoeksema & Morrow, 1991). When exploring the main research questions of this study, the brooding and reflection subscales were used (Treynor et al., 2003). Confirmatory factor analytical support has been provided for this two-subscale measure, with appropriate reliability and convergent and discriminant validity for the subscales (Schoofs, Hermans & Raes, 2010). It is likely, therefore, that the findings in the current study more reliably reflect the relationship between repetitive thinking and depression, rather than being an extended measure of depressive symptomatology.
With regard to brooding and reflection, one main concern within this study was the lack of differentiation between the two scales. Indeed, rather than being
adaptive, the reflection scale positively related to depressive symptomatology and negatively related to self-compassion. This finding may be due to the difficulty distinguishing between different types of rumination, particularly when experiencing a depressive mood state (Joorman et al., 2006). For example, those who are depressed and are brooding may also be more likely to score higher on the reflection items. Indeed they may score highly on the reflection item of “analyse recent events to try to understand why you are depressed”, but rather than considering this in an emotionally neutral way they may instead be brooding about the event. It would therefore be beneficial to consider brooding and reflective rumination longitudinally. For example Treynor and colleagues (2003) propose that reflection may initially lead to negative affect, but over time may reduce negative affect, perhaps due to its role in facilitating effective problem solving.
The CBAS has only been used with an adolescent sample once before, and the scale was adapted for this use (Dickson et al., 2012). Within this study it was decided to not adapt the scale, and the internal consistency was shown to be very high, exceeding the internal consistency for Dickson and colleagues’ (2012) adapted scale. The CBAS is beneficial as it assesses the degree of avoidance across cognitive and behavioural domains. However, the CBAS only assesses successful behavioural avoidance (withdrawing from activity) and unsuccessful cognitive avoidance (trying to not think about problems). Therefore, it would be useful to examine unsuccessful behavioural avoidance or successful cognitive avoidance as additional processes involved in the relationship between self-compassion and depression.