Despite the positive findings of the study a number of limitations exist which, at least in part suggest caution in the interpretation of the results. Perhaps most obviously the present study lacks a clear comparison or control group. In order to fully establish firm conclusions from the results, particularly with reference to reflective self-functioning, it would be necessary to interview a non-clinical sample of adolescents, matched with the present sample on such attributes as age, gender, and socio-economic status. Within the time and resource constraints of the present study this was not possible, given the length of time needed to interview and transcribe AAI's. However, this does not limit the usefulness of the present study in terms of providing further important information on attachment patterns as measured by the AAI in a sample of in-patient adolescents. Although data does exist for non-clinical samples of adolescents (Kobak & Sceery, 1988), there are a number of problems with making direct comparisons. Firstly with respect to demographic variables, the age range in these studies has not included those adolescents as young as 13. As the present study suggests there may be particular effects on attachment distributions, and in particular capacity for mentalising dependent on the age of the sample used. Thus it would be crucial to examine patterns of mentalising capacity in non-clinical samples of adolescents, to further our understanding of normative changes in the capacity for mentalising as the young person moves through this developmental phase. In addition existing normative data was derived from populations in the USA and Canada, indicating the need for investigation of non-clinical British samples. A further methodological issue related to choosing an appropriate control group would seem to relate to the impact of abuse and trauma on the capacity for self-reflection. It may be argued, that for a valid comparison of this construct between clinical and non-clinical samples, one would need a control group of adolescents who had experienced trauma of some kind, but who did not present with any psychopathology. Clearly this would be an extremely difficult group to identify.
Further methodological issues relate to the way in which participants were selected for the study. Given the relatively small sample pool, i.e. adolescents resident on tertiary care in-patient units, the sample had to be drawn from five different sites. Clearly an assumption was made about the shared nature of the therapeutic approach adopted by each unit. However, there appears to be some slight differences in the context and nature of each unit, which may account for some of the variance in the sample. For example the Cassel hospital adolescent unit provides a therapeutic community approach to treating adolescents, exposing them to the influence of the adult population in the hospital. In a contrast Brookside Young Person Unit, accepts referrals for 13 to 18 year olds, to a unit exclusively for young people. Related to the selection process, there may be implicit gender selection factors such that the pool of possible participants is unusual. Thus on all the units studied there were clearly greater numbers of female than male residents. This may reflect differential expression of distress between male and females, such that females present their distress in such a manner that fulfils the criteria for placement at an in-patient unit for emotional and behavioural problems. In contrast it may be that their male counterparts are more likely to manifest distress in ways that will lead them into the judicial system, and placement at young offenders units i.e. delinquency and vandalism. Thus it may be that the clinical population resident on the units studied does not fully reflect the differential ways in which male and female adolescents manifest their distress. The implications of this for future research is discussed below.
An objection frequently raised with regards to the Adult Attachment Interview is the apparent differential effects of psychopathology on response during the interview itself. It has been argued that a disorder such as depression with it's concomitant effects on memory and attention may confound the classification given to an individuals AAI. However, this argument is only relevant if one thinks of the AAI as a means of assessing actual biographical memories. The AAI does not make claims to perform such a task, but rather provides a narrative which reflects the individual’s
'state of mind' at that particular moment. The implications of this for the measurement of outcome will be discussed below.
As indicated above a further possible limitation related to the Adult Attachment Interview exists in using the Reflective Self-Functioning scale with a population of adolescents. Thus the developmental stage of the adolescent, particularly in terms of cognitive acquisitions may be intimately linked to apparent differential capacities in terms of mentalising. Although it is acknowledged that children beyond the age of 12 may acquire the ability for formal operational thought, or the capacity for abstraction and hypothetical thinking (Piaget, 1972), subsequent studies have demonstrated that many younger adolescents do poorly on standard tests assessing these abilities (Neimark, 1975). This suggests that mentalising skills in younger adolescents may not be as well developed as in older teenagers. This may have implications for the usefulness of the Reflective Self-Functioning scale with this sample. However, as the interview extracts demonstrate, some younger adolescents may show good levels of mentalising, further highlighting the need for normative studies employing the Reflective Self-Functioning Scale in non-clinical adolescent samples.
In the original conception of the current study, the author had planned to conduct the AAI along with relevant outcome measures, and follow-up subjects after a period of in-patient therapeutic treatment. This would have provided extremely useful data regarding the impact of in-patient treatment on adolescent capacity for self-reflection, and it’s implications for improvement (discussed below). As part of the battery of measures selected to access information on outcome it was intended that the Child Behaviour Checklist (Achenbach, 1991) would be used to gain an objective measure from each adolescent’s primary nurse, about their respective behaviour patterns. Clearly this would have been a useful addition in the current study. However, when this was presented to respective units as a part of the research, it was often difficult to ensure that checklists were completed near enough to the time
of the interview for valid associations to be made. Thus unfortunately it was not possible to a valuable objective measure to complement self-report measures.
With respect to self-report measures used in the study, all were limited in their capacity to discern differences between various sub-groups in the data. Although the BDI and SCL-90 have been validated in such populations their effectiveness may more appropriately lie in distinguishing psychological distress between clinical and non-clinical populations, not just within clinical samples. In particular the present study represents the first to use the revised IIP-32 in the context of a clinical sample of adolescents. Although the IIP-32 has been shown to correspond highly in terms of sensitivity to it’s original counterpart, it may lack the specificity with a sample of young people with diverse psychological and social problems.
Related to this, the analysis of that particular set of results concerned with self-report measure subscales involved multiple comparison t-tests in order to investigate the relationship between symptom reporting and other attachment-related constructs. In order to mediate against the potential effects of Type I error, i.e. finding significant results that may have occurred by chance, Dunn’s t (bonferroni) correction was employed to provide a more conservative cut-off point of significance. However, this may have limited the power of the comparison tests used, and although we can be more certain that those results found were actually significant, it highlights the potential limitations of carrying out large numbers of multiple comparisons. Having noted this limitation, the problem of Type I error only applied to this particular set of data and does not affect the other central findings of the study.
As suggested above there were difficulties associated with assignment of diagnosis. In particular it was not possible, given the numbers employed in the study, to distinguish substantial groups of conduct disordered participants for comparison with affective disorders, without making assumptions about co-morbidity i.e. the primacy of presenting complaint. In using what Cantwell (1996) has called ‘higher order’, or broad based patterns of diagnosis i.e. externalising and internalising
disorders, one may lose the complexity of co-morbidity, which particularly in adolescence may be crucial to understanding manifestations of distress.
In previous studies, e.g. Rosenstein and Horowitz (1996), diagnosis was assigned both on the basis of Axis I disorders from the DSM-IV, and Axis II disorders. Thus participants were assigned a diagnosis of a specific personality disorder, in addition to whether there presenting problems were affective or conduct based. The present study did not assign Axis II diagnoses to any of the participants. Thus it is not possible to make comparisons between the two studies in terms of this dimension of disorders. In addition, as Fonagy et al (1996) have shown there is an established link between Borderline Personality Disorder (BPD) and a deficit in mentalising capacity in an adult psychiatric population. Indeed a number of the participants in the present study exhibited numerous symptoms of BPD, among other personality disorders, in particular in terms of highly impulsive behaviour, dissociative phenomena, and self-harm. However, although assignment of such diagnoses may have proved useful in further establishing the link between reflective self-functioning and BPD, it was felt inappropriate in the current study, given the lack of information for some participants on which to base such a diagnostic judgements, and the reluctance on the part of professionals to label young people with personality disorders, during a phase of developmental change.