ARNTZ ET AL.
BORDERLINE PD SEVERITY INDEX
RELIABILITY AND VALIDITY OF THE
BORDERLINE PERSONALITY DISORDER
SEVERITY INDEX
Arnoud Arntz, PhD, Marije van den Hoorn, MA,
Jurgen Cornelis, MD, Roel Verheul, PhD,
Wies M.C. van den Bosch, MA, and Arthur J.H.T. de Bie, MD
The psychometric properties and validity of the Borderline Personality Disorder Severity Index (BPDSI), a semistructured interview assessing the frequency and severity of manifestations of Borderline Personality Disorder (BPD) during a circumscribed period, were investigated in two studies. In study 1, patients with BPD (n= 15), with other personality dis-orders (PD;n= 18), and without Axis II disorders (but with Axis I disor-ders;n = 10) assessed with the SCID were interviewed with the BPDSI (1-yr. version). Patients also filled out a number of questionnaires. A sec-ond rater judged taped BPDSI interviews. The BPDSI appeared to yield highly reliable (ICC = .93) and internally consistent (Cronbach’sα= .85) scores. The BPDSI strongly discriminated BPD patients from other pa-tients, and was not related to other Axis II pathology. Concurrent and construct validity was excellent. In study 2, a version of the BPDSI suit-able for use in treatment outcome research was investigated (3-month version) in a sample of 64 BPD patients, 23 Cluster C PD patients, and 20 nonpsychiatric controls. Again, reliability coefficients were excellent (ICC = 0.97; Cronbach’sα = 0.93), and validity indices were good. Clinical norms were also derived. In a sample of 28 BPD patients, the instrument detected improvement during 6 months of psychotherapy.
The use of semistructured interviews has led to a marked increase of the re-liability and validity of personality disorder diagnoses (Zimmerman, 1994). However, it is generally acknowledged that patients with the same personal-ity disorder diagnosis might vary considerably as to the extent and severpersonal-ity
45 © 2003 The Guilford Press
From Maastricht University (A. A., M. v. d. H.); Psychiatric Hospital Vijverdal, Maastricht (J. C., A. J. H. T. d. B.); Amsterdam Institute for Addition Research, Academic Medical Center - Uni-versity of Amsterdam (R. V., W. M. C. v. d. B.); and De Tender, Deventer (W. M. C. v. d. B.). Terri Weaver and George Clum provided us with a version of their borderline severity interview. Anja van den Hout is acknowledged for her second assessment of BPDSI interviews (study 1). The authors would like to thank Yvonne van Beijsterveldt, Monique Burger, Hanny Keizers, Evelien Riedijk, Geny Visbach, and Wijnand van der Vlist for their help in patient recruitment, interviewing, and other data collection. Gerthe Veen helped with data entry and statistical computations. Frans Tan gave statistical advice.
Address correspondence to Arnoud Arntz, Department of Medical, Clinical and Experimental Psychology, Maastricht University, P.O. Box 616, NL-6200 MD Maastricht, the Netherlands; E-mail: [email protected]. An English version can be obtained from the first au-thor.
of their pathology. Since most interviews assess, by definition, long-term functioning, these personality disorder interviews are unsuitable for the as-sessment of current severity of personality pathology. This holds especially for Borderline Personality Disorder (BPD), a condition with a large range of severity and instability of its manifestations. For evaluation of treatments there is a particular need for dimensional assessment of current borderline pathology.
The present studies present data on the reliability and validity of an in-strument developed to assess short-term BPD pathology as defined by DSM-III-R/DSM-IV criteria (APA, 1987, 1994). The purpose of the instru-ment is not to diagnose BPD, but to yield a quantitative index of the current severity of BPD manifestations. The semistructured interview is an adapta-tion of an interview developed by Weaver and Clum (1993). The interviewer asks for the frequency and/or severity of specific manifestations of BPD in a circumscribed period. A total score is assumed to represent a dimensional score of severity of BPD pathology over a given period. Weaver and Clum (1993) did not report on the psychometric properties of their interview, al-though they demonstrated a strong relationship between BPD pathology as assessed with their instrument and childhood sexual abuse. Given that the association between BPD and childhood sexual abuse is not unequivocal (Fossati, Madeddu, & Maffei, 1999), and other pathogenic factors might contribute to the formation of BPD, this is not a validation of their interview. Direct demonstration of concurrent validity by correlating the interview scores with other BPD measures is needed.
The first study assessed the psychometric properties of a slightly adapted version of Weaver and Clum’s original interview in a mixed sample of psy-chiatric patients (N = 43). The second study assessed the psychometric properties of a revised version of the interview, using data from BPD pa-tients, psychiatric controls, and nonpsychiatric controls. This version was adapted to the DSM-IV and revised so that the frequency of BPD manifesta-tions was assessed in more detail, in a shorter timeframe, thus making the instrument suitable for treatment-outcome research.
STUDY 1
This study assessed the following psychometric properties of a slightly adapted version of Weaver and Clum’s (1993) original interview, which we called the Borderline Personality Disorder Severity Index (BPDSI), in a mixed sample of psychiatric patients: interrater reliability, internal consis-tency, and discriminant, concurrent and construct validity. As to concur-rent and construct validity, we expected that the BPDSI would positively correlate with the number of BPD symptoms, as assessed with the Struc-tured Clinical Interview for DSM-III-R Personality Disorders (SCID-II; Spitzer, Williams, Gibbon, & First, 1990), with the experienced severity of BPD symptoms during the last months as assessed with a self-report check-list, with lower educational levels (because of not finishing education, cf. APA, 1994, p. 652), with beliefs hypothesized and demonstrated to be spe-cific for BPD patients (Arntz, Dietzel & Dreessen, 1999), with low self-es-teem (because of the belief of being bad and self-punishing attitudes, cf.
APA, 1994, pp. 650-651; Beck, Freeman, & Associates, 1990), and with large self-ideal discrepancies (mainly because the self was expected to be viewed as very negative; see self-esteem). No particularly strong relation-ship was expected between self-other discrepancy, as this index is related to psychotic disorders (Miskimins, Wilson, Braucht, & Berry, 1971; Miskimins & Baker, 1973). Since the symptoms assessed with the BPDSI are hypothesized to be specific for BPD, no relationship between BPDSI and nonBPD personality disorder pathology was expected.
METHOD
SUBJECTS
Forty-three patients (29 women, 14 men) from the local Psychiatric Hospital and the Community Mental Health Center at Maastricht participated. The mean age was 31.1 years (SD= 8.8). According to the SCID-II, 10 patients had no personality disorder, 18 patients had one or more personality disor-ders (but no BPD diagnosis), and 15 patients (3 men, 12 women) met criteria for BPD. If the sample was divided according to the number of BPD traits, it was found that 17 patients met 2 or fewer BPD criteria, 11 patients met 3 or 4 criteria, and 15 patients met 5 or more criteria (i.e., received a BPD diag-nosis). There were no significant differences between groups whether the sample was divided by male/female proportion or by age. However, BPD pa-tients had lower educational levels than the other papa-tients (t(41) = -2.96,p= .005), a phenomenon possibly intrinsically related to the BPD diagnosis (given the unstable patterns, choices and identity as defined by DSM crite-ria).
MATERIALS
Axis II disorders were assessed by means of the SCID-II (DSM-III-R version; Spitzer, Williams, Gibbon & First, 1990; Psychiatrisch Centrum Bloemendaal, 1991). Previous research demonstrated that the SCID-II yields reliable Axis II diagnoses (overallκ= 0.80; BPDκ= 0.79; Arntz et al., 1992; see also Zimmerman, 1994).
The severity of BPD symptoms was assessed with the BPDSI. The Ameri-can interview was translated into Dutch and slightly adapted. The inter-viewer assessed the BPD manifestations during the previous year. A psychiatrist (JC) conducted all the interviews. A subsample (N= 19) of the interviews was audiotaped and scored by another rater who was blind to the SCID-II diagnoses.
The following instruments were used as criterion measures. A 71-item questionnaire with assumptions about the self (i.e., “I’m bad,” “I’m power-less”), about others (i.e., “People cannot be trusted”), and about conditional relationships, mainly between self and others (i.e., “If I show uncertainty, people will reject me”), was used to measure strength of BPD-related beliefs. These assumptions were generated from cognitive theories on BPD (Arntz, 1994; Beck et al., 1990) and from cognitive therapies of individual BPD
pa-TABLE 1. Intraclass Correlation Coefficients (ICCs) of Ratings of Individual Questions, Criterion Scores, and Total Score of the BPDSI
ICC
Criterion question Study 1a Study 2b
1. Unstable relationships 0.88 0.99
ups and downs in relationships 0.76 0.99
new relationships 0.96 0.94
2. Self–damaging impulsivity 0.82 0.97
financial problems due to spending —1 0.92
impulsive sexual contacts (unsafe) 0.69 0.92
impulsive sexual contacts (with strangers) 0.69 0.66
abuse of alcohol 0.91 0.95
abuse of soft drugs 0.91 0.82
gambling 0.62 0.80
use of hard drugs 0.93 0.94
binge eating 0.88 0.94
reckless driving 0.68 0.99
shoplifting 0.79 1.00
other —1 0.85
3. Affective instability 0.97 0.89
frequency of affective shifts 0.98 n/a
depression 0.65 0.88
irritability 0.46 0.81
anxiety 0.62 0.89
4. Lack of anger–control 0.71 0.93
subjective rage n/a 0.92
verbally 0.90 0.78
throwing objects 0.72 0.99
physical fights 0.29 0.97
5. Recurrent suicidal behavior 0.93 0.95
Self–mutilation (in general) 0.83 n/a
hit oneself n/a 0.99
cut oneself n/a 1.00
burned oneself n/a 1.00
pricked with needle n/a 1.00
taking dangerous substances n/a 1.00
other n/a 0.52
suicidal threats 0.64 0.93
suicidal gestures 0.99 0.83
suicidal behavior —1 0.97
6. Identity disturbance (severity of uncertainty about): 0.90 0.86
self–image 0.82 0.86
long–term goals 0.91 0.76
moral values 0.91 0.72
preferred values 0.91 0.74
type of friends desired 0.89 0.80
sexual orientation 0.57 0.74
7. Chronic feelings of emptiness 0.81 0.82
feelings of emptiness or boredom 0.77 0.79
avoidance of emptiness by going out 0.50 0.65
avoidance of emptiness by calling friends 0.70 n/a
difficulty of being alone 0.57 0.92
8. Avoids any abandonment 0.68 0.80
despair when (perceived) abandoned n/a 0.76
frantic efforts to prevent abandonment 0.86 0.99
asking too much affirmation 0.70 0.63
asking reassurance that other loves patient 0.70 0.66
physical efforts to avoid abandonment 0.49 1.00
tients. Factor-analysis and reliability analysis led to a decrease from 71 to 63 items, specific for BPD patients, which were summed to a highly inter-nally consistent total score (Cronbach’sα= .98).
Self-esteem was assessed by means of a 15-item, self-report scale adapted from Rosenberg (Rosenberg, 1965; van den Hout, Arntz, & Kunkels, 1995). In the present sample, the total score proved to be highly reliable (Cronbach’sα= .91).
Self-other and self-ideal discrepancies were assessed with Miskimins’s Self-Goal-Other Discrepancy Scale (Miskimins, Wilson, Braucht, & Berry, 1971; Miskimins & Baker, 1973). Two discrepancy scores are derived: the mean difference between self and ideal, and the mean difference between self and (perceived) other’s evaluations. In a sample of 53 patients from the 2 Maastricht institutes (including the current 43 patients) the internal con-sistency was found to be excellent (Cronbach’sαof self-other discrepancy = .86, of self-ideal discrepancy .89).
A 40-item, self-report questionnaire asked for ratings of severity of bor-derline symptoms during the last month on 5-point Likert scales. ThisBPD
symptom checklist(Arntz & Dreessen, 1992) showed excellent internal
con-sistency in the present sample, Cronbach’sα= .94.
RESULTS
INTERRATER RELIABILITY
Interrater reliability was estimated by computing intraclass correlation co-efficients (ICC) for individual questions, the criterion scores, and the total score (see Table 1). ICCs ranged from poor to excellent (range = .29 - .99), but were on the whole satisfactory (median = .74). For three questions, the ICC could not be computed because of lack of variance. Criterion scores had good ICCs (median = .85; range = 0.68 - 0.97). The total BPDSI score had ex-cellent interrater reliability (ICC = 0.93).
TABLE 1. continued
ICC
Criterion question Study 1a Study 2b
9. Dissociation and paranoid ideation n/a 0.97
dissociation
depersonalization n/a 0.93
derealization n/a 0.99
amnesia of known objects/persons n/a 1.00
amnesia of autobiographic memories n/a 0.98
paranoid ideation
suspicion n/a 0.89
persecutory beliefs n/a 0.98
unfairly treated n/a 0.95
Total BPDSI score 0.93 0.97
Note.1ICC could not be computed because of lack of variance.aStudy 1: N = 19 (mixed patients), one inter-viewer and one independent rater.bStudy 2: N = 50 (27 BPD, 23 Cluster–C patients), 5 interviewers and one independent rater. n/a = not assessed.
INTERNAL CONSISTENCY
For unweighted scores, internal consistency proved to be very good (Cronbach’s α = 0.82). One question did not correlate positively with the item-rest score (r= -.09): calling friends to avoid boredom or emptiness. This item was excluded from further computations.
DISCRIMINANT VALIDITY
Mean scores of patients without a PD (n= 10) and with a PD other than BPD (n= 18) did not differ significantly,F(1, 27) = .001. The BPDSI also did not correlate significantly with severity of Axis-II pathology other than BPD, ex-pressed as the number of nonBPD personality disorders (r= .09,p= .29).
CONCURRENT AND CONSTRUCT VALIDITY
The mean BPDSI score was much higher in BPD patients (m= 42.27) than in nonBPD patients (m= 15.69),F(1, 42) = 134.02,p<0.001. The BPDSI score was strongly associated with the number of SCID-II BPD criteria (r= .91,p<
.001). The correlation between BPDSI and number of BPD criteria in the 15 BPD patients of the sample was 0.70 (49% explained variance), which sug-gests that the BPDSI, although relating to the number of BPD traits, taps a considerable amount of unique information.
Table 2 presents an overview of the associations between BPDSI and other variables. Additional regression analyses were executed to control for the possible influence of educational level and sex. If this correction resulted in a considerable change in the association between BPDSI and other vari-ables, the corrected correlation level is also presented in Table 2.
As was expected, there was a negative association between educational level and the BPDSI (Table 2). The BPDSI correlated positively with the BPD symptom checklist and the BPD assumption questionnaire (Table 2). Sex
TABLE 2. Correlations between BPDSI and Other Variables
Study 1 Study 2
Variable rwith BPDSI with BPDSICorrected1r rwith BPDSI
educational level –0.51*** no change2 –0.20*
BPD symptom checklist 0.60*** no change2,3 0.85***
BPD assumptions 0.50*** no change2,3 0.52*** self–esteem –0.27+ –0.53***3 –0.49*** self–other discrepancy 0.08 0.31 0.33* self–ideal discrepancy 0.23+ 0.53***2,3 0.53*** SCL–90 0.70*** DSQ immature 0.74*** DSQ neurotic 0.06 DSQ mature –0.11
and educational level did not influence these associations. After correction for educational level, a significant negative correlation between the BPDSI and self-esteem emerged (Table 2). As was expected, the BPDSI related to self-ideal discrepancy (after correction for sex and educational level), but not to self-other discrepancy (Table 2). In short, correlations with other vari-ables indicate good validity for the BPDSI.
STUDY 2
A revised version of the BPDSI was constructed based on Study 1 results. First, one item with poor item-rest correlation was deleted. Second, affective instability was assessed with three instead of four items. The revision asked for frequency of affective shifts of depressive, irritable, and anxious sub-types separately, instead of asking for frequency of affective shifts in general and then assessing the extent by asking for depressive, irritable, and anx-ious subtypes. This is more in line with the way the other symptoms are as-sessed. Because frequency of shifts was now explicitly assessed for each affect separately, this may increase the ICCs of the three affective items. Third, when possible, we separated different behaviors (e.g., frequency of impulsive soft drug use was scored separately from impulsive alcohol abuse). We also added seven items to assess manifestations of the ninth cri-terion on transient losses of reality testing added in the DSM-IV (APA, 1994).
An instrument like the BPDSI is designed to assess the current severity of BPD manifestations. Such an index is needed for investigations of the ef-fects of therapy. For use in treatment outcome studies, we decided to assess BPD symptoms with the BPDSI during the last 3 months. Such a timeframe opens the possibility of studying changes during treatment in a detailed manner. A further refinement was to use an 11-point scale to assess the fre-quency of the BPD manifestations during this time period. The following psychometric properties of this new BPDSI were investigated in BPD and Cluster-C PD patients, and in nonpsychiatric controls: interrater reliability, internal consistency, test-retest reliability, discriminant validity, and con-current validity. As to validity, we expected the same relationships as in Study 1 with other variables. Four measures were added: general psychopathological symptoms (SCL-90), immature defenses, neurotic de-fenses, and mature defenses (DSQ; Andrews, Pollock, & Stewart, 1989; An-drews, Singh, & Bond, 1993). We expected a strong association between SCL-90 and BPDSI (the more BPD-specific symptoms, the more general psychopathological complaints were expected). Based on Kernberg’s psy-choanalytic theory, we expected that the BPDSI would be associated with immature defenses, but not substantially with neurotic and mature de-fenses (Kernberg, 1996).
Although the BPDSI is not a diagnostic instrument, clinical norms and specificity/sensitivity were also assessed, since these figures are useful in the context of treatment and treatment studies. We also assessed the capa-bility of the BPDSI to detect reduction of BPD pathology due to treatment.
METHOD
SUBJECTS
Sixty-four BPD and 23 Cluster-C PD patients participated from the follow-ing centers: Amsterdam Jellinek Clinic for Addiction, Community Mental Health Center at Maastricht, Psychiatric Outpatient Clinic Vijverdal at Maastricht, Psychotherapeutic Inpatient Center at Lunteren, Psychiatric Center Amsterdam, and Jelgersma Psychiatric Outpatient Clinic at Oegstgeest. There was no sample overlap with Study 1. Twenty nonpsychiatric controls were recruited by advertisements in newspapers in Amsterdam and Maastricht. Subjects were selected by means of SCID-I and -II interviews. BPD patients had to meet a BPD diagnosis, Cluster C patients had to meet one or more Cluster C PD diagnoses and no BPD criterion. Ex-clusion criteria were age below 18 or above 65 years, bipolar disorder, pres-ent psychotic disorder, substance dependence or abuse needing clinical detoxification, and mental retardation. Nonpsychiatric controls aged 18-65 years did not have a psychiatric diagnosis according to the SCIDs, and did not meet any BPD criterion.
Six (9.4%) of the BPD patients, seven (30.4%) of the Cluster C patients, and three (15%) of the nonpsychiatric controls were men,χ2(2,n= 107) =
5.9,p= 0.052. The mean age was 34.3 years (SD= 8.7), BPD patients (m= 31.3,SD= 7.4) tending to be somewhat, but not significantly younger than the other groups (Cluster C,m= 36.8,SD= 8.0; nonpsychiatric controls,m= 41.1,SD= 8.6),F(2, 104) = 1.55,p= 0.064. There were no significant differ-ences between the groups as to proportions having an intimate partner (av-erage 48.6%), and having a job or being a student (av(av-erage 43%), but, as in Study 1, BPD patients had, on the average, lower educational levels than the other two groups,F(2, 104) = 3.39,p= 0.02.
MATERIALS
Axis I disorders were assessed with a Dutch version of the SCID-I for DSM-III-R (Spitzer & Williams, 1986) or DSM-IV (First, Spitzer, Gibbon, & Williams, 1996; Groenestijn, Akkerhuis, Kupka, Schneider, & Nolen, 1999). Axis II disorders were assessed by means of the SCID-II for DSM-III-R or for DSM-IV (First, Gibbon, Spitzer, Williams, & Benjamin, 1997; Weertman, Arntz, & Kerkhofs, 2000).
As previously mentioned, the BPDSI was adapted to make it suitable for detailed assessment of BPD features present during a 3-month period. An 11-point scale was used to rate the frequency of each item. To reduce skewed distributions, the higher frequencies were condensed in the scoring options, and the lower frequencies were stretched. The scoring options for each item were (note: timeframe is 3 months): 0 = never, 1 = once, 2 = twice, 3 = three times, 4 = 4 or 5 times, 5 = 6 or 7 times, 6 = 8 to 10 times, 7 = 11 to 15 times (once per week), 8 = several times a week but less than half of the week, 9 = more than half of the week, 10 = daily. Comparing the distribu-tions of this scoring method with raw frequency scoring indeed revealed that this scoring reduced skewness to acceptable levels. Identity questions were assessed with 5-point Likert scales, on which the interviewer rated the
se-verity of the disturbance on the particular area. For each DSM criterion an average score was derived (range = 0-10). The sum of these 9 scores gives the total score (range = 0-90).
Five psychologists and psychological assistants trained by the first author conducted the interviews. One second judge, blind to the DSM diagnoses, rated audiotapes of 50 interviews (27 BPD patients, 23 Cluster C patients). Thirty-five BPD patients were again interviewed 3 months after the first BPDSI assessment, and 28 of these patients again after another 3 months. Note that the interview periods didnotoverlap. These patients participated in pilot treatments of psychodynamic or schema-focused psychotherapy. This data was used to estimate test-retest reliability1and the instrument’s
capacity to detect change due to treatment.
The following self-report instruments assessed the criterion variables. The Personality Disorder Beliefs Questionnaire (PDBQ) was used to mea-sure strength of beliefs hypothesized to be specific for six PDs (Avoidant, De-pendent, Obsessive-Compulsive, Paranoid, Histrionic, and Borderline). This 120-item questionnaire asks subjects to score believability of assump-tions about the self, about others, and about conditional relaassump-tionships, mainly between self and others on 100 mm. Visual Analogue scales. Arntz, Dietzel, and Dreessen (1999) demonstrated good psychometric properties (e.g., Cronbach’sα= .88-.97) and a specific relationship of the BPD subscale with DSM BPD pathology.
Self-reported severity of BPD symptoms, self-esteem and self-other and self-ideal discrepancies were measured in the same way as in Study 1.
The SCL-90 was used as a general index of psychopathological symptoms. The Dutch version yields a highly reliable sumscore, Cronbach’s α = .97 (Arrindell & Ettema, 1981).
The Defensive Style Questionnaire (DSQ; Andrews et al., 1989; Andrews et al., 1993) was used to measure defense mechanisms. From a Dutch 48-item version, a 36-item form (Muris & Merckelbach, 1994; Spinhoven, van Gaalen, & Abraham, 1995) was derived, with extra neurotic items added to increase the strength of this factor. Three scores were derived: ma-ture defenses (Cronbach’s α = 0.76), neurotic defenses (Cronbach’s α = 0.85), and immature defenses (Cronbach’sα= 0.86).
RESULTS
INTERRATER RELIABILITY
The ICCs for the individual items, criterion scores, and total score of the BPDSI are presented in Table1. The ICCs were generally very high. The me-dian ICC for the items was .92 (range = .52-1.00). The 9 criterion scores had
1. Note that the reliability estimated by a test-retest design is by definition restricted by the in-stability of the assessed construct. With state-like constructs, lower test-retest reliabilities are to be expected than with trait-like constructs. Since the BPDSI assesses manifestations of BPD, that are, by nature, unstable, only modest test-retest reliability coefficients are to be ex-pected when different timeframes are used in the test-retest design, such as in the present study.
very high interrater reliabilities (median = .93; range .80-.99), as did the to-tal score (ICC = .97).
INTERNAL CONSISTENCY
Internal consistency of the total score proved to be very good (Cronbach’sα in total sample = 0.93; in BPD subsample = 0.82). In the total sample, cor-rected item-total correlations ranged from .07 to .78. The scores per DSM criterion had the following internal consistencies (expressed as Cronbach’s
α): unstable relationships = 0.41, self-damaging impulsivity = 0.67, affective instability = 0.80, lack of anger-control = 0.69, recurrent (para)suicidal be-havior= 0.68, identity disturbance = 0.76, chronic feelings of emptiness = 0.69, avoids any abandonment = 0.64, dissociation and paranoid ideation = 0.83. Note that the unstable relationship subscale consisted of only two items. If 10 items with identical intercorrelations had been used, the inter-nal consistency would have been 0.78 (Spearman-Brown correction). On the individual-item level, three items had item-rest correlations that were too low: gambling, sex with strangers, and doubt about sexual orientation.
TEST-RETEST RELIABILITY AND CHANGE DURING TREATMENT
Thirty-five BPD patients were again interviewed 3 months after the first BPDSI assessment. At that point, they had received 3 months of treatment. We expected no substantial change in this early phase of treatment (pa-tients did not enter psychotherapy in acute crisis). Indeed, no significant changes in the group’s mean scores were observed on the criterion and total score levels. The mean BPDSI score was 28.81 on the first assessment, and 28.22 on the second assessment (t(34) = 0.43,p= 0.67). The test-retest relation of the total score was 0.72. On the criterion level, the test-retest cor-relation ranged from 0.20 (unstable cor-relations, only two items) to 0.69 (impulsivity and dissociation/paranoid ideation).
A second test-retest index could be computed from 28 BPD patients from the third to the sixth months assessment. The average total score decreased somewhat from 26.95 to 24.28, t(27) = 1.83, p = 0.08 (two-tailed). The test-retest correlation was comparable to the first finding, 0.77. The BPDSI score of these 28 patients decreased significantly during these 6 months of treatment,t(27) = 2.17,p<0.05. The 0 to 6 months’ test-retest correlation was 0.72.
DISCRIMINANT VALIDITY
The BPD group received significantly higher scores on the BPDSI than the Cluster C group, with means of 30.58 and 6.95,t(78.58, separate variance estimates) = 15.36, p < 0.001; and the nonpsychiatric controls (mean = 3.21),t(63.65, separate variance estimates) = 16.27,p<0.001. All criterion scores discriminated between BPD and both control groups. Only four items did not discriminate between BPD and both groups (three items) or Cluster C (one item): gambling, shoplifting, physical efforts to avoid
aban-donment, sexual orientation. The first three items had very low frequencies in the BPD sample, and zero frequency in the control groups. Cluster-C pa-tients also reported some uncertainty about sexual orientation.
To control for sample differences a multiple regression was executed with the BPDSI score as dependent variable, and the following predictors: BPD, Cluster C PD, sex, age, educational level, living with a partner, having work/study, number of Axis I disorders, number of Axis II disorders. With a forward analysis (p<0.05) only BPD was entered,β= 0.83,p<0.001, ex-plained variance = 69%. When all variables were entered, BPD was still the strongest predictor,β= 0.90,p<0.001, but number of Axis-I disorders was also significantly related to BPDSI score,β= -2.48,p= 0.015 (note the nega-tive association), and number of Axis II disorders related posinega-tively to BPDSI score,β= 0.13,p= 0.08. In total the 9 predictors explained 73% of the vari-ance. The fact that one general pathology index (number of Axis I disorders) was negatively correlated with BPDSI, the other (number of Axis II disor-ders) positively, their significance levels below Bonferroni corrected levels, and their disappearance as predictors in a forward approach suggest that the associations were accidental.
CONCURRENT AND CONSTRUCT VALIDITY
In the patient sample, the BPDSI total score correlated significantly with the following other variables: Number of SCID-II BPD criteria (r = 0.88, p <
0.001), BPD-symptom checklist (r= 0.85,p<0.001), SCL-90 (r= 0.70,p<
0.001), self-esteem (r= -0.49,p<0.01), DSQ - immature (r= 0.74,p<0.001), other-self discrepancy (r= 0.33,p<0.05; but not very strongly: explained variance 10.9%), ideal - self discrepancy (r= 0.53,p<0.01), and all PDBQ subscales (rs = 0.34 - 0.52, ps < 0.05), but most strongly with the BPD subscale of the PDBQ (r= 0.52,p<0.01). The BPDSI did not correlate signif-icantly with DSQ neurotic defenses,r= 0.06, ns, and DSQ mature defenses,
r= -0.11, ns.. To compare these findings with Study 1, see Table 2. The correlation between BPDSI and number of BPD criteria in the BPD patients was 0.47 (explained variance = 22%), which indicates that the BPDSI, though being associated with the number of BPD traits in a BPD sample, taps a considerable amount of unique information.
CLINICAL NORMS AND SPECIFICITY/SENSITIVITY
Three cutoff scores were determined using formulas by Jacobson and Truax (1991). The dysfunctional cut-off score indicates where dysfunction (i.e. BPD pathology) starts. People above this score differ less than two SD from the BPD population. It was found to be 11.32. After deleting outliers from the data, the dysfunctional cut-off score increased to 15.04. The functional cut-off score indicates where normality starts: people scoring below this point are within two SD of the nonpsychiatric mean. This was found to be 13.24, after deleting outliers, 9.43. Thirdly, the point between the BPD sam-ple and the nonpsychiatric controls was determined (Jacobson and Truax, 1991): 12.58. Deleting outliers was of some influence, 11.17. Finally,
sensi-tivity and specificity of both dysfunctional cut-off scores were computed in the complete data set. The dysfunctional cut-off score of 15.04 was found to be the most sensitive (0.95) and the most specific (0.95). The alternative cut-off score of 11.32 had lower sensitivity (0.92) and specificity (0.90).
Taken together, a clinical norm of 15 seems a justified choice given the dif-ferent cut-off scores and its high sensitivity and specificity. The following norms can be used: scores below 10 are in the “normal” range, scores be-tween 10 and 15 are on the borderline bebe-tween normality and BPD, and scores above 15 signify BPD pathology.
DISCUSSION
The findings of both studies suggest that it is possible to construct a psychometrically sound variable that represents the severity of Borderline PD pathology based on data derived by a clinical interview of the patient. Both BPDSI versions had high interrater reliability, high internal consis-tency, and high concurrent and discriminant validity. In addition, reliability indices indicated that we were successful in increasing the reliability of the BPDSI in the second version (i.e., ICC increased from 0.93 to 0.97, internal consistency from 0.82 to 0.93). Given the state character of the 3 months BPDSI, and the fact that we used assessments over different periods, test-retest reliabilities (in the .72-.77 range for the total score) were accept-able. Lastly, the 3-month version appeared to have the capacity to detect re-duction of BPD pathology during treatment. Thus, the instrument showed treatment sensitivity.
Looking at the validity indices in more detail (Table 2), one sees higher cor-relations of the self-report BPD symptom checklist with the 3-month BPDSI version than with the 1-year version: 0.85 versus 0.60. This is in line with the interpretation that the 1-year version is less a state-measure than the self-report form, covering the previous months. The 1-month self-report form and the 3-month BPDSI have a very high correlation, despite the differ-ence in method. The relationships with other validity indices that were as-sessed in both studies proved to be very similar in Studies 1 and 2 (Table 2). The absence of correlations in Study 2 between BPDSI and DSQ neurotic and mature defense factors supports the validity of the BPDSI, as the BPDSI seems to assess only psychopathology that is related to primitive defenses. The BPDSI scores of Cluster C PD patients also support the validity of the BPDSI in this respect.
The formulation of new items in the 3-month version of the BPDSI seemed, in general, to be successful. The same holds for the addition of seven new items on dissociation and paranoid ideation, to adapt the BPDSI to DSM-IV. There was however a number of items that had questionable psychometric indices: gambling, sex with strangers, shoplifting, uncer-tainty about sexual orientation (also reported by Cluster C patients), and physical efforts to avoid abandonment. We are reluctant to delete them from the BPDSI as they are so clearly thought to be BPD-related pathology (e.g., APA, 1994). But, if others also demonstrate poor performance for these indi-ces, changes in our conceptualization of BPD may be in order.
Another issue pertains to the content of the items related to DSM-IV crite-rion 7. Whereas the DSM-IV defines this critecrite-rion as “chronic feelings of emp-tiness” (APA, 1994, p. 654), the description on p. 651 adds that “Easily bored, they may constantly seek something to do.” Likewise, the SCID-II user’s guide operationalizes this criterion as follows: “Chronic feelings of emptiness often are associated with feeling bored, hollow, alone, or without definition.” (First, Gibbon, Spitzer, Williams, & Benjamin, 1997, p. 27). The BPDSI (as its predecessor, Weaver & Clum, 1993) assesses both emptiness and boredom, avoidance of these feelings, and problems with being alone. The DSM-IV sourcebook, on the other hand, states that boredom is not specific to BPD (Gunderson, Zanarini, & Kisiel, 1996, pp. 723-724). As to difficulty of being alone (only rated under criterion 7 in the BPDSI when being alone contributes to feelings of emptiness and boredom), the issue can be raised whether or not this item is more related to DSM-IV criterion 1 (efforts to avoid abandon-ment). Clearly, further research is needed to clarify these issues.2
The design used to estimate test-retest reliability and the capacity of the BPDSI to detect changes due to treatment was far from ideal. Since 3 months passed between the first and second assessment, and the two inter-views therefore covered a completely different period of time, the test-retest correlations should be considered as underestimations of the true test-re-test reliability. A better option would be to retest-re-test the patient a week later and have both interviews cover exactly the same period. Given these consider-ations, the observed test-retest correlations (in the .72-.77 range) are en-couraging. The BPDSI’s capacity to validly detect change due to treatment should be assessed by comparing it to other indices of improvement. The first reports using such a design are encouraging (Giesen-Bloo, Arntz, van Dyck, Spinhoven, & van Tilburg, 2001).
The 3-month version is now used as one of the outcome measures in a controlled clinical trial, comparing psychodynamic psychotherapy with schema-focused cognitive therapy for BPD. An English version is available so that the instrument can be investigated in other countries. Future re-search will indicate the instrument’s usefulness as a therapy outcome mea-sure. Investigations into other samples, including in other countries, are also needed to further define clinical norms. Further research should also address the instrument’s potential susceptibility to be influenced by state effects due to, for example, Axis I comorbidity. Although interviewers are ex-plicitly instructed to score manifestations of BPD traits only (and to ignore BPD-like symptoms due to other psychiatric conditions as much as possi-ble), several studies have shown that Axis II diagnoses obtained by semistructured interviews are sensitive to contamination by state condi-tions such as anxiety and mood symptoms (e.g., Ames-Frankel et al., 1992; Peselow, Sanfilipo, & Fieve, 1995; Ricciardi et al., 1992; Stuart, Simons, Thase, & Pilkonis, 1992; Trull & Goodwin, 1993; Verheul et al., 2000; how-ever see Loranger et al., 1991; Zimmerman & Coryell, 1990). With respect to the BPDSI, this concern is somewhat mitigated due to the fact that the
view items target borderline symptoms at the lowest possible abstraction level, so interviewers are able to inquire into potential trait-state artifacts on an item-by-item basis. On the other hand, the BPDSI aims to measure re-cent pathology that could more easily be confused with state conditions un-related to BPD than when one evaluates enduring behavior patterns using one of the diagnostic interviews.
REFERENCES
Ames-Frankel, J., Devlin, M.J., Walsh, B.T., Strasser, T.J., Sadick, C., Oldham, J.M., & Roose, S.P. (1992). Personality disorder diagnosis in patients with bulimia nervosa: clinical correlates and changes with treatment. Journal of Clinical Psychiatry,53, 90-96. Andrews, G., Pollock, C., & Stewart, G.
(1989). The determination of defense style by questionnaire.Archives of Gen-eral Psychiatry,46, 455-460.
Andrews, G., Singh, M., & Bond, M. (1993). The defense style questionnaire. The Journal of Nervous and Mental Disease, 181, 246-256.
American Psychiatric Association (APA). (1987).Diagnostic and Statistical Man-ual of Mental Disorders( third edition, revised). Washington, DC: Author. American Psychiatric Association (APA).
(1994).Diagnostic and Statistical Man-ual of Mental Disorders (fourth edition). Washington, DC: Author.
Arntz, A. (1994). Treatment of Borderline Per-sonality Disorder: A challenge for cog-nitive-behavioural therapy.Behaviour Research and Therapy,32, 419-430. Arntz, A., van Beijsterveldt, B., Hoekstra, R.,
Hofman, A., Eussen, M., & Sallaerts, S. (1992). The interrater reliability of a Dutch version of the Structured Clini-cal Interview for DSM-III-R personality d i s o r d e r s . A c t a P s y c h i a t r i c a Scandinavica,85, 394-400.
Arntz, A., Dietzel, R., & Dreessen, L. (1999). Assumptions in borderline personality disorder: Specificity, stability and rela-tionship with etiological factors. Behav-iour Research and Therapy, 37, 545-557.
Arntz, A., & Dreessen, L. (1992).Borderline Personality Disorder Symptom Check-list. Internal document, Maastricht University.
Arrindell, W.A., & Ettema, H. (1981). Dimen-sional structure, reliability and validity of the Dutch version of the Symptom
C h e c k l i s t ( S C L - 9 0 ) . N e d e r l a n d s Tijdschrift voor de Psychologie, 36, 77-108.
Beck, A.T., Freeman, A., & Associates. (1990). Cognitive therapy of personality disor-ders. New York: Guilford.
First, M.B., Gibbon, M., Spitzer, R.L., Wil-liams, J.B.W., Benjamin, L.S. (1997). User’s guide for the structured clinical interview for DSM-IV Axis II personality disorders (SCID-II). Washington, DC: American Psychiatric Press.
First, M.B., Spitzer, R.L., Gibbon, M., & Wil-liams, J.B.W. (1996).Structured clinical interview for DSMIV Axis I disorders -patient edition (SCID-I/P), version 2.0. New York: Biometrics Research De-partment, New York State Psychiatric Institute.
Fossati, A., Madeddu, F., & Maffei, C. (1999). Borderline Personality Disorder and childhood sexual abuse: A meta-ana-lytic study.Journal of Personality Disor-ders,13, 268-280.
Giesen-Bloo, J., Arntz, A., van Dyck, R., Spinhoven, P., & van Tilburg, W. (2001).Outpatient treatment of Border-line Personality Disorder: Analytical psychotherapy versus cognitive behav-ior therapy. Paper presented at the World Congress of Behavioral and Cog-nitive Therapies, July 17-21, 2001, Vancouver.
Groenestijn, M.A.C., Akkerhuis, G.W., Kupka, R.W., Schneider, N., & Nolen, W.A. (1999). Gestructureerd klinisch interview voor de vaststelling van DSM-IV as I stoornissen. (Structured clinical interview for DSM-IV Axis I dis-orders). Lisse, Netherlands: Swets & Zeitlinger.
Gunderson, J., Zanarini, M.C., & Kisiel, C.L. (1996). Borderline Personality Disor-der. In: T.A Widiger, A.J., Frances, H.A., Pincus, R., Ross, M.B. First, & W.W. Davis, (Eds.),DSM-IV sourcebook,
volume 2 (pp. 717-731), Washington, DC: American Psychiatric Association. Jacobson, N. S., & Truax, P. (1991). Clinical
significance: A statistical approach to defining meaningful change in psycho-therapy research.Journal of Consulting and Clinical Psychology,59, 12-19. Kernberg, O. (1996). A psychoanalytic theory
of personality disorders. In J.F. Clarkin and M.F. Lenzenweger (Eds.), Major Theories of Personality Disorders (pp. 106-140). New York: Guilford. Loranger, A.W., Lenzenweger, M.F., Gartner,
A., Lehmann Susman, V., Herzig, J., Zammit, G.K., Gartner, J.D., Abrams, R . C . , a n d Y o u n g , R . C . ( 1 9 9 1 ) . Trait-state artifacts and the diagnosis of personality disorders. Archives of General Psychiarty,48, 720-728. Miskimins, R.W. & Baker, B.R. (1973).
Self-concept and the disadvantaged. Journal of Community Psychology, 1, 347-361.
Miskimins, R.W., Wilson, L.T., Braucht, G.N., & Berry, K.L. (1971). Self-concept and psychiatric symptomatology.Journal of Clinical Psychology,27, 185-187. Muris, P., & Merckelbach, H. (1994). Defense
style, trait anxiety, worry, and bodily symptoms.Personality and Individual Differences,16, 349-351.
Perry, J.C. (1992). Problems and consider-ations in the valid assessment of per-sonality disorders.American Journal of Psychiatry,149, 1645-1653.
Peselow, E.D., Sanfilipo, M.P., & Fieve, R.R. ( 1 9 9 5 ) . R e l a t i o n s h i p b e t w e e n hypomania and personality disorders before and after successful treatment. American Journal of Psychiatry, 152, 232-238.
Psychiatrisch Centrum Bloemendaal. (1991). N e d e r l a n d s e v e r t a l i n g v a n h e t gestructureerd klinisch interview voor DSM-III-R persoonlijkheidsstoornissen. (Dutch translation of the Structured Clinical Interview for DSM-III-R Per-sonality Disorders.) Bloemendaal, Netherlands: author.
Ricciardi, J.N., Baer, L., Jenike, M.A., Fischer, S.C., Sholtz, D., & Buttolph, M.L. (1992). Changes in DSM-III Axis II diagnosis following treatment of Obses-sive-Compulsive Disorder. American Journal of Psychiatry,149, 829-831. Rosenberg, M. (1965).Society and the
adoles-cent self-image. Princeton, NJ: Prince-ton University Press.
Spinhoven, P., van Gaalen, H.A.E., & Abra-ham, R.E. (1995). The defensive style questionnaire: A psychometric exami-nation. Journal of Personality Disor-ders,9, 124-133.
Spitzer, R.L., & Williams, J.B.W. (1986). Structured Clinical Interview for DSM-III (SCID).New York: New York State Psy-chiatric Institute, Biometric Research. Spitzer, R.L., Williams, J.B.W., Gibbon, M., & First, M.B. (1990). Structured Clinical Interview for DSM-III-R Personality Dis-orders (SCID-II, version 1.0). Washing-ton, DC: American Psychiatric Press. Stuart, S., Simons, A.D., Thase, M.E., &
Pilkonis, P. (1992). Are personality as-sessments valid in acute major depres-sion?Journal of Affective Disorders,24, 281-290.
Trull, T.J., & Goodwin, A.H. (1993). Relation-ships between mood changes and the report of personality disorder symp-toms. Journal of Personality Assess-ment,61, 99-111.
van den Hout, J.H.C., Arntz, A., & Kunkels, F.H.J. (1995). Efficacy of a self-control therapy program in a psychiatric d a y - t r e a t m e n t c e n t e r . A c t a Psychiatrica Scandinavica,95, 25-29. Verheul, R., Kranzler, H.R., Poling, J.,
Tennen, H., Ball, S.A., & Rounsaville, B.J. (2000). Axis I and Axis II disorders in alcoholics and drug addicts: Fact or artifact?Journal of Studies on Alcohol, 61, 101-110.
Weaver, T.L., & Clum, G.A. (1993). Early fam-ily environments and traumatic experi-ences associated with Borderline Personality Disorder. Journal of Con-sulting and Clinical Psychology, 61, 1068-1075.
Weertman, A., Arntz, A., & Kerkhofs, M.L.M. (2000). Gestructureerd klinisch inter-v i e w v o o r D S M - I V p e r s o o n-l i j k h e i d s s t o o r n i s s e n ( S C I D - I I ) . (Structured Clinical Interview for DSM-IV Axis II Personality Disorders [SCID-II]). Lisse, Netherlands: Swets & Zeitlinger.
Zimmerman, M. (1994). Diagnosing personal-ity disorders.Archives of General Psy-chiatry,51, 225-245.
Zimmerman, M., & Coryell, W.H. (1990). Di-agnosing personality disorders in the community: A comparison of self-re-port and interview measures.Archives of General Psychiarty,47, 527-531.
