Using the MMPI-2 to Detect Substance Abuse in an Outpatient Mental Health Setting

Using the MMPI-2 to Detect Substance Abuse in an Outpatient

Mental Health Setting

L. A. R. Stein, John R. Graham, Yossef S. Ben-Porath, and John L. McNulty

This study examined the utility of the MMPI-2 in detecting substance-abuse problems in an outpatient mental health setting. Specifically, the utility of the Addiction Acknowledgment Scale (AAS; N. C. Weed, J. N. Butcher, T. McKenna, & Y. S. Ben-Porath, 1992), the Addiction Potential Scale (APS; N. C. Weed et al., 1992), and the MacAndrew Alcoholism Scale—Revised (MAC-R; J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegan, & B. Kaemmer, 1989) in the prediction of substance abuse was evaluated. In addition, the incremental validity of the AAS and the APS in comparison to the MAC-R scale was evaluated. The sample consisted of 500 women and 333 men from a large community mental health center in Northeastern Ohio. Results indicated that the MAC-R scale, the AAS, and the APS were related to interviewer ratings of substance abuse in this outpatient treatment setting. Specifically, the results pointed to the superiority of AAS over APS in substance-abuse identification and the significant contribution of AAS to the information available from the MAC-R scale alone.

Alcohol and drag use and abuse at times are an integral part of problems presented by clients at outpatient mental health clinics. These difficulties may go undetected when they are not the pri-mary reason for referral, as might be the case in pripri-mary mental health settings (Weed, Butcher, McKenna, & Ben-Porath, 1992). One of the most widely used instruments for assessing mental health problems in such settings is the MMPI-2 (Butcher, Dahl-strom, Graham, Tellegen, & Kaemmer, 1989). This study exam-ined the utility of the MMPI-2 in detecting substance-abuse prob-lems in an outpatient setting where substance abuse is not the primary focus of treatment.

The MacAndrew Alcoholism Scale (MAC; MacAndrew, 1965) of the original MMPI was developed to differentiate alcoholic from nonalcoholic psychiatric patients. The scale was constructed by contrasting the MMPI responses of 200 male alcoholics seeking treatment at an outpatient clinic with 200 male nonalcoholic psy-chiatric patients from the same facility. MacAndrew (1981) re-viewed more than two dozen studies that demonstrated the ability of the MAC scale to differentiate alcoholic from nonalcoholic participants. The scale seems to be effective with men and women, as well as inpatients and outpatients (Graham & Stronger, 1988). Gottesman and Prescott (1989) indicated that the routine use of the MAC scale to identify substance abusers is not as compelling as many users had assumed. They noted poor construct and pre-dictive validity. However, most of the studies reviewed by them used a cutoff score of 24 in discriminating abusers from nonabus-ers, whereas Graham (1993) recommended a cutoff of 28. In addition, as Graham (1993) pointed out, Gottesman and Prescott (1989) seemed to imply that professionals would decide whether or not a person abuses substances on the basis of MMPI (or

L. A. R. Stein, John R. Graham, Yossef S. Ben-Porath, and John L. McNulty, Department of Psychology, Kent State University.

Correspondence concerning this article should be addressed to L. A. R. Stein, who is now at the Center for Alcohol and Addiction Studies, Box G-BH, Brown University, Providence, Rhode Island 02912.

MMPI-2) data alone. As Graham (1993) recommended, no deci-sions should be made on the basis of MMPI (or MMPI-2) data alone, and high scores on MAC (or MAC-R) should alert clini-cians to obtain corroborating data regarding the possibility of substance abuse.

With the development of the MMPI-2, four of the original MAC items were among those eliminated from the test because of objectionable content. These items were replaced with four new items selected because they differentiated alcoholic from nonalco-holic men and women. The revised scale is labeled MAC-R (Butcher, Dahlstrom, Graham, Tellegan, & Kaemmer, 1989). In an effort to expand the assessment of alcohol and drug problems with the broader item pool of the MMPI-2, the Addiction Potential Scale (APS) and the Addiction Acknowledgment Scale (AAS) were developed (Weed et al., 1992). The APS consists of 39 items that were endorsed differently by substance abusers compared with both nonclinical and psychiatric inpatient samples. In an effort to avoid reliance on obvious content in abuse potential determination, items were eliminated from the APS if they contained obvious reference to substance abuse. The AAS, a 13-item content-based scale, was constructed, using rational item selection augmented by internal consistency procedures (Weed et al.). Items were selected for the AAS because of their obvious content relation to substance abuse and, therefore, it is a more face-valid scale than the APS.

Weed et al. (1992) presented data showing that the APS and AAS discriminated well between substance-abuse, nonclinical, and psychiatric samples and did so substantially better than the MAC-R scale. Specifically, the AAS appeared to discriminate between the substance-abuse and nonclinical samples better than the APS, whereas the APS appeared to distinguish between substance-abuse and psychiatric samples better than the AAS. Weed et al. noted that their findings were limited because the samples used in development and cross-validation of the AAS and APS came from the same settings. Greene, Weed, Butcher, Arren-dondo, and Davis (1992) found that in a different setting the APS discriminated between psychiatric inpatients and substance-abuse

samples more effectively than the AAS and that both scales discriminated better than the MAC-R scale. However, although Weed et al. found the MAC-R scale to be quite ineffective at discriminating between psychiatric and substance-abuse samples, results presented by Greene et al. (1992) indicated that the MAC-R scale discriminated between psychiatric and substance-abuse samples, effectively. In addition, the magnitude of discrim-ination by the APS in the study conducted by Greene et al. was less than in the study conducted by Weed et al.

No study to date has directly addressed the utility of the AAS and APS in assessing substance abuse in an outpatient mental health setting where substance abuse, although often of consider-able relevance, is not the primary focus of the assessment. It is important to continue efforts to validate the AAS and APS with a variety of samples, including clients in outpatient settings. In addition, as recommended by Butcher, Graham, and Ben-Porath (1995), new MMPI-2 scales should add significantly to prediction of relevant behaviors and characteristics beyond what is possible using existing scales. The purpose of this study was to explore the effectiveness of the MAC-R scale, the APS, and the AAS in the identification of substance abuse, and to examine the incremental validity of the AAS and APS in comparison to the MAC-R scale in an outpatient mental health facility where substance abuse is not the primary focus of treatment.

On the basis of the body of research available about the MAC/ MAC-R scale (for a summary of this research, see Graham, 1993), and the research regarding the APS and AAS reviewed above, we expected that the MAC-R scale, the APS, and the AAS would all be effective in identifying substance abuse. This study was con-ducted in an effort to determine which scale or scales are most effective in an outpatient setting where substance abuse is not the primary focus of treatment, and the extent to which the new MMPI-2 substance abuse scales add incrementally to the MAC-R in identifying substance abusers in this setting.

Methods

Participants

Data were collected at a large community mental health center (CMHC) in Northeast Ohio. A variety of treatment programs was available for the clients, including partial hospitalization and individual and group outpa-tient treatment. Clients with substance abuse as a primary problem were referred elsewhere. However, the CMHC served dually diagnosed clients who did not have a substance-abuse disorder as the primary diagnosis. A specialized program was available for these clients involving individual counseling, psychoeducational group intervention, family therapy, support groups, and referral to Alcoholics Anonymous, Narcotics Anonymous, and Al-Anon. On the basis of an in-depth interview during intake, and where possible, previous treatment records, the determination was made as to whether substance abuse was a primary or secondary problem.

The sample used in this study was a subset of a larger sample collected by Graham, Ben-Porath, and McNulty (in press). The larger sample in-cluded all persons seeking services at the mental health center from April 1991 through December 1992 and included 1,035 men and 1,447 women. Out of a total of 2,482 clients seeking services at the CMHC, 1,263 participants did not receive services beyond intake and so did not complete the MMPI-2. Valid MMPI-2 profiles were defined as follows: Cannot say raw score <31, Variable Response Inconsistency Scale T score <81, and True Response Inconsistency Scale raw score between 6 and 12, inclusive (Butcher et al., 1989); and F raw score <29 for women, F raw score <27

for men, Back F raw score <24 for women, and Back F raw score <23 for men (Graham, Watts, & Timbrook, 1991).

Clients who produced a valid MMPI-2 and for whom intake ratings were available concerning relevant substance-abuse data were identified as candidates for this study. Of the 501 men and the 718 women who completed the MMPI-2, 410 men and 610 women produced valid profiles. Of the 1,020 clients who produced valid MMPI-2s, 77 men and 110 women did not have completed intake ratings regarding substance-abuse history and were eliminated. The current study's sample consisted of 500 women and 333 men who produced a valid MMPI-2 profile and for whom substance-abuse data were available. Demographic characteristics and mental health history of the participants are presented in Table 1. These data were collected at the time of the intake interview and were gathered through self-report, and where possible, corroborated by previous treat-ment records.

For participants producing a valid MMPI-2, comparisons were made on the basis of demographic characteristics and mental health history between clients included in the study (those who had a valid MMPI-2 and relevant substance-abuse data) and clients excluded from the study (those who produced a valid MMPI-2 but for whom substance-abuse data were not available). On the demographic and mental health variables as presented in Table 1, we found no significant differences between those clients included in and those excluded from the study.

Table 1 Characteristics of Participants Characteristic Demographics Age M SD Years of education M SD Ethnicity (%) Caucasian African American/Other Marital status (%) Never married Divorced Married Separated Widowed Employment status (%) Unemployed Full time Part time Disabled Other Mental Health Previous psychiatric hospitalizations (%) Previous outpatient treatment (%) Any Axis I diagnosis (%) Specific Axis I diagnoses (%)

Adjustment disorder Depression Anxiety disorders

Substance abuse/dependence Any Axis II diagnosis (%) Axis V current level of functioning

M SD Women (n = 500) 32.82 10.08 12.10 2.10 79 21 35 26 23 13 3 47 18 13 4 18 27 58 99 32 30 16 12 30 62.83 10.08 Men (n = 333) 33.63 10.93 11.98 2.03 78 22 47 23 17 11 2 43 27 11 7 12 29 44 99 25 20 16 19 27 63.31 10.04

Instruments

MMPl-2. The MMPI-2 (Butcher et al., 1989), a revised and updated

version of the original MMPI (Hathaway & McKinley, 1940), is a 567-item personality inventory. The MMPI-2 includes the validity and clinical scales of the original MMPI, as well as new content, supplementary, and validity scales. Adequate internal consistencies and test-retest reliabilities of the MMPI-2 scales are reported in the test manual (Butcher et al., 1989) along with preliminary validity information.

Internal-consistency coefficients on MAC-R are a = .45 for women and

a = .56 for men, based on the normative sample (Butcher et al., 1989).

Also on the basis of the normative sample, test-retest reliabilities for a 1-week interval are r = .78 for women and r = .62 for men on MAC-R (Butcher et al., 1989). Although Weed et al. (1992) reported no internal-consistency coefficient for APS on a combined sample of substance abus-ers, psychiatric patients, and normative subjects, a = .74 for AAS (Gra-ham, 1993). According to Weed et al., on the basis of the normative sample, test-retest reliabilities for a 1-week interval are r = .77 for women and r = .69 for men on APS and r = .84 for women and r = .89 for men on AAS. Although they share no items, the Pearson product-moment correlation coefficient between AAS and APS is r = .33 for normative participants, r = .33 for a psychiatric sample, and r = .36 for a substance-abuse sample (Weed et al.).

Intake form. An intake form was designed for the larger study (Graham

et al., in press) and was completed by a trained intake worker on the basis of a personal interview with each client.' Intake workers consisted of psychologists, nurses, and social workers and were highly experienced in interviewing. Data from the intake form included in this study were the demographic information (e.g., age, education, marital status, employment status, and race), selected information concerning mental health history (e.g., psychiatric hospitalizations and outpatient treatment), diagnosis, and substance-abuse history. Intake workers inquired regarding participants' past and current experience with alcohol, marijuana, cocaine, heroin, and other substances. For each substance, ratings were made on a 4-point Likert scale as follows: 1 (no use), 2 (some use), 3 (possible abuse), and 4

(definite abuse).

During several training sessions, intake workers were instructed to limit the definite abuse category to those clients for whom they had great confidence that substance abuse was a problem. The possible abuse cate-gory was to be used with clients for whom there was strong indication of substance abuse, but for whom intake workers were unsure as to whether a rating of definite abuse applied. The some-use rating was to be used when some substances had been used but not to levels that had impacted the client's life adversely. The no-use category was reserved for clients who abstained from substances. Before substance-abuse ratings were made on the intake form, specific information regarding past and present use of specific substances was obtained. Information was gained through an in-depth interview, and where possible, previous treatment records. Inquir-ies regarding substance abuse dealt with frequency, amount, and duration for specific substances, as well as negative effects of use.

Procedure

Intake forms were completed following an extensive interview con-ducted at the time clients requested services. Interviewers were instructed to structure the interview around the intake form. Each client completed the MMPI-2 shortly after an intake interview. The median number of days between intake and completion of the MMPI-2 was 7. Ratings of each of the specific substances were combined into a single substance-abuse index by assigning the maximum rating for any substance as the overall indicator of substance abuse. For example, if a client had abuse ratings of 1 for alcohol, 3 for marijuana, 2 for cocaine, 1 for heroin, and 3 for other substances, an overall substance-abuse rating of 3 was recorded for that client. Participants were assigned to one of four substance use groups based on this variable. For women, the average substance-abuse rating was

M = 2.3 and SD = 0.93 (with Mdn = 2 and mode = 2), whereas, for men, M = 2.6 and SD = 0.99 (with Mdn = 2 and mode = 2).

Results

Table 2 presents mean scores and standard deviations on the MAC-R, AAS, and APS for participants grouped by substance-abuse rating. T scores were derived based on the normative sample (see Butcher et al., 1989). As reported in Table 2, analysis of variance (ANOVA) indicated, for both genders, that the four groups differed significantly for each substance-abuse scale. Ta-ble 2 also reports the results of post hoc t tests and their associated effect sizes. For both genders and all three scales, differences between no use and all other substance-abuse groups were statis-tically significant. Similarly, for both genders and for all three scales, differences between some use and definite abuse were statistically significant. Examining data for women and men, for both the AAS and MAC-R scale but not for the APS, differences between possible abuse and definite abuse were statistically sig-nificant. For both genders, only for the AAS was the difference between some use and possible abuse statistically significant. In addition, for both genders, effect sizes were generally largest for

AAS.

Before conducting a series of hierarchical regressions, intercor-relations among MAC-R, AAS, and APS scores, and substance-abuse ratings were examined (see Table 3). The positive but modest correlations between scales suggest that they are not com-pletely redundant. For men and women, the highest correlations were between AAS and MAC-R (r = .49 for women; r = .45 for men) and between AAS and substance-abuse rating (r = .52 for women; r = .50 for men). These data indicate that although AAS and MAC-R are moderately intercorrelated, the ability of AAS to add to the identification of substance abuse beyond MAC-R is not attenuated substantially by shared variance among these scales.

To test the incremental contribution of the APS and the AAS to the identification of substance abuse, we conducted a series of hierarchical regression analyses, with substance-abuse rating as the dependent variable. As reported in Table 4, the independent vari-ables were entered in three blocks. The MAC-R scale was entered in the first block. AAS was entered in the second block to test its incremental contribution in predicting substance abuse beyond MAC-R. APS was entered in the third block to test its incremental contribution beyond MAC-R and AAS. For men and women, AAS added significantly to the variance in substance abuse ac-counted for by the MAC-R scale alone, whereas the addition of the APS produced a significant contribution for women only. In the second set of analyses presented in Table 5, the order of entry for APS and AAS was reversed. For both genders, the addition of both APS and AAS contributed significantly to the proportion of vari-ance in rated substvari-ance abuse.

To evaluate the accuracy with which the substance-abuse scales could classify participants as abusers or nonabusers, we calculated the positive and negative predictive powers, sensi-tivity, specificity, and overall classification accuracy of the

1 A copy of the intake form is available, on request, from L. A. R. Stein,

Center for Alcohol and Addiction Studies, Box G-BH, Brown University, Providence, Rhode Island 02912.

Table 2

Substance-Abuse Scores for Four Substance-Abuse Groups Substance-abuse group

No use Some use Possible abuse Definite abuse

Scale" M SD M SD M SD M SD df Significant group differences'" Women MAC-R 18.80 3.32 85 20.34 4.18 267 21.27 4.03 67 23.06 4.73 81 15.95** 3,496 a (0.39), b (0.68), c (0.22), e (0.63), f (0.40) AAS 49.77 9.29 85 56.34 10.62 267 63.19 11.38 67 71.19 13.97 81 59.86** 3,496 a (0.64), b (1.31), c (1.81), d (0.64), e(1.29), f (0.62) APS 47.28 9.69 85 51.91 10.05 267 55.09 9.61 67 56.10 10.73 81 12.77** 3,496 a (0.46), b (0.81), c (0.86), e (0.41) Men MAC-R 19.41 3.86 32 22.54 4.11 153 23.50 4.00 58 25.84 4.92 90 20.87** 3,329 a (0.77), b (1.04), c (1.38), e (0.75), f (0.51) AAS 46.22 7.69 32 52.99 9.83 153 60.31 10.15 58 65.18 13.34 90 36.80** 3,329 a (0.71), b (1.51), c (1.56), d(0.74), e (1.08), f (0.40) APS 45.34 10.11 32 51.26 10.97 153 55.17 10.42 58 56.60 10.47 90 11.05** 3,329 a (0.55), b (0.95), c (1.09), e (0.50) Note. MAC-R = MacAndrew Alcoholism Scale—Revised; AAS = Addiction Acknowledgment Scale; APS = Addiction Potential Scale. " MAC-R is in raw score as is traditionally the case (see Graham, 1993); AAS and APS are in linear T scores. b Significant group differences are as follows: a = mean

MMPI-2 scale scores are significantly different (p £ .022) between no use and some use groups; b = mean MMPI-2 scale scores are significantly different (p £ .001) between no use and possible abuse groups; c = mean MMPI-2 scale scores are significantly different (p £ .001) between no use and definite abuse groups; d = mean MMPI-2 scale scores are significantly different (p £ .001) between some use and possible abuse groups; e = mean MMPI-2 scale scores are significantly different (p ^ .006) between some use and definite abuse groups; f = mean MMPI-2 scale scores are significantly different (p •& .042) between possible abuse and definite abuse groups. Numbers within parentheses are effect sizes for the post hoc tests and are expressed as d.

**p < .001.

scales, using data from those participants about whom we had definitive data. As stated above, all participants were rated on a 4-point scale. Participants rated at the two extremes of this scale (no use and definite abuse) were clearly nonabusers or abusers of substances. Because of ambiguity regarding those rated at the two intermediate levels (some use and possible abuse), they were not included in the classification analyses. Although this procedure reduced the sample size (N = 166 for women; N = 122 for men), it eliminated ambiguity resulting in

Table 3

Pearson Product-Moment Correlation Coefficients Between Substance-Abuse Scales and Substance-Abuse Rating

Scale MAC-R AAS APS Rating MAC-R _ 0.45*** 0.36*** 0.39*** AAS 0.49*** — 0.37*** 0.50*** APS 0.27*** 0.29*** — 0.29*** Rating 0.30*** 0.52*** 0.26*** — Note. Correlations involving MacAndrew Alcoholism Scale—Revised (MAC-R) were computed using raw scores, those for Addiction Acknowl-edgment Scale (AAS) and Addiction Potential Scale (APS) were computed using linear T scores. Correlations above the diagonal refer to women, those below the diagonal refer to men.

***/> < .0005.

a more precise evaluation of the classification accuracy of the MMPI-2 substance abuse scales.

Table 6 shows the classification accuracy analyses using the Table 4

Predicting Substance Abuse From Hierarchical Regression Analyses: Addiction Potential Scale (APS) Entered Last

Scales entered8 MAC-R AAS APS MAC-R AAS APS R .30 .52 .53 .39 .53 .54 K2 .09 .27 .28 .15 .28 .29 p2 "adjusted Women .09 .27 .27 Men .15 .28 .28 ANOVA F(l,498) = 47.31** F(2, 497) = 90.75** ''change ~~ l££.VJ F(3, 496) = 63.87** Change = 7-68* f(l, 331) = 58.94** F(2, 330) = 65.31** ^change = 60.94** F(3, 329) = 44.51** Change = 2.37

Note. ANOVA = analysis of variance. a MacAndrew Alcoholism

Scale—Revised (MAC-R) is in raw score; Addiction Acknowledgment Scale (AAS) and APS are in linear T score.

Table 5

Predicting Substance Abuse From Hierarchical Regression Analyses: Addiction Acknowledgment Scale (AAS) Entered Last

Scale entered" MAC-R APS AAS MAC-R APS AAS R .30 .35 .53 .39 .42 .54 R2 .09 .12 .28 .15 .18 .29 "adjusted Women .09 .12 .27 Men .15 .17 .28 ANOVA F(l, 498) = 47.31** F(2, 497) = 33.95** Change = 18.90** F(3,496) = 63.87** Change = 108.96** F(l, 331) = 58.99** F(2, 330) = 35.65** F = 1 0 S9** r change 1U.J7 F(3, 329) = 44.51** Fchange = 51.36**

Note. ANOVA = analysis of variance. " MacAndrew Alcoholism Scale—Revised (MAC-R) is in raw score; AAS and Addiction Potential Scale (APS) are in linear T score. ** p < .001.

resulting base rate of 49% for women and base rate = 74% for men. These percentages reflect the base rate of substance abuse found in the smaller sample of participants for whom

substance-abuse ratings were unambiguous (i.e., ratings of 1 or 4). Generally, overall hit rates for women and men were rather poor for the MAC-R and APS (ranging from 47% to 69%). For women, T score >60 on AAS yielded the best overall hit rate (86%) with a sensitivity of 85%, a specificity of 87%, and positive and negative predictive powers of 86% each. For men, T score a60 on AAS yielded the best overall hit rate (76%) with a sensitivity of 71%, a specificity of 91%, and positive and negative predictive powers of 96% and 53%, respectively.

Because the actual base rates obtained in the narrower sample were unrealistically high, analyses were again conducted on the participants for whom substance-abuse ratings were unambiguous; however, base rates were adjusted (Meehl & Rosen, 1955) to reflect the base rate of substance abuse found in the larger sample of participants (N = 500 for women; N — 333 for men). These results are reported in Table 7. Generally, overall hit rates were acceptable for both women and men on MAC-R and APS (ranging from 76% to 86%). For women, T score >65 on AAS yielded the best overall hit rate (92%) with a sensitivity of 69%, a specificity of 96%, and positive and negative predictive powers of 79% and 94%, respectively. For men, T score 5:60 on AAS yielded the best overall hit rate (85%) with a sensitivity of 71%, a specificity of 91%, and positive and negative predictive powers of 74% and 89%, respectively.

Table 6

Sensitivity, Specificity, Positive Predictive Power (PPP), Negative Predictive Power (NPP), and Overall

Hit Rate in Percentages

Table 7

Sensitivity, Specificity, Positive Predictive Power (PPP), Negative Predictive Power (NPP), and Overall

Hit Rate in Percentages Scale cutoff3 MAC-R >24 >28 AAS >60 >65 APS >60 >65 MAC-R >24 >28 AAS >60 >65 APS >60 >65 Sensitivity 36 14 85 69 46 21 61 33 71 54 48 29 Specificity Women 93 100 87 96 91 96 Men 81 100 91 91 91 97 PPP 83 100 86 95 82 85 90 100 96 94 93 96 NPP 60 55 86 77 64 56 43 35 53 41 38 33 Overall hit rate 65 58 86 83 69 60 66 51 76 64 59 47 Scale cutoff" MAC-R >24 >28 AAS >60 >65 APS >60 >65 MAC-R >24 >28 AAS >60 >65 APS >60 >65 Sensitivity 36 14 85 69 46 21 61 33 71 54 48 29 Specificity Women 93 100 87 96 91 96 Men 81 100 91 91 91 97 PPP 50 100 56 79 48 53 55 100 74 68 65 77 NPP 88 86 97 94 90 86 85 80 89 84 82 79 Overall hit rate 84 86 87 92 83 84 76 82 85 81 79 79 Note. For women, base rate = 49%, and for men, base rate = 74%.

" MacAndrew Alcoholism Scale—Revised (MAC-R) raw score >24 is based on Gottesman and Prescott (1989); MAC-R raw score >28 is based on Graham (1993). Addiction Acknowledgment Scale (AAS) and Addic-tion Potential Scale (APS) linear T score >60 is based on Graham (1993); AAS and APS linear T score a65 is based Butcher et al. (1989).

Note. For women, base rate = 16%, and for men, base rate = 27%. "MacAndrew Alcoholism Scale—Revised (MAC-R) raw score >24 is based on Gottesman and Prescott (1989); MAC-R raw score >28 is based on Graham (1993). Addiction Acknowledgment Scale (AAS) and Addic-tion Potential Scale (APS) linear T score £60 is based on Graham (1993); AAS and APS linear T score >65 is based Butcher et al. (1989).

Discussion

Scores on the three MMPI-2 substance abuse scales, MAC-R, APS, and AAS, were related to interviewer ratings of substance abuse in.this outpatient mental health sample. The three scales differentiated significantly between individuals identified by ex-perienced interviewers as evidencing varied levels of substance abusive behavior. Hierarchical regression analyses indicated that AAS added significantly and substantially to MAC-R and APS in predicting variance in substance abuse and that APS added only modestly to MAC-R on this task in this sample. Although classi-fication accuracy analyses indicated that MAC-R and APS yielded acceptable overall hit rates, AAS produced the best classification rates in this sample.

MAC-R explained a modest, but significant amount of variance in substance use and abuse (about 10% and 15% for women and men, respectively). For women and men, AAS added substantially to the amount of variance explained by MAC-R alone (variance increased by 18% and 13% for women and men, respectively). The addition of APS to MAC-R and AAS did not add as much information to the amount of variance in substance abuse ex-plained. Similarly, whereas the addition of APS to MAC-R was statistically significant, variance increased by only 3% for women and 2% for men. Finally, AAS predicted a statistically significant and substantial proportion of additional variance beyond MAC-R and APS (variance increased by 15% and 11% for women and men, respectively). The amount of variance in substance use and abuse explained by all three scales in the regression equation was a little less than 30%, which suggests that these scales have practical utility in screening for substance abusive behavior in outpatient mental health settings.

The overall hit rates obtained in the classification accuracy analyses were higher when base rates were adjusted to reflect the base rate of substance abuse in the total sample (N = 500 for women; N = 333 for men). As compared with base rate = 49% for women and base rate = 74% for men, adjusted base rates of 16% and 27% for women and men, respectively, may be more realistic estimates of the base rates of substance abuse found in outpatient settings. For both genders, a cutoff of 28 provided the highest overall accurate classification rate on MAC-R. The positive and negative predictive powers were high; however, sensitivity was poor for both genders. For women, optimal classification accuracy was found on AAS at a T-score cutoff of 65, whereas for men, the optimal r-score cutoff was 60 on AAS. For both genders, all four accuracy indicators were acceptably high. Overall classification accuracy was equal at T-score cutoffs of 60 and 65 for both women and men on APS. However, positive predictive power was poor for women at both cutoffs and acceptable for men only at a cutoff of 65.

These classification accuracy analyses indicate that an elevated score on any of these scales in outpatient mental health settings should alert clinicians to obtain more data regarding the possibility of substance abuse. The higher the score, the greater the possibility of substance-abuse problems. The hierarchical regression results indicate that scores on these scales are not completely redundant, and therefore, each of the scales warrants examination when screening for a possible substance-abuse problem in outpatient mental health clients.

The finding that AAS demonstrated the greatest effect size in this sample warrants further consideration. This brief, face valid content-based scale consists primarily of items related overtly to substance abusive behavior. As its label implies, individuals with elevated scores on this scale are acknowledging problems in the area of substance abuse. The current results are consistent with previous research, indicating that content-based scales can provide incrementally valid information when compared with empirically keyed scales. Archer, Elkins, Aiduk, and Griffin (1997); Ben-Porath, Butcher, and Graham (1991); and Ben-Porath, Mc-Cully, and Almagor (1993) have reported similar findings for the MMPI-2 Content Scales compared with the empirically keyed clinical scales. The finding that AAS adds incrementally to the empirically keyed MAC-R and APS scales provides further evi-dence of the importance of considering item content themes in MMPI-2 interpretation.

As is the case with the MMPI-2 Content Scales, the transparent nature of the AAS items imposes limitations on its interpretability. Although this study did not address the susceptibility of the sub-stance abuse scales to distortion, for obvious reasons, content scales may be more susceptible to intentional distortion than empirically keyed ones. Therefore, it is of particular importance to examine scores on the validity scales of the MMPI-2 before interpreting AAS. A defensive test-taking approach is likely to result in artificially low scores on AAS and other content scales. Moreover, even if a general defensive approach to the test is not indicated, it remains possible that some test-takers would be re-luctant to acknowledge problems specifically in the area of sub-stance abuse. Therefore, the absence of elevation on this scale cannot be taken as a negative indicator of substance abuse. Thus, higher scores on AAS can be used to suggest the possibility of substance-abuse difficulties in outpatient mental health settings; however, low scores on this scale cannot be used to rule this problem out. Future studies may address the effects of defensive test-taking style on the substance abuse scales.

Findings regarding APS indicate a significant, but perhaps, more limited role for this scale in detecting substance-abuse prob-lems in outpatient settings. Results for this scale were generally comparable to those of the MAC-R scale. Unlike the findings reported by Weed et al. (1992), the present results do not point to a substantial contribution of APS beyond MAC-R in identifying substance abuse in this setting. Population differences may account for the discrepant findings. Participants in the Weed et al. study came from two different types of settings: an inpatient facility devoted exclusively to substance-abuse treatment and a number of inpatient psychiatric units. It is possible that APS is better able to differentiate between individuals more clearly identified as having either substance-abuse or significant psychiatric difficulties than it can distinguish among outpatients, all receiving mental health services and some of whom have secondary substance-abuse problems.

A limitation of our study concerns the broad definition used for rating substance abuse. Our substance-abuse variable did not mea-sure a specific class of substances. Significant levels of comorbid-ity precluded breaking our sample down into mutually exclusive alcohol- or drug-abuse subsamples. Consequently, the ability of the MMPI-2 substance abuse scales to identify alcohol-only or drug-only abuse could not be examined here and remains to be studied in future investigations. Another limitation of this study is

that both interrater reliability and temporal stability data were unavailable regarding substance-abuse ratings by intake workers. It was simply unrealistic for intake workers at this large commu-nity mental health center to conduct two interviews per client on a subsample of participants for this field study. In addition, future studies may measure substance use and abuse using methods other than intensive interview data. For example, standardized instru-ments may be used as may reports from significant others.

In summary, the current findings indicate that scores on the MMPI-2 substance abuse scales are related to rated substance abuse in an outpatient mental health setting. These results illustrate that the utility of the recently developed AAS and APS scales generalizes beyond inpatient settings where they were initially constructed and studied. Identifying potential substance-abuse problems in individuals referred primarily for outpatient mental health services is important to the development of a successful treatment plan. The MMPI-2 substance abuse scales can play a significant role in this process. There may be times when a client is not honest in an interview but is honest on a test that does not involve face to face contact, and this argues for the usefulness of testing. This study depended in part on the truthfulness of the client and the skill of the interviewer in determining substance abuse, even when perhaps the client was reluctant to divulge information. However, there may be times when a more skillful and experi-enced interviewer is unavailable to conduct a thorough interview, and testing can alert professionals to areas needing more attention. The MMPI-2 can also assist in streamlining following interviews. This study indicates that the MMPI-2 is useful in screening for substance abuse in outpatient mental health settings where sub-stance abuse is not the primary reason for referral.

References

Archer, R. P., Elkins, D. E., Aiduk, R., & Griffin, R. (1997). The incre-mental validity of MMPI-2 Supplementary scales. Assessment, I,

193-205.

Ben-Porath, Y. S., Butcher, J. N., & Graham, J. R. (1991). Contribution of the MMPI-2 Content scales to the differential diagnosis of schizophrenia and major depression. Psychological Assessment, 3, 634-640. Ben-Porath, Y. S., McCully, E., & Almagor, M. (1993). Incremental

validity of the MMPI-2 Content scales in the assessment of personality

and psychopathology by self-report. Journal of Personality Assess-ment, 61, 557-575.

Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaem-mer, B. (1989). MMPI-2: Manual for administration and scoring. Min-neapolis: University of Minnesota Press.

Butcher, J. N., Graham, J. R., & Ben-Porath, Y. S. (1995). Methodological problems and issues in MMPI, MMPI-2, and MMPI-A research. Psy-chological Assessment, 7, 320—329.

Gottesman, I. I., & Prescott, C. A. (1989). Abuses of the MacAndrew MMPI Alcoholism scale: A critical review. Clinical Psychology Re-view, 9, 223-242.

Graham, J. R. (1993). MMPI-2: Assessing personality and psychopathol-ogy (2nd ed.). New York: Oxford University Press.

Graham, J. R., Ben-Porath, Y. S., & McNulty, J. L. (in press). Using the MMPI-2 in outpatient mental health settings. Minneapolis: University of Minnesota Press.

Graham, J. R., & Stronger, V. E. (1988). MMPI characteristics of alcohol-ics: A review. Journal of Consulting and Clinical Psychology, 56,

197-205.

Graham, J. R., Watts, D., & Timbrook, R. E. (1991). Detecting fake-good and fake-bad MMPI-2 profiles. Journal of Personality Assessment, 57, 264-277.

Greene, R. L., Weed, N. C., Butcher, J. N., Arrendondo, R., & Davis, H. G. (1992). A cross-validation of MMPI-2 Substance Abuse scales. Journal of Personality Assessment, 58, 405-410.

Hathaway, S. R., & McKinley, J. C. (1940). A multiphasic personality schedule (Minnesota): I. Construction of the schedule. Journal of Psy-chology, 10, 249-254.

MacAndrew, C. (1965). The differentiation of male alcoholic outpatients from nonalcoholic psychiatric outpatients by means of the MMPI. Quar-terly Journal of Studies on Alcohol, 26, 238-246.

MacAndrew, C. (1981). What the MAC scale tells us about alcoholics: An interpretive review. Journal of Studies on Alcohol, 42, 604-625. Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the

effi-ciency of psychometric signs, patters, or cutting scores. Psychological Bulletin, 52, 194-216.

Weed, N. C., Butcher, J. N., McKenna, T., & Ben-Porath, Y. S. (1992). New measures for assessing alcohol and drug abuse with the MMPI-2: The APS and AAS. Journal of Personality Assessment, 58, 389-404.

Received April 7, 1998 Revision received October 22, 1998 Accepted October 26, 1998