Stressing About Posttraumatic Stress Disorder

(1)

LETTERS TO THE EDITOR

Statements appearing here are those of the writers and do not represent the ofﬁcial position of the American Academy of Pediatrics or its Committees. Comments on any topic, including the contents of PEDIATRICS, are invited from all members of the profession; those accepted for publication will not be subject to major editorial revision but generally must be no more than 400 words in length. The editors reserve the right to publish replies and may solicit responses from authors and others.

Please see www.pediatrics.org for instructions on submitting letters.

Stressing About Posttraumatic

Stress Disorder

To the Editor.—

We are writing to you about a recent article that ap-peared inPediatrics.1_{The authors of this very interesting}

article claimed that posttraumatic stress disorder (PTSD) “symptoms and cortisol levels at baseline are associated with changes in hippocampal volume over an ensuing 12- to 18-month interval” after a stressful life event. However, we believe that this strong claim is not sup-ported by the data the authors presented.

The first problem is that the methods the authors used to measure hippocampal volume are of low precision. In all studies that use MRI to measure brain volume, ex-perimental variance can be introduced during data ac-quisition (eg, patient motion, changes in scanner hard-ware or softhard-ware, scanner field variation) and during data analysis (eg, partial-volume problems, voxel mis-classification, manual delineation error).2 _{Among the}

15 children who they evaluated, the authors acknowl-edged (in line 3 of “Results”) that there was 1 child who showed an increase in right hippocampal volume that amounted to a full SD, or ⬃15%. This child was ex-cluded from additional evaluation. However, the exis-tence of 1 outlier this extreme suggests that the method of measuring hippocampal volume was relatively impre-cise. This inference is consistent with the fact that the authors measured interrater reliability but did not report the results. If we hypothesize that the volume-measure-ment methods used were imprecise and that hippo-campal volume of these subjects did not actually change over the time interval studied, what results would have been obtained? We would expect that hippocampal vol-ume change scores would have fluctuated randomly and would not have been significantly different from 0. This is precisely the outcome shown in Table 1, in which change scores are all small and nonsignificant. The only way to compensate for measurement imprecision is to increase the size of the study population.2

The second problem is that the sample of 15 children evaluated was far too small to provide meaningful re-sults. We have characterized the precision of measure-ment of brain volume in a cohort of 52 healthy adults imaged twice.3 _{Our goal was to determine the sample}

size needed to provide adequate statistical power in stud-ies of brain volume by MRI. We imaged volunteers at weeks 0 and 12 using the same scanners and the same methods at both time points. We assumed that there would be no change in brain volume in the absence of an intervening neurologic event. Sample sizes necessary to yield 80% statistical power to detect a 5% change in brain volume were calculated for several experimental designs. The percentage difference between the 2 sets of 19 measurements averaged just 0.18%, but the required sample sizes, nevertheless, were far larger than antici-pated. For example, in a 2-sample (patients versus con-trols) longitudinal study of whole brain volume, the required sample size is 10 subjects, although whole brain does not change in young adults and can be mea-sured with much more precision than volume of the hippocampus. Our findings suggest that there may be more uncertainty than anticipated in MRI volumetric studies. We did not report results for the hippocampus in the accepted article, because the required sample sizes were so large as to be prohibitive. As far as we can tell, our methods were similar to the methods used here, including scanner-field strength and image-analysis methodology. Our unpublished results for the hippo-campus are shown in Table 1.

The results shown suggest that, for a longitudinal study of hippocampal volume in adult patients and con-trols, a minimum sample size of 58 to 66 subjects is required for 80% statistical power, even if we assume that there is a 5% change in patient hippocampal vol-ume. Our results suggest that the Carrion et al study1

had only⬃25% of the subjects needed for 80% statisti-cal power, even if hippocampal volume had changed by 5%, which it did not (average change reported in this study was⫹2.9%). It could be argued that the authors

232 LETTERS TO THE EDITOR

at Viet Nam:AAP Sponsored on August 29, 2020 www.aappublications.org/news

(2)

actually presented a longitudinal 1-group study, which would require a sample size of only 16 to 18 subjects, but we think that such a study design is grossly inappropri-ate in subjects who are likely to be undergoing volumet-ric brain growth.

The third problem is that no data from healthy con-trols were presented, so we have no way of knowing what “normal” looks like by the methods that the au-thors used. This is a particularly serious error of omission in the case of children (the average age of subjects in their study was 10.4 years [range: 8 –14 years]), in whom brain volume is likely to be changing.4

Further-more, it would be a mistake to assume that hippocampal volume must be increasing simply because brain volume is still increasing in children under 12 years of age.5_In

the absence of control data, we cannot assume that the reported decrease in hippocampal volume was abnor-mal. We note that it could be argued that analysis in this article was entirely focused on whether changes in hip-pocampal volume are correlated with PTSD symptoms, so that brain changes in healthy controls are irrelevant. Yet, such an analysis would require a sophisticated use of statistics.

A fourth problem is that there were several problem-atic uses of statistical tests. For example, there were 8 reported hippocampal measures (see their Table 2), and all 8 measures were tested for correlation with PTSD symptoms, with hyperarousal, and with cortisol levels. This means that 24 separate tests were performed with-out correction for multiple comparisons. If correction for multiple comparisons was performed, we think that none of the reported results would be significant. Also, Pearson correlation coefficients were used to compare hippocampal volume at baseline and at follow-up. There are several ways to determine the equivalence of multi-ple sets of measurements, or the extent to which 1 set of measurements can be substituted for the other. Pearson correlation coefficients are a particularly poor choice, because they only reflect the extent to which 2 sets of measurements are linearly related. More appropriate measures are intraclass or concordance correlations, be-cause they reflect the extent to which measurements are fungible, or invariant in terms of mean and SD. For

example, if 2 sets of brain-volume measurements are available, and all volumes in 1 set are exactly fivefold larger than those in the other set, the Pearson correla-tion would be 1.00, whereas a concordance correlacorrela-tion would be much lower, showing that the 2 sets of mea-sures are not interchangeable. Last, an outlier was arbi-trarily deleted from analysis when the statistical test used was not sensitive to outliers. Once a decision has been made to use Spearman correlations, there is no need to exclude outliers; all scores are turned into ranks, so a large change in absolute value has little impact.

A final problem with this article is that, although it was not rigorously done, it confirms 1 of 2 competing theories about how PTSD affects the hippocampus in children. As such, it may be given inordinate weight, although it is a weak study. An earlier longitudinal study of children that involved 18 subjects (rather than the 15 reported here) concluded that hippocampal volume did not differ between PTSD patients and controls at base-line, at follow-up, or over time.6 _{An earlier study by}

Carrion et al7_{also failed to find a difference in}

hippocam-pal volume between pediatric patients with PTSD and controls. Among 37 adult trauma survivors who were imaged prospectively within 1 week of trauma, there was no significant difference between the subjects who did and did not develop PTSD.8 _{Given that at least 1}

earlier study included both pediatric patients and con-trols and followed the patients for a longer time period,6

the weight of evidence suggests that PTSD is not associ-ated with a change in hippocampal volume, at least among children.

We agree with Carrion et al that longitudinal studies are inherently stronger than cross-sectional studies and that the effects of PTSD on hippocampal volume in chil-dren can only be understood in the context of a strong longitudinal study. However, we do not believe that an underpowered, uncontrolled, and overanalyzed study is a step toward that goal.

R. Grant Steen, PhD Department of Psychiatry Robert M. Hamer, PhD Departments of Psychiatry and Biostatistics University of North Carolina Chapel Hill, NC 27599-2719

ACKNOWLEDGMENTS

Dr Steen was supported by the National Alliance for Research on Schizophrenia and Depression as a Hof-mann Trust Investigator.

REFERENCES

1. Carrion VG, Weems CF, Reiss AL. Stress predicts brain changes in children: a pilot longitudinal study on youth stress, posttrau-matic stress disorder, and the hippocampus.Pediatrics.2007;119: 509 –516

2. Steen RG, Mull C, McClure R, Hamer RM, Lieberman JA. Brain

TABLE 1 Minimum Required Sample Size (PatientsⴙControls) for at Least 80% Statistical Power, Under a Variety of Assumptions About Study Design

Structure Cross-sectional: 2 Groups

Longitudinal 1 Group 2 Groups

(Change)

N Power N Power N Power

Left hippocampus 250 80.2 16 80.9 58 81.0 Right hippocampus 276 80.0 18 80.5 66 80.6

The estimated statistical power is shown for a sample size that was calculated to yield at least 80% power.

PEDIATRICS Volume 120, Number 1, July 2007 233

(3)

volume in first-episode schizophrenia: systematic review and meta-analysis of magnetic resonance imaging studies.Br J Psy-chiatry.2006;188:510 –518

3. Steen RG, Hamer RM, Lieberman JA. Measuring brain volume: the impact of precision on sample size in magnetic resonance imaging studies.Am J Neuroradiol.2007; In press

4. Castellanos FX, Lee PP, Sharp W, et al. Developmental trajecto-ries of brain volume abnormalities in children and adolescents with attention-deficit/hyperactivity disorder.JAMA.2002;288: 1740 –1748

5. Tupler LA, De Bellis MD. Segmented hippocampal volume in children and adolescents with posttraumatic stress disorder.Biol Psychiatry.2006;59:523–529

6. De Bellis MD, Hall J, Boring AM, Frustaci K, Moritz G. A pilot longitudinal study of hippocampal volumes in pediatric mal-treatment-related posttraumatic stress disorder.Biol Psychiatry.

2001;50:305–309

7. Carrion VG, Weems CF, Eliez S, et al. Attenuation of frontal asymmetry in pediatric posttraumatic stress disorder.Biol Psychi-atry.2001;50:943–951

8. Bonne O, Brandes D, Gilboa A, et al. Longitudinal MRI study of hippocampal volume in trauma survivors with PTSD. Am J Psychiatry.2001;158:1248 –1251

doi:10.1542/peds.2007-0867

In Reply.—

Drs Steen and Hamer raise 5 questions about our recent article inPediatrics1_{: (1) the reliability of the}

hippocam-pus measurement; (2) sample size; (3) lack of a control group; (4) the number and type of statistical tests; and (5) the theoretical implications of its consistency with 1 of 2 competing hypotheses. We appreciate their interest in our work and address each of these issues below.

Regarding the issue of hippocampal reliability, we had strong interrater reliability with a 0.95 intraclass corre-lation coefficient. Our laboratory has published on hip-pocampal measurement approaches to address the in-herent challenges of measuring small volumes (see ref 2). Imaging studies can always be affected by software and hardware; however, our laboratory has maintained backward compatibility with previous scanning tech-niques, and we routinely examine the effects of hard-ware or major softhard-ware upgrades on image quality and characteristics to understand if there are any changes. We use the same scanner hardware and identical pulse sequences and software processing techniques. For ex-ample, we exclude scans that are technically unaccept-able because of motion or other artifacts.

The second issue addresses sample size. We agree that any study can be improved with a larger sample size. Recognizing the limitation, we called it a pilot study. Steen and Hamer, however, raise the issue of power. Their comments do not apply to our findings, because the effect size obtained was large enough to meet the criteria set for statistical significance in the study. Their comments, however, do provide an important sugges-tion for future work in this area, specifically, that larger sample sizes and power may be particularly important for studies that attempt to replicate these findings in the

future. In our article we stressed that our conclusions were “limited by the sample size” and that “findings should be considered preliminary until replicated.”

We agree with Steen and Hamer that performing this study with controls would have strengthened the re-sults. As we mentioned in our article, however, our goal was to study the pathogenesis of hippocampal reduction in this sample and not to compare how hippocampal volume may differ between clinical and nonclinical groups. We have reported that in terms of functional impairment, there is no significant difference between children with posttraumatic stress disorder (PTSD) and children with subthreshold posttraumatic reactions.3

Studying the natural continuum of symptoms in trau-matized youth and how it may impact brain develop-ment have been some of our goals.

The fourth issue is the statistical approach. We care-fully explained in the article why we chose the statistical approach we took. We disagree that intraclass or con-cordance correlations would have been more appropri-ate, because we were attempting to establish the associ-ation between cortisol and PTSD symptoms and hippocampal change; we were not trying to show that PTSD and cortisol are “invariant in terms of mean and SD.” Steen and Hamer also mentioned potential error caused by the multiple correlations presented in our Table 2 (type 1 error). The multiple tests were not probes for a significant finding among a host of possible associ-ations but, rather, were theoretically directed tests. At least 20 of the 24 correlations presented in Table 2 were provided for completeness or descriptive or exploratory purposes only. What we find remarkable, from a theo-retical perspective, was that both cortisol levels and PTSD symptoms at time 1 were associated with change in hippocampal volumes. If only PTSD symptoms or only cortisol levels were associated with change in hippocam-pal volumes, a chance association would seem more likely; however, this was not the case.

We note again that we made it clear in the discussion section of our article that ours was preliminary evidence and that larger studies are needed to confirm these find-ings and clarify the role that additional factors may play in this association. However, given the difficulty in con-ducting and obtaining funding for such studies without any evidence, we feel that our study was an important contribution at this point in the development of the literature, because it points to the need for such larger studies.

On their last point, Steen and Hamer cite 2 studies: one by DeBellis et al4 _{and another by our group.}5_We

would like to clarify their comments on these studies because, as presented, they can be misconstrued. In the DeBellis et al article, there were only 9 subjects with PTSD; the addition of 9 controls made it an 18-subject study. Our article, which showed no hippocampal dif-ferences, was a cross-sectional study. As we mentioned

234 LETTERS TO THE EDITOR

(4)

DOI: 10.1542/peds.2007-0867

2007;120;232

Pediatrics

R. Grant Steen and Robert M. Hamer

Stressing About Posttraumatic Stress Disorder

Services

Updated Information &

http://pediatrics.aappublications.org/content/120/1/232 including high resolution figures, can be found at:

References

http://pediatrics.aappublications.org/content/120/1/232#BIBL This article cites 7 articles, 2 of which you can access for free at:

Subspecialty Collections

y_sub

http://www.aappublications.org/cgi/collection/psychiatry_psycholog Psychiatry/Psychology

al_issues_sub

http://www.aappublications.org/cgi/collection/development:behavior Developmental/Behavioral Pediatrics

following collection(s):

This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtml in its entirety can be found online at:

Information about reproducing this article in parts (figures, tables) or

Reprints

http://www.aappublications.org/site/misc/reprints.xhtml Information about ordering reprints can be found online:

(5)

DOI: 10.1542/peds.2007-0867

2007;120;232

Pediatrics

R. Grant Steen and Robert M. Hamer

Stressing About Posttraumatic Stress Disorder

http://pediatrics.aappublications.org/content/120/1/232

located on the World Wide Web at:

The online version of this article, along with updated information and services, is

the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2007 has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it