Can you apply this valid, important evidence about an intervention in caring for your patient/client? What is the external validity?
45. Are there any threats to external validity in this study?
The sample size of the study for the control group was low, n=22. This is a threat to external validity.
What is the bottom line?
Appraisal Criterion Reader’s Comments
PEDRO score (see scoring at end of form) 5
Summarize your findings and relate this back to clinical
significance From the systemic evidence quantified form this study, patients with PCS maybe
able to safely participate in exercises at a low-levels (<75% age predicted max heart rate). However, the exercise prescription should be individualized based on results from a graduated treadmill test.
Name: David Scussel
Intervention – Evidence Appraisal Worksheet
Citation (use AMA or APA format):
Leddy, J. J., Baker, J. G., Kozlowski, K., Bisson, L., & Willer, B. (2011). Reliability of a graded exercise test for assessing recovery from concussion. Clinical
Journal of Sport Medicine, 21(2), 89-94.
Level of Evidence (Oxford scale):
1b, Individual inception cohort study with > 80% follow-up
Is the purpose and background information sufficient?
Appraisal Criterion Reader’s Comments
Study Purpose
Stated clearly?
Usually stated briefly in abstract and in greater detail in introduction. May be phrased as a question or hypothesis.
A clear statement helps you determine if topic is important, relevant and of interest to you. Consider how the study can be applied to PT and/or your own situation. What is the purpose of this study?
Literature
Relevant background presented?
A review of the literature should provide background for the study by synthesizing relevant information such as previous
research and gaps in current knowledge, along with the clinical importance of the topic.
Describe the justification of the need for this study
Yes, the purpose of the study was to assess the retest reliability (RTR) and interrater reliability (IRR) of a standardized graded treadmill exercise test in
determining return to play (RTP) status of both concussed athletes and nonathletes.
Yes, background information was
presented making the case the stepwise program for recovery from concussions starting with rest and ending with full contact practice while progressing through each stage every 24 hours may be
outdated and is not a proven reliable method. Further background was presented to indicate provocative
exercising testing of concussed individuals maybe proven to be a reliable method with replicable studies.
Does the research design have strong internal validity?
Appraisal Criterion Reader’s Comments
Discuss possible threats to internal validity in
the research design. Include:
Assignment Attrition
Assignment: Twenty-one subjects
(11athletes and 10 non-athletes) experiencing concussion symptoms ≥6 weeks but <52 weeks referred
Instrumentation Maturation Testing Compensatory Equalization of treatments Compensatory rivalry Statistical Regression
consecutively to a concussion clinic, and 10 control subjects (no history of
concussions and sedentary for the past 6 weeks) were assigned to the study. The study would have had a narrow confidence interval if a greater number of participants were recruited for both the experimental and control groups.
Attrition: All participants completed the
study.
History: The authors addressed the
history and lack of reliability of the stepwise progression methods used to clear individuals for return to play after being concussed.
Instrumentation: The treadmill test
followed the Balke protocol to the first sign of concussive symptom exacerbation. “The treadmill speed was set at 3.3 mph at 0.0% incline. After 1 minute, the grade increased to 2.0% while maintaining the same speed. At the start of the third minute and each minute thereafter, the grade increased by 1.0%, maintaining the speed at 3.3 mph.” RPE and heart rate were recorded every minute with BP being recorded every 2 minutes.
Maturation: Concussions are very
subjective in their diagnosis to begin with and tracking the heeling process requires tracking subjective symptoms reported by the patient.
Testing: The test was performed on all
subjects 2 to 3 weeks apart. I would
assume testing at least once or twice more would have validated the results further. Although, increasing the length of the study would have probably resulted in an increase in attrition.
Compensatory equalization of
treatment: Specific protocols were used.
Blinding of patients and researchers would have increased the validity of the study.
Statistical regression: Those who had
experienced concussions were put into the PCS group while those who had never experienced a concussion were put into the control group. The PCS group was not split into subsequent groups.
Are the results of this therapeutic trial valid?
Appraisal Criterion Reader’s Comments
46. Did the investigators randomly assign subjects to treatment groups?
a. If no, describe what was done b. What are the potential consequences
of this assignment process for the study’s results?
No.
a. 21 consecutive concussion patients with symptoms ≥6 weeks but <52 weeks
referred to a concussion clinic were placed in the PCS group. 10 control subjects with no history of concussions and having been sedentary for the past 6 weeks were
placed in the control group.
b. No potential consequences exist as all subjects participated in the treadmill protocol and gave their consent to participate in the study.
47. Did the investigators know who was being assigned to which group prior to the allocation?
a. If they were not blind, what are the potential consequences of this knowledge for the study’s results?
a. The researchers were not blinded to whether the participants were members of the PCS group or the control group. No potential consequences existed as all subjects completed the study’s protocol.
48. Were the groups similar at the start of the trial? Did they report the demographics of the study groups?
a. If they were not similar – what differences existed?
b. Do you consider these differences a threat to the research validity? How might the differences between groups affect the results of the study?
The groups were similar in demographics at the beginning of the study and no
significant differences existed between the two groups concerning demographics.
49. Did the subjects know to which treatment group they were assign?
a. If yes, what are the potential consequences of the subjects’ knowledge for this study’s results
Yes, all subjects knew which group they were in. As all participants completed the treadmill protocol no potential
consequences existed.
50. Did the investigators know to which treatment group subjects were assigned ?
a. If yes, what are the potential consequences of the subjects’ knowledge for this study’s results
Yes, the investigators knew which group the subjects were in. As all participants completed the treadmill protocol no potential consequences existed.
the actual experimental treatment? a. If not, what are the potential
consequences of this knowledge for the study’s results?
protocol and were managed equally.
52. Was the subject follow-up time sufficiently long to answer the question(s) posed by the research?
a. If not, what are the potential consequences of this knowledge for the study’s results?
Yes the subject follow-up time was sufficient. If the follow-up time were lengthened it would have increased the validity of the results of the study but also increased the attrition of participants.
53. Did all the subjects originally enrolled complete the study?
a. If not how many subjects were lost? b. What, if anything, did the authors do
about this attrition?
c. What are the implications of the attrition and the way it was handled with respect to the study’s findings?
Yes all participants completed the study.
54. Were all patients analyzed in the groups to which they were randomized (i.e. was there an intention to treat analysis)?
a. If not, what did the authors do with the data from these subjects? b. If the data were excluded, what are
the potential consequences for this study’s results?
Yes all subjects’ data was analyzed in the groups to which they were assigned.
Are the valid results of this RCT important?
Appraisal Criterion Reader’s Comments
55. What were the statistical findings of this study?
a. When appropriate use the
calculation forms below to determine these values
b. Include: tests of differences With p- values and CI
c. Include effect size with p-values and CI
d. Include ARR/ABI and RRR/RBI with p- values and CI
e. Include NNT and CI
f. Other stats should be included here
a, c, d, e, f N/A
b. Cohen’s criteria for effect size for intraclass correlation coefficient (ICC) (SPSS version 16.0; SPSS, Inc., Chicago, Illinois) was used to report agreement of maximal heart rate, SBP, DBP, and RPE between the 2 treadmill tests.
95% CI was used.
P was reported, but not set to a
significance level such as P < 0.05.
PCS group ICC and P stats: heart rate ICC 0.79 P<0.36, max SBP ICC 0.37 P<0.02, DBP ICC 0.20 P<0.80, RPE ICC 0.42
P<0.09.
Control group ICC and P stats: heart rate ICC 0.64 P<0.97, max SBP ICC 0.90
P<0.71, DBP ICC 0.52 P<0.37, RPE ICC
proved to not be a reliable reproducible measurement. The treadmill test is a reliable method test to use in the clinic for those who have been concussed.
57. Do these findings exceed a minimally
important difference? Was this brought up or discussed?
a. If the MCID was not met, will you still use this evidence?
Minimally important difference was not discussed in this study.
Can you apply this valid, important evidence about an intervention in
caring for your patient/client? What is the external validity?
Appraisal Criterion Reader’s Comments
58. Does this intervention sound appropriate for use (available, affordable) in your clinical setting? Do you have the facilities, skill set, time, 3rd party coverage to provide this
treatment?
Yes the intervention is both available and affordable in all clinical settings I have been exposed to as all outpatient ortho PT clinics I have visited have a treadmill, and the Borg RPE is a measurement tool taught in PT school. As this intervention is considered part of an evaluation 3rd party payers will pay for it.
59. Are the study subjects similar to your patient/ client?
a. If not, how different? Can you use this intervention in spite of the differences?
Yes, the study participants are similar to the concussed patients I have seen in practice.
60. Do the potential benefits outweigh the potential risks using this intervention with your patient/client?
As the treadmill test is terminated immediately upon exacerbation of concussion symptoms or if the patient fatigues out, while heart rate and BP are being monitored frequently, there is little risk to the patient in participating in this intervention.
61. Does the intervention fit within your
patient/client’s stated values or expectations? a. If not, what will you do now?
Yes, as my goal as a clinician is to help return the patient to play/work asap, and the patients goal is usually to RTP and reduce symptoms ASAP as well.
62. Are there any threats to external validity in this study?
The sample size of the study for both the experimental and control groups were low, 21 and 10 respectively. This is a threat to external validity.
What is the bottom line?
Summarize your findings and relate this back to clinical significance
The Balke exercise treadmill test for symptom identification in concussed patients has “sufficient maximum heart rate RTR,” and appears to be a more reliable method for tracking the progress of athletes trying to RTP than the stepwise program, which is subjective. Further studies need to be performed.
Name: David Scussel
Intervention – Evidence Appraisal Worksheet
Citation (use AMA or APA format):
Leddy, J. J., Cox, J. L., Baker, J. G., Wack, D. S., Pendergast, D. R., Zivadinov, R., & Willer, B. (2013). Exercise treatment for postconcussion syndrome: A pilot study of changes in functional magnetic resonance imaging activation, physiology, and symptoms. The Journal of Head Trauma Rehabilitation, 28(4), 241-249.
Level of Evidence (Oxford scale):
1b, Individual RTC
Is the purpose and background information sufficient?
Appraisal Criterion Reader’s Comments
Study Purpose
Stated clearly?
Usually stated briefly in abstract and in greater detail in introduction. May be phrased as a question or hypothesis.
A clear statement helps you determine if topic is important, relevant and of interest to you. Consider how the study can be applied to PT and/or your own situation. What is the purpose of this study?
Literature
Relevant background presented?
A review of the literature should provide background for the study by synthesizing relevant information such as previous
research and gaps in current knowledge, along with the clinical importance of the topic.
Describe the justification of the need for this study
To compare before and after fMRI results between subjects with postconcussion syndrome (PCS) who received an exercise program, subjects with PCS who received a placebo intervention, and a healthy control group.
Yes, background information was
presented stating fMRI has been used with subjects diagnosed with PCS when
completing cognitive tasks but has yet to be utilized to show results from before and after treatment techniques meant to help resolve PCS symptoms.
Does the research design have strong internal validity?
Appraisal Criterion Reader’s Comments
Discuss possible threats to internal validity in
the research design. Include:
Assignment Attrition History
Assignment: The first 4 PCS patients
eligible for treatment were assigned to the exercise PCS groups, the next 4 PCS patients were assigned to the placebo stretching group. The assignment process
Instrumentation Maturation Testing Compensatory Equalization of treatments Compensatory rivalry Statistical Regression
between the patients separate and to help them to think they were receiving the best available treatment to them. The
assignment process decreased the risk to internal validity.
Attrition: All participants completed the
study.
History: The authors addressed the
history of fMRI used in the PCS population and the need for this study.
Instrumentation: The PCS exercise
group performed a progressive aerobic exercise program at 80% of the heart rate subthreshold to exacerbation of
concussion symptoms found through treadmill testing. The aerobic exercise program was 20 minutes/day, 6days/week, in the home or at a gym. The PCS placebo group received a sham stretching program (stretched 20 minutes/day 6 days/week) and were instructed not to exceed 40%- 50% of their age predicted max heart rate. The control group was comprised of healthy subjects who were matched to the demographics of the PCS subjects. fMRI results were compared between the 3 groups before treatment began and 12 weeks later.
Maturation: Concussions are very
subjective in their diagnosis and tracking the heeling process requires either tracking subjective symptoms or
comparing brain imaging. For this study
fMRI results were compared between the
3 groups before treatment began and 12 weeks later.
Testing: The group performed their
respective treatments for 12 weeks. fMRI results were compared between the 3 groups before treatment began and 12 weeks later.
Compensatory equalization of
treatment: Specific protocols were used.
were assigned to their respective groups in succession as described in the assignment section in order to prevent interaction and help the subjects to think they were receiving the best available treatment.
Statistical regression: Demographics
between the groups were similar and the control group was matched to the PCS groups to help eliminate the threat to internal validity.
Are the results of this therapeutic trial valid?
Appraisal Criterion Reader’s Comments
63. Did the investigators randomly assign subjects to treatment groups?
a. If no, describe what was done b. What are the potential consequences
of this assignment process for the study’s results?
No.
a. The first 4 PCS patients eligible for treatment were assigned to the exercise PCS groups, the next 4 PCS patients were assigned to the placebo stretching group. The assignment process was chosen in order to keep interaction between the patients separate and to help them to think they were receiving the best available treatment to them.
b. Without randomization the potential to have discrepancies between groups could result in skewed results.
64. Did the investigators know who was being assigned to which group prior to the allocation?
a. If they were not blind, what are the potential consequences of this knowledge for the study’s results?
a. The researchers did not know which subjects would be placed into which group before hand as the subject had not yet been patients at the clinic were the experiment was being held. The
assignment to the two PCS groups was based on who was referred to the clinic first. The first 4 PCS subjects referred to the clinic were assigned to the exercise group. The next 4 referred were assigned to the placebo group.
65. Were the groups similar at the start of the trial? Did they report the demographics of the study groups?
a. If they were not similar – what differences existed?
b. Do you consider these differences a threat to the research validity? How might the differences between
The demographics of the participants were disclosed and no significant discrepancies existed. The control, healthy subjects were matched to the PCS subjects.
study?
66. Did the subjects know to which treatment group they were assign?
a. If yes, what are the potential consequences of the subjects’ knowledge for this study’s results
No. The PCS subjects were kept apart from each other and did not know what type of treatment the other groups were receiving. However the control group knew they were the healthy control group only receiving fMRIs twice over a 12 week period.
67. Did the investigators know to which treatment group subjects were assigned ?
a. If yes, what are the potential consequences of the subjects’ knowledge for this study’s results
Yes, the investigators knew which group the subjects were in. As the study’s results depended on imaging results, which are objective, results and free of bias, potential consequences were eliminated.
68. Were the groups managed equally, apart from the actual experimental treatment?
a. If not, what are the potential consequences of this knowledge for the study’s results?
All participants were managed equally.
69. Was the subject follow-up time sufficiently long to answer the question(s) posed by the research?
a. If not, what are the potential consequences of this knowledge for the study’s results?
Yes the subject follow-up time was sufficient to show significant changes in results from before treatment
implementation and after.
70. Did all the subjects originally enrolled complete the study?
a. If not how many subjects were lost? b. What, if anything, did the authors do
about this attrition?
c. What are the implications of the attrition and the way it was handled with respect to the study’s findings?
Yes all participants completed the study.
71. Were all patients analyzed in the groups to which they were randomized (i.e. was there an intention to treat analysis)?
a. If not, what did the authors do with the data from these subjects? b. If the data were excluded, what are
the potential consequences for this study’s results?
Yes all subjects’ data was analyzed.
Are the valid results of this RCT important?
Appraisal Criterion Reader’s Comments
72. What were the statistical findings of this study?
a. When appropriate use the
calculation forms below to determine these values
a, c, d, e, f N/A
b. fMRI results before treatment: No difference between the exercise and placebo PCS groups, the control group
CI
d. Include ARR/ABI and RRR/RBI with p- values and CI
e. Include NNT and CI
f. Other stats should be included here
treatment: No differences between the exercise PCS group and the control group, the placebo PCS group had significantly (P<0.05) less cerebellar activity.
73. What is the meaning of these statistical findings for your patient/client’s case? What does this mean to your practice?
A prescribed progressive aerobic exercise program, as used in this study, appears beneficial in the recovery from PCS and to