NSPCC Inclusion/Exclusion Assessment
Paper reference (author/title): Pub year Analysed by
Link to paper/abstract:
RQ1 (ROOTS) X RQ2 (interventions)
Paper Type 1 META ANALYSIS/SYSTEMATIC REVIEW 2 TRIAL 3 CASE CONTROL 4 COHORT STUDY 5 QUALITATIVE 6 EVALUATION QUANTITATIVE 7 MIXED METHODS
Recommend Decision (INCLUDE/EXCLUDE)
Abstract or key findings:
Limitations:
Any notes/comments/ page numbers of good quotes:
Methodology overview
Participants and ages: Setting:
Intervention Type: Time period of research:
1 – META ANALYSES / SYSTEMATIC REVIEWS YES NO UNCLEAR N/A
1. Did the study address a clearly focused question? 2. Was a comprehensive literature search conducted using
relevant research databases (i.e. ABI/INFORM, Business Source Premier psycINFO, Web of Science, etc.).
3. Is the search systematic and reproducible (e.g. were searched information sources listed, were search terms provided)?
4. Has publication bias been prevented as far as possible (e.g. were attempts made at collecting unpublished data)?
5. Are the inclusion and exclusion criteria clearly defined (e.g. population, outcomes of interest, study design) 6. Was the methodological quality of each study assessed
using predetermined quality criteria?
7. Are the key features (population, sample size, study design, outcome measures, effect sizes, limitations) of the included studies described?
8. Has the meta-analysis been conducted correctly? 9. Were the results similar from study to study? 10. Is the effect size practical relevant?
11. How precise is the estimate of the effect? Were confidence intervals given?
2 – TRIAL YES CAN’T TELL
NO A – Are the results of the trial valid?
1. Did the trial address a clearly focused question?
2. Was the assignment of patients to treatments randomised? 3. Were all of the patients who entered the trial properly
accounted for at its conclusion?
IS IT WORTH CONTINUING?
4. Were patients, health workers and study personnel ‘blind’ to treatment?
5. Were the groups similar at the start of the trial?
6. Aside from the experimental intervention were the groups treated equally?
B – What Are the results?
7. How large was the treatment effect?
8. How precise was the estimate of the treatment effect?
C – Will the results help locally?
9. Can the results be applied to your context (or to the local population)?
10. Were all clinically important outcomes considered? 11. Are the benefits worth the harms and costs?
3 – CASE CONTROL STUDY YES CAN’T TELL
NO A – Are the results of the study valid?
1. Did the study address a clearly focused issue? 2. Did the authors use an appropriate method?
IS IT WORTH CONTINUING?
3. Were the cases recruited in an acceptable way? 4. Were the controls selected in an acceptable way?
5. Was the exposure accurately measured to minimise bias? 6a. What confounding factors have the authors accounted for?
(LIST)
6b. Have the authors taken account of the potential confounding factors in the design and/or in their analysis?
B – What are the results?
7. What are the results of the study?
8. How precise are the results? How precise the estimate of risk?
9. Do you believe the results
C – Will the results help locally?
10. Can the results be applied to the local population?
4 – COHORT STUDY YES CAN’T TELL
NO A – Are the results of the study valid?
1. Did the study address a clearly focused issue? 2. Was the cohort recruited in an acceptable way?
IS IT WORTH CONTINUING? B – What are the results?
3. Was the exposure accurately measured to minimise bias? 4. Was the outcome accurately measured to minimise bias? 5a. Have the authors identified all important confounding
factors?
5b. Have they taken account of the confounding factors in the design and/or analysis?
6a. Was the follow up of subjects complete enough? 6b. Was the follow up of subjects long enough? 7. What are the results of this study?
8. How precise are the results?
9. Do you believe the results?
C – Will the results help locally?
10. Can the results be applied to the local population?
11. Do the results of this study fit with other available evidence? 12. What are the implications of this study for practice?
5 – QUALITATIVE YES NO UNCLEAR
1. Was there a clear statement of the aims of the research? 2. Is a qualitative methodology appropriate?
IS IT WORTH CONTINUING?
3. Was the research design appropriate to address the aims of the research?
4. Was the recruitment strategy appropriate to the aims of the research?
5. Was the data collected in a way that addressed the research issue?
6. Has the relationship between researcher and participants been adequately considered?
7. Have ethical issues been taken into consideration? 8. Was the data analysis sufficiently rigorous? 9. Is there a clear statement of findings? 10. How valuable is the research?
6 – QUANTITATIVE EVALUATION – MARYLAND SCALE INDICATE LEVEL Level 1 Correlational study
Correlation between intervention programme and dependent variable at a single point in time
Level 2 Pre and post test
Measures of the dependent variable before and after intervention. No comparable control group
• Samples hold stable in size and composition at T1 and T2
• Transparency regarding test conditions that can confound results at each time point administrator, environment for testing, time of testing and so on.
Level 3 Cohort study with matched control
Measures of the dependent variable before and after intervention, in both experimental comparable control conditions
• Year effects are included
• appropriate time varying controls are used
• Control group would have followed same trend and treatment group • Known time period for treatment
• How well-matched were control group and treatment groups
Level 4 Quasi experimental with controlled conditions
As with 3, plus: Variables known to have influence on dependent variable are controlled for in analysis.
Level 5 Random Controlled Trial
Random assignment of intervention and control condition to comparable units. Before and after measures, plus retest if possible. Randomisation is successful