Systematic review

According to Torgerson (2003) the systematic review is intended as a thorough, completely transparent review of all existing research to identify consistency (and anomalies) across large sets of empirical data. A systematic review is intended to reduce bias and enable critical appraisal by combining relevant research in a systematic way. Systematic reviews have been adopted from the medical research community

Page | 4-88 and have a fairly rigorous set of criteria to adhere to ensure a transparent, unbiased review of the literature.

Torgerson (2003, p. 7) outlines the objectives of a systematic review:

 “to address a specific, well focused, relevant question

 “to search for, locate and collate the results of the research in a systematic way”

 “to reduce bias at all stages of the review (publication and other forms of bias)”

 “to appraise the quality of the research in light of the research question”

 “to synthesize the results of the review in an explicit way”

 “to make the knowledge more accessible”

 “to identify gaps; to place new proposals in the context of existing knowledge;”

 “to propose a future research agenda: to make recommendations:

 “to present all stages of the review in the final report to enable critical appraisal and replication”

A systematic review are often synonymous with meta analyses however a systematic review need not include a meta-analysis (Torgerson 2003 citing Chalmers, 2002) and there may be reasons why meta-analysis is not appropriate for a systematic review. For example, if all available literature cannot be found (Torgerson 2003), if existing data is known to contain biases, and if data is not homogenous (Hemingway 2001), a systematic review should not include a met- analysis.

Another concern regarding meta-analyses is that of unpublished literature. Another concern is inability to obtain all appropriate research for inclusion due to unpublished data being unavailable. Torgerson (2003) points out that some journal editors refuse to publish articles that result in negative or non-significant results due to a perceived lack of ‘interest’ on the part of readers. However, publication of studies with small samples that do yield a significant result combined with failing to report other small studies that do not, can lead to misleading results for any potential meta analyses undertaken, as part of a systematic review, particularly if the significant result is a consequence of a Type 1 error (Torgerson 2003). The presumption being that research that fails to achieve a certain result is somehow ‘inferior’ or inaccurate, in some way. This, however, results in a bias. Similarly, failure to include all available published research

Page | 4-89 without explanation or justification, again, results in bias. These difficulties are not intended to suggest that systematic reviews or meta analyses should not be undertaken. However, it can be argued that many of the protocols outlined by the Cochran Review (2008) and Torgerson (2003) cannot be adhered to as they do not apply to psychological research. For example, for a study to be considered appropriately conducted primary empirical medical study to be included for review, it should be a double blind design. Double blind research in psychology would potentially violate research ethics (BPS 2006). This is but one example of how the medical and social sciences discipline differ in the empirical research protocols. Similarly, randomised control designs are considered the ideal for empirical research and most appropriate for inclusion in systematic reviews (Torgerson 2003) again, rarely is this sampling method employed in psychological research. Similarly sample size may be an issue, according to Torgerson (2003), however, psychology research often includes studies with small sample sizes, but this, consequently can alter the statistical power of a meta-analysis, resulting in bias, therefore is frowned upon. It would be impractical to spend substantial time exploring these differences, it is necessary to be cognisant of some of the key issues as it impacts the strategies employed for systematic reviews for psychological research. Consequently, recommendations outlined by the Cochrane Review(2008), Hemingway (2001) Torgerson (2003) and Gagnier, et al. (2012), have been considered and applied where possible, however, this has been combined with strategies employed in practice for systematic reviews of psychopathy related research such as Miller and Lynam (2012) who conducted similar research examining the construct validity of the PPI and derivatives.

Page | 4-90 One issue that should not be ignored when considering whether or not to conduct a systematic review whether or not the data that is being collected is homogenous. At least, if a systematic review is to include a meta-analysis. According to Torgerson (2003), there are numerous reasons for this the underpinnings of a particular intervention, assessment or treatment may be similar, if they are not identical the consequence of pooling them for the purposes of meta-analysis, will result in point estimates that will not apply to any of the studies. Further, pooling of data that is similar but not identical can result in result in false confidence intervals and effect sizes (Torgerson, 2003). A more substantial issue with pooling data for meta-analysis in the case of the PPI and derivatives is that of bias with the research. According to Skeem and Cooke (2010) and in their response Hare (2010) all acknowledge an over reliance on the PCL-R to validate other measures of psychopathic traits has resulted in systematic bias within psychopathy literature. Consequently, the construct of psychopathy is now closely associated with features not part of the original Cleckley/Hare model which is said to be the nomological model of psychopathy; specifically, criminality and violence. A meta-analysis, in this instance may not adequately address this bias in the literature, as well as a narrative empirical synthesis because so much of the research, to date, has relied upon the PCL-R for construct validity of other measures of psychopathic traits resulting in some of these measures actually being re structured to make them more consistent with the PCL-R. Further, so much of the research has relied on male offenders for research purposes (Mahmut et al. 2007) that there is an over-representation of traits associated with offending psychopaths being superimposed on the construct of psychopathy as a whole (Skeem and Cooke 2010). Consequently, a narrative empirical synthesis has been used in an effort to reduce some potential bias of data and also to mitigate potential issues regarding the substantial heterogeneity within the data.

Page | 4-91