DEFINING THE TOPIC

Clear definition of the review topic is essential as this will limit the research to a key area, facilitating decisions about other review elements such as the search strategy and the inclusion criteria. By forming a focused definition, a great deal of time can be saved further on in the review process (Chalmers & Altman, 1995).

2. OBJECTIVES

The objectives of the review need to be well defined as the entire review will be shaped according to these.

3. SEARCH

As the purpose of a systematic review is to draw together all the research pertaining to the topic at hand and to offer conclusions, the search for relevant studies has to be sufficiently extensive to account for all possible research. An ideal systematic review will uncover all related studies whose quality can then be assessed. To do so, several methods of searching should be employed.

Published literature: Numerous electronic databases are now available and hold the details o f scientific papers which have been published. Rigorous searching of these databases is required to assure accountability of all possible data relating to the review’s objectives. These databases, however, vary on a number of measures. A) Specific databases hold information for specific fields of research and the appropriate ones should be accessed. For example, EMBASE (Excerpta Medica online) is a largely pharmaceutical-based database and may prove fruitless for people carrying out a review of a topic which is not directly related. With the same token, this is not to say that relevant papers will not appear on this database. Therefore time saving exercises can often run the risk of missing potential studies. B) Electronic databases also differ from one another in terms of the origin of the material and language specificity. For example, certain databases deal only with European research while others specialise in American-

based literature. For this reason, if a world-wide review is the objective and one is imposing no language restrictions then such a limitation must be taken into account. C) Often systematic reviews are carried out over many months, sometimes years, therefore current awareness searches are an effective means of updating one’s register of studies to stay abreast of the latest developments in the field. They also allow recently published studies to be incorporated into the review, producing a comprehensive and ‘up-to-the minute’ review of the evidence. As different databases are updated at varying rates, ongoing searches of several databases would be required to ensure new studies are captured.

Unpublished literature: Material not yet published is equally important when carrying out a systematic review as the findings may have a considerable effect on the overall conclusions of the review, particularly if the research is recent, having built on the results of previous work. Databases have now been compiled which deal specifically with ‘grey literature’, as it is referred to in Europe, and ‘black literature’, as it is known in the United States. Both types of databases’ studies are indexed according to keywords which have been selected by the authors of each paper. As no standard rules or methods of indexing have been implemented, studies can often be missed if selected keywords do not correspond with those chosen by the systematic reviewer for inclusion in the search strategy. This problem is one that needs to be addressed to increase the reliability of systematic reviews by assessing the inclusion o f all relevant material.

Despite the drawbacks and potential pitfalls of electronic databases, the output is largely determined by the search strategy put in. A search strategy must be sufficiently sensitive so as to pick up all potentially valid papers but equally selective to limit the search to literature which is likely to be relevant. By carefully preparing the search strategy, the

recall of the search and its precision can be optimised (CRD, January 1996). This will involve skilled adaptation of the search strategy and increasing one’s knowledge of the relevant indexing terms by manuals and other relevant papers to determine the commonly used keywords (Lowe & Barnett, 1994). Suggested generic search strategies have been published as a starting point for many reviewers to uncover RCTs, articles about diagnosis, cause of disease and treatment.

Additional searches: In addition to the searching of electronic databases, experts in the field can be contacted to highlight any work which is ongoing or that they know of and feel would be appropriate. The success of this process is clearly subject to the willingness of the experts and may yield much or little information. It may be possible to include interim results o f studies still ongoing, provided permission is obtained to report unpublished data.

Following the retrieval of fiill-text papers, the references of possible studies can be scanned to uncover any further studies which have so far slipped through the net. Handsearching may be a time consuming process but one which can be very useful. By handsearching key journals in the field, articles can be retrieved which databases have failed to identify and have been overlooked by other forms of searching due to poor

indexing, or the fact that the work is yet to be cited and entered in the database. When the handsearch yields no articles, this time, effort and information is just as important as when many articles are retrieved. The Cochrane Collaboration is in the process of compiling registers of trials relating to all fields of health care research. Their primary interest is the identification of all randomised controlled trials and by recording the results of individual searches, journals do not need to be continually re-searched. Instead, reviewers can consult these registers to check for any possible missing studies. Registers for other study designs would be a great asset and a possible directive for the future.

When reviewing trials involving pharmaceutical products, it may be worthwhile contacting related companies who often hold results ‘on file’ but are as yet unpublished. Moves have been made within the UK, however, to make this data more easily accessible. This range of measures helps to increase the reliability of the review and the accuracy o f its conclusions (Chalmers & Altman, 1995).