Identification of key potential CLUSTERs
To exploit the potential of realist synthesis approaches requires rich conceptual and contextual data. Reporting limitations and the varied emphases of published reports make it unlikely that all relevant data are included in a single report of a study. However, the scale and expense of a RCT increases the likelihood that multiple research reports have been produced relating to the study of interest. Such reports may include supplementary qualitative work, process evaluations, student projects, pilot studies, feasibility studies and follow-up studies. All such papers may help us to understand the study context, mechanisms and outcomes. Therefore, a key task is to move from analysis of a single study report to a detailed examination of a cluster of related papers. Such forensic examination looks not only for directly related ‘sibling studies’, but also for tangentially related ‘kinship’ papers (i.e. papers that may represent replication of an existing programme in a different context, thus allowing for comparison and contrast).
Finally, syntheses, analyses and theoretical papers may locate the study within a wider context of exemplars or case studies, thereby expanding the potential for comparison. Selection of clusters is necessarily limited by the resources available for analysis. In-depth analysis, as typically performed for realist synthesis, typically precludes the comprehensive and exhaustive approaches prescribed by systematic review methods.
In selecting focal study clusters, the team considered both the likely success of the programme and the availability of sibling and/or kinship study reports. At this stage, the Preparing for Parenthood cluster was excluded, as even though it possessed several companion reports the trial did not demonstrate
potential effectiveness.
Searching for CLUSTER documents
Searching for documents to populate a study cluster has, until recently, been viewed as essentially an unsystematic and arbitrary procedure. Conceptually it draws upon the long-established retrieval practices
outlined in Bates’ seminal paper139on‘berry picking’ including ‘backward chaining’ (following up cited
references) and‘forward chaining’ (following up cited articles). Recent years have revealed a prodigious
potential yield from supplementary documents. For example, a review by Jagosh et al.140revealed several
clusters with an average of 12 reports per cluster. We used systematic methods previously developed by one of the authors of our study (AB) for implementing cluster to become CLUSTER searching, for which a full published description of the CLUSTER methods (Citations, Lead authors, Unpublished materials, Scholar
searches, Theories, Early examples, Related projects) is openly available.141In essence, the research team
undertook persistent pursuit of study links, contextual links and theoretical links from the source study or studies to other related reports which then, themselves, initiated a further cause for searching. CLUSTER searching is reliant on relatively rapid judgements on potential links between a referring document and its
referent.141When papers shared a study identifier or acronym (e.g. PoNDER) or a RCT identifying number,
such connections were easy to establish. However, more typically a sibling relationship between papers relies on similarities in authorship, study context and sponsoring institution. However, further checks involve pursuing cross-citation and co-citation so that a network of studies could be constructed. Synthesis and construction of a theoretical model
For the synthesis stage we developed a rapid realist review approach, provisionally labelled as‘best-fit
realist synthesis’. This involves:
1. identification of a provisional‘best-fit’ conceptual framework as a starting point for data analysis
2. population of the conceptual framework with‘if–then’ statements from the identified articles
3. construction of pathways or chains from‘if–then’ statements to surface potential mechanisms by which
outcomes might be achieved
4. identification of existing theory underpinning individual mechanisms
5. development of a programme theory to explain how PND prevention programmes may work 6. testing of the programme theory with contextual data from included studies.
Identification of provisional‘best fit’ conceptual framework
Given the prominence of group care approaches among the candidate interventions (e.g. CenteringPregnancy or IPT) the research team decided to focus initial analytical attempts on the group-care model and then to seek to highlight similarities and differences with behavioural interventions delivered on an individual basis, either via face to face or via telephone. A search was conducted on Google Scholar (Google Inc., Mountain
View, CA, USA), harnessing its extensive full-text searching functionality, using the terms‘group care’ OR
‘group visits’ AND ‘health education’ AND ‘model’ OR ‘framework’.
Population of the conceptual framework
In examining CLUSTER documents the research team sought to identify mechanisms by which outcomes were achieved in a particular context. Mechanisms were operationalised by construction of a series of ‘if–then’ statements based on causal relationships advanced by the RCT, or hypothesised explanations
proposed by either the qualitative research or derived from the‘Discussion’ sections of the associated
study reports. REVIEW METHODS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
Construction of pathways or chains from‘if–then’ statements
‘If–then’ statements were subsequently constructed into complete pathways or partial chains to form an embryonic basis for a theoretical model that attempted to explain how the intervention works for different populations in different contexts from first action through to ultimate outcome. Given the heterogeneity of the interventions present in the initial clusters, it is unsurprising to note the presence of different mechanisms (e.g. between group- and individual-based approaches) and yet common success factors,
for example, the establishment of‘trust’, whether this be between a woman and a health-care provider,
between a woman and other members in her group, or between peers. This modelling process provided the facility to explain both generalisable mechanisms and specific areas of variance.
Identification of existing theory underpinning individual mechanisms
Examination of mechanisms by which the interventions sought to meet the various needs of the pregnant women identified several key concepts. In several instances, these concepts were explicitly linked within the study to specific theory or an implicit connection was readily identifiable (e.g. by using terminology associated with a theory).
Development of a programme theory
Based on the conceptual framework, and starting from the premises involved in the group-based model, the research team constructed a programme theory to explain how such a model might work in
preventing PND. This overarching programme theory was then examined in more detail to identify where individual-based approaches were unable to meet the same programme requirements and either
attempted to substitute for them (e.g. in substituting the resources of the individual peer supporter for the collective resources of facilitator plus group) or offered features not possible within the constraints of the group approach (e.g. in targeting and making application of strategies to the specific needs of the individual). Data from included studies, quantitative and qualitative, were used to examine the evidence in support of the programme theory. Realist synthesis also accommodates the bringing to bear of a wider evidence base.
In this review more proximate evidence was first accessed, identified via a CLUSTER searching approach,141
and then expanded where necessary to a wider set of theoretical and empirical papers. For example, ‘direct’ qualitative data related to the experience of group-based interventions was used to identify the features of such approaches and this was then supplemented by theoretical understandings of the basis
underpinning the interventions142and by middle-range theory examining mechanisms for PND.143In this
way the explanatory power of the review was broadened beyond the tight focus prescribed by the inclusion criteria.
The supporting data may be limited and may be at a level of abstraction that makes it difficult to identify the exact mechanism by which cause achieves effect. As a consequence, synthesis is to a certain extent an
interpretive process which may require the reviewer to identify hypothetical intermediate links in a chain144
by which, for example, training leads to self-efficacy. A further challenge of this method relates to reliance on the detail and quality of reporting; while it is legitimate for the reviewer to generate potential
explanations by which a particular outcome is affected, more typically these connections are advanced in the published literature. As a consequence certain explanations may be well rehearsed but poorly substantiated, whereas others may be novel and consequently unsupported. A key stage is therefore the
subsequent validation of the‘if–then’ statements such that they are supported by empirical data or at
the very least they are internally consistent with a range of published data sources. To a certain extent the process is analogous with brainstorming processes in which idea generation is deliberately divorced from subsequent validation. In summary, a complete explanation is initially privileged over a high-quality one, with the realisation that a poorly constructed study may perversely yield valuable explanatory insights.