Adopting a mixed-method approach: some reflections

This project combined the use of qualitative and quantitative techniques to answer the literature’s call for better quality, more in-depth data to inform the debate on school smoking policies. This call arguably parallels a broader move in tobacco control (and arguably in public policy more generally) towards the increased use of qualitative methodologies to inform the evidence base. This is to redress the traditional skew towards quantitative methods in these disciplines emphasised by Mehl (2003) in the abstract to his paper at the 12th

World Conference on Tobacco or Health in a session revealingly titled

“Adding tools to the methods toolbox: using qualitative research methods to improve programme and policy responses”:

The use o f qualitative research methods have been dwarfed by the wide use o f survey methods in the public health community, meanwhile the tobacco industry has been perfecting and making extensive use o f qualitative methods.

Mehl (2003:413)

The approach of this work was both multi-method (using more than one technique) and multi-methodological (combining the use of qualitative and quantitative approaches) and the adoption of such an approach cannot be left without some brief comment or reflection. Contrary to Mehl, others would argue that contemporary health research is full of multi-method approaches (Sale et al, 2002:43), and the adoption of such strategies often means addressing the issues of marrying together quantitative and qualitative techniques of research (Bryman, 2001; Gray & Densten, 1998; Sale et al, 2002). Having said this, as already highlighted, aside from this thesis very little investigation into the role of school smoking policies had used qualitative research, and even less had used this in association with more traditionally employed quantitative methods. Despite the increasing use of multiple techniques, the qualitative-quantitative debate is ongoing, and arguably will continue to be so, pervading the foundations upon which our methodological frameworks are built. Despite this, Sale et al (2002:44-5) argue that while such approaches are commonly practiced and accepted, this is often done uncritically, and without regard to the differences between the fundamental ontological and epistemological assumptions which underlie each method. This is not a methodological piece and there is not the space here to exhaustively detail the debate surrounding these issues. I have long seen value in a “right-tools-for-the-job” approach to method(ology) and this section is intended only as an acknowledgement of the issues surrounding mixing methods and methodologies, while providing a justification for this approach.

In my early consideration of mixing methods, I found that it was useful to summarise approaches to mixed-method research as a dichotomy which I labelled philosophical and mechanical approaches to the issue. It was after this that I discovered both Bryman’s categorisation, in which he attempts a similar reduction of the literature, using the terms epistemological and technical versions of the quantitative-qualitative division (2001:446) and Gray & Densten’s discussion of differences in approach which made reference to Bryman’s earlier writing (1998). I make explicit reference to these later discoveries for the sake of transparency. However, while acknowledging these writings, I will continue by using my own terms that I have found useful. That said, Gray & Densten’s description of divergent methods (1998:419) is useful (especially if contrasted with the term convergent which is used by Bryman but only in discussing validity through comparative methods (2001:73-4)). The adoption of these descriptors as suffixes to my own terms, is useful and hence I would argue that there are two broad approaches to the use of multi­ methodologies (1) Philosophical Approaches: Divergent Paradigms and

Separate Spheres and (2) Mechanical Approaches: Convergent Paradigms and Right Tools fo r the Job.

In the first approach Philosophical Approaches: Divergent Paradigms and

Separate Spheres there is a clear methodological divide (Table 1.1). It is

reasoned that qualitative and quantitative techniques are immersed in, emerge from and are underpinned by fundamentally separate paradigms (Bryman, 2001; Gray &Densten, 1998; Sale et al, 2002). These paradigms are defined by different ontologies, epistemologies and methodologies (Sale et al, 2002:44). So fundamentally different are these beliefs about the world being studied, that, even if looking at the same topic, a quantitative or qualitative approach could not be said to be studying the same phenomena.

Table 5.1 Fundamental differences between Quantitative and Qualitative paradigms as outlined by Sale et al (2002:44-6)

Quantitative Qualitative

Epistemology

Positivism

• All phenomena reducible to empirical indicators representing the truth

• Investigator and investigated are independent (i.e. one does not influence the other) • Objectivity

Interpretivism and Constructionivism

• There is no access to reality independent of our minds / no external referent by which to compare claims o f truth

• Investigator and investigated interactively linked

• Findings are mutually created within the context o f the situation which shapes the inquiry

• Suggests that reality has no existence outside o f the boundaries of the inquiry • Subjectivity

Ontology

• There is an absolute truth (objective reality) which exists independent o f human perception

• There are multiple truths / realities based on one’s construction o f reality

• Reality is socially constructed and therefore constantly changing

Aims

• Measure and analyse causal relationships between variables within a value-free framework

• Illumination o f process and meaning

Techniques

Randomisation; blinding; highly structured protocol;

questionnaires with limited range or responses

In-depth / focus group interviews; participant observation

Samples

• Larger than qualitative research

• Allows use o f statistical techniques which mean that sample data can be seen as representative and therefore generalisable to the

population

• Samples are not meant to represent populations

• Small, purposive samples o f articulate respondents to provide important but not representative data

This school o f thought traditionally says that because the world views of these two approaches are so different, they can never be said to study the same phenomena (Bryman, 2001:446; Sale et al, 2002:44). Even if a qualitative and quantitative study were both investigating, for example, chronic back pain, their separate epistemologies are so different that they cannot be said to be studying the same phenomenon. This clearly has consequences for cross- validation or triangulation between different methods within the same study.

It is useful to consider the second approach, Mechanical Approaches:

Convergent Paradigms and Right Tools fo r the Job, as residing around a

spectrum. At one end are the qualitative methods, and at the other the quantitative methods. This spectrum tends from (broadly) interpretive and constructionist methods at one end towards (broadly) positivistic methods at the other. This understanding is common today (Mendlinger & Cwikel, 2008). In this approach epistemological differences are celebrated and seen as compatible. Methods are regarded as tools for a job, and may be selected for use from any point along this spectrum. Ihe limitations o f each method are key, the strengths of one method being used to enhance the other. Contrary to the first school of thought, researchers who favour multi-method approaches argue that quantitative and qualitative methods may be seen as different ways of examining the same research question, and that the use o f multiple methods “strengthens the researcher’s claims for the validity of the conclusions drawn where mutual confirmation of results can be demonstrated.” (Bryman cited in Gray & Densten, 1998:420). It is argued that this is a more useful approach as techniques can be selected on the basis of their suitability to the topic, standing in direct contrast to alternative claims that this cannot be the case. As Bryman (2001:446) observes, because research methods are viewed as autonomous of their underlying paradigms, it is possible to combine these strategies. Summarising the writings of several writers, Sale et al (2002:46-47) argue that quantitative and qualitative methods are compatible because: they share a unified logic and therefore the same rules of inference apply to both; both qualitative and quantitative approaches share a goal of understanding and improving the world and the human condition; they both have goals of

disseminating knowledge for practical use; they both have a commitment to rigour and they both involve conscientiousness and critique in the research progress. They also argue that the complexity of phenomena (including most public health problems / social interventions) require the use of a broad spectrum of qualitative and quantitative methods. Another way to address these differences may be to acknowledge that more philosophical debates around the epistemological and ontological differences between approaches are important. However, instead of letting them prevent multi-methodological research getting done, these may be taken into consideration when designing research and returned to when interpreting findings.

This dichotomisation of approaches may be a simplification of a much more complex debate. However, it is a useful model for justifying the adoption of a multi-methodological approach to this work. While I have some reservations regarding the extent to which the paradigms are in fact convergent, the argument for a mechanical, convergent paradigms approach which allows the use of mixed methods is a strong one. Mendlinger & Cwikel (2008) suggest that events such as international conferences and the founding a mixed- methods journal all evidence the increasing acceptability o f such approaches across the fields of social sciences and health care.

While pragmatic approaches overcome issues regarding whether we should mix methods, doing so demands consideration of how differing approaches may be best combined. Despite increasing acceptance and use of mixed- methods research, the field is still very much a developing one (Tashakkori & Creswell, 2007) and there is a lack of consensus as to nomenclature and no standard protocols on how best to successfully combine qualitative and quantitative methods (Mendlinger & Cwikel, 2008). In light o f this, the work of Teddlie & Tashakkori (1998, 2006) is useful. They argue that since the emergence of mixed-method research, those working in the field have developed typologies of mixed designs (2006). Although they can never be exhaustive, one o f the reasons such typologies are useful, they argue, is because they help researchers to design their mixed-method studies (2006).

Tashakkori & Teddlie’s own typology has developed over time (e.g. 1998, 2006) and provides a useful classification of mixed methods research based upon the combination of methods. They argue that mixed methods approaches traditionally enable comparison of quantitative and qualitative data either simultaneously or sequentially in order to improve analysis (1998). The sequential combination of methods consists of either converting qualitative data into numerical codes for statistical analysis (quantitizing techniques producing quantitized data) or converting quantitative data into narratives for qualitative analysis (qualitizing techniques producing qualitized data). In 2006, they outlined a 2x2 matrix detailing a typology of mixed-method research designs. Along one side of the matrix, research design could be either

monomethod (study uses either a qualitative or a quantitative approach only)

or mixed method (qualitative and quantitative methods are mixed across the study). Along the other side o f the matrix, research design could be either

monostrand (there is only one strand to the research) or multistrand research

(there are more than one strand to the research). A strand is a phase of study which includes (1) a conceptualization stage (abstract operations including formulation of research purposes; questions etc); (2) a experiential stage (concrete observations and operations such as data generation and analysis and (3) an inferential stage (abstract explanations and understandings including emerging theories and explanations).

Under Teddlie & Tashakkori’s typology, the current study may be described as a mixed-methods multistrand design as it contains more than one method and more than one stage of the research. Table 1.1 outlined the stages o f this study as related to the Research Objectives. It should be noted here that the stages in this table relate method to the Research Objectives, and do not equate to Teddlie & Taskakkori’s definition of a strand. However, such strands are visible within the method, making their typology still useful to understanding the mixed-method approach adopted in the current study. Within Teddlie & Tashakkori’s typology, there are various types of multistrand design for mixed-methods of which the current study can be said to follow a sequential

there are at least two strands that occur chronologically (QUAN? QUAL or QUAL? QUAN. The conclusions that are made on the basis o f the first strand lead to formulation o f questions, data collection, and data analysis fo r the next strand. The final inferences are based on the results o f both strands o f the study ”

[while the authors use the simplest, two stranded example for conciseness, they highlight that a sequential approach may include more than two strands]

Tedlie & Taskakkori (2006:21-22)

Tedlie & Tashakkori argue that, although difficult, such a design is easier to undertake by a solo researcher than other mixed-method approaches as it is easier to keep strands separate and studies tend to unfold both more slowly and predictably than in more complex approaches. Consequently, a sequential, multistrand mixed-method design is a good approach to adopt by a doctoral student seeking to undertake mixed-methods research. Table 1.1 clearly demonstrates how the current study follows this design, with a quantitative phase leading to a qualitative phase leasing to a quantitative phase. Ultimately, the final inferences are based on data generated from all o f these phases. In the remainder o f this chapter, the technicalities surrounding the conduction of the various approaches (i.e. teacher survey; teacher interviews; development and analysis of school- level indicators) are discussed individually. Before this, it is useful to briefly reflect on the relationship between the phases of the study. This will deal with the phases as set out in Table 1.1.

The first stage, relating to Research Objective 1, combines quantitative and qualitative methods sequentially in order to collect rigorous data on the development, content and enforcement of school policies. Although not conducted by the author as part of the current study, the HBSC study is included in this table to show its use within the study. Using Teddlie & Tashakkori’s definition, the first strand is the design, implementation and analysis of the teacher survey in order to collect quantitative data on school smoking policies from several sources in each school. The second strand of the research consists of interviews with smoking policy experts in each school. Although a generic interview schedule was created for these interviews,

analysis of the results of the teacher survey in each school are used to inform the interview schedule in each school so that interviews can be used to probe and follow up these data. In this way, the first quantitative strand is used sequentially to inform the second qualitative strand in order that rigorous data may be collected on school smoking policies.

To fulfil Research Objective 2 the interviews were first analysed in order to identify characteristics of policy and its enforcement that may moderate the extent to which school smoking policies moderate adolescent smoking behaviour. While this analysis stands alone, it was also used in order to develop indicators which described school-level variation in these characteristics (Research Objective 3). This is an accepted practice that Teddlie & Tashakkori refer to as the quantizing the data. Providing several examples of previous studies which q u an tized data, they highlight the different ways in which this may be undertaken (1998; 2006). This includes a school-based study conducted which converted narrative data into likert-type scales. This is very similar to, and, along with the other examples they cite, sets a precedent for the quantititzation of qualitative data in the current study. At this point, the second qualitative strand fed into a third and final, quantitative strand (Research Objective 4) in which the indicators were analysed in order to assess their association with adolescent smoking behaviour. The final analysis then drew directly on the second and third strands (Research Objective 5). Due to problems with the teacher interview (see Section 5.3) the first strand did not contribute as much to the study as intended. However, under Teddlie & Tashakkori’s definitions, it is still arguable that the current study is an example of a sequential mixed-methods, multistrand design as the analysis draws its conclusions from sequentially linked qualitative and quantitative strands

Having justified the adoption of a multi-method and multi-methodological approach, in order to conduct a rigorous investigation of school smoking policies, the rest of this chapter will provide a description of the various elements of research conducted by the author. It should be reiterated here that

while these are portrayed as distinct stages, this is for ease of both project management and discussion only. All phases are interrelated, and the boundaries between them blurred with the movement of data occurring between each stage.