‘Quantitative research is hard and reliable … qualitative research is deep and rich’
(Bryman, 1996:94)1
‘Qualitative and quantitative methods are more than just differences between research strategies and data collection procedures. These approaches represent fundamentally different epistemological frameworks for conceptualising the nature of knowing, social reality, and procedures for comprehending those phenomena’.
(Filstead, 1979:45)2
Teaching and learning objectives:
1. To understand the origins, merits, strengths and weaknesses, claims and counter-claims of qualitative and quantitative research.
2. To consider whether ‘mixed methods’ are incompatible with best research scholarship.
3. To enable each student to reach and defend their own preferences.
Introduction
Qualitative and quantitative methods have historically been portrayed as mutually exclusive research approaches promoted by competing specialists appealing to the hearts and minds of new students.
The arguments and antagonisms have become very tired over the years as each new cohort of students is asked to critically compare the competing claims. However, the traditional divergence of opinion has been challenged by the increasing interest expressed by Politics researchers in ‘mixed methods’ and the availability of sophisticated CAQDAS (Computer-Assisted Qualitative Data Analysis System) software. This uses quantitative methods to analyse qualitative data.3These include programs which systematically analyse ‘talk and text’. However, they depend
42 Research Methods in Politics
on systems of coding designed and applied by the researcher. So the analysis is not therefore automatic. They also rely on assumptions, especially content analysis software, about the significance of the frequency and regularity of words which are essentially positivist.
The full, definitive discussion of the claims and counter-claims of qualitative and quantitative methods is provided by Alan Bryman’s Quantity and Quality in Social Research, London, Routledge, 1996/88. Read it.
This chapter seeks to summarise briefly the origins, characteristics, merits, strengths and weaknesses, claims and counter-claims of qualitative and quantitative methods. You will already be aware of the particular pitfalls and incoherence of comparing two or more concepts. The conventional method of comparative, descriptive narrative can become easily weighed down with repeated use of ‘on one hand’ and ‘on the other hand’. A weakness of text is its linear, one-dimensional presentation of information. This makes ready comparison difficult and time-consuming. An alternative approach, which this book advocates for writing research reports, is to tabulate (set out in a table) for ready comparison the core characteristics of the concepts. Tabulation enables specific aspects to be extracted for detailed discussion. This approach is used in Table 4.1 to highlight the essential differences between the two methods.
The strengths and weaknesses of the two research methods are not mirror-images of each other and are best considered separately.
Quantitative methods: strengths
The greatest strength of quantitative method lies in its general acceptance by others as being rational, logical, planned and systematic. The findings are regarded as credible. This method is therefore particularly favoured by public, research-funding bodies keen to justify their investments to a sceptical public. It is also preferred by news media whose audiences, they claim, regard percentages as ‘hard news’.
Therefore quantitative method, however complex the statistical methods employed, is regarded as being straightforward and providing the facts. The researcher is seen as dispassionate, objective and, therefore, trustworthy. It employs very large samples designed to reflect and be representative of the population being studied. The use of questionnaires ensures that every member of the sample is asked the same question in the same manner. Supporters argue that attitudes can be measured by using scales, e.g.
the Likert scale of strongly agree, agree, etc. (see Chapter 9). Face-to-face contact is not necessary and may contaminate the data. Research objects are best kept at arm’s length. Geographically remote or immobile people can be surveyed by postal survey, telephone or internet. Quantitative method makes best use of computers and other new technology. So the time-scale for data analysis and the publication of findings can be relatively short. Furthermore, the data can be re-examined, audited
Qualitative Versus Quantitative Methods: A Relevant Argument? 43
Table 4.1 Characteristics of qualitative and quantitative methods
Characteristics Quantitative Research Qualitative Research
Origins Natural science Social science
Philosophic roots Positivism Naturalism, feminism
Research design Deductive Inductive
Systematic Flexible
Contribution to theory Theory testing Theory building Seeks explanation and
universal laws
Seeks explanation and solutions
Researcher Observer, outsider Participant, insider Researcher’s attitude Objective Subjective, empathetic
People are regarded as: Objects Subjects
Location Research centre or
laboratory
Field based
Samples or cases Random Non-random
Large Small
Focus on universes Focus on minorities
Focus The individual The group (family, clan, peer group,
work group)
Records Frequency Social meanings
Data Numeric concept indicators Non-numeric concepts Data collection Surveys and structured
questionnaires
Personal interviews and unstructured interviews Represented by Tables and charts Transcripts
Analysis Statistical Non-statistical
Generalisability High None claimed
Costs High Low
Findings Nomothetic (general laws) Ideographic (specific to historical and cultural context)
and re-analysed or used for other purposes. The process of analysis is overt in which the scope for bias by the researcher is deliberately minimised. The method enables research projects to be carried out by teams in which specialist talents can be properly exploited and work sub-contracted to agencies. Quantitative research effectively conforms to the modern-day business model.
In terms of academic research, quantitative method is particularly suited to the development of grand, meta and micro theory by testing logical hypotheses.
Applying Popper’s tests of falsifiability ensures that the research hypothesis can only be accepted after all alternative hypotheses have been disproved. It offers the best means of identifying and comparing the distribution between people, places and times of phenomena such as party membership, voting, income, poverty, housing conditions, and changing attitudes. It can also be used to identify clusters of relatively small-scale phenomena and to analyse these statistically to identify whether
44 Research Methods in Politics
particular concentrations may be attributed to chance or potentially localised causes.
Quantitative method can mine and analyse existing data banks, for example, census data and voting records. It is particularly useful in electoral research which focuses increasingly on the problem in western states of declining turnout in what commentators have termed the ‘period of post-democracy’ (Crouch, 2004).4 One notable study used the data available from the British Election Study (BES) and local government records to explain potential causes of the drop in UK turnout in the 1997 general election (when, after 18 years in opposition, Labour replaced the Conservative party in government). The drop, from 77.9% (in 1992) to 71.6%, could be attributed mainly to the combination of the closeness of party ideology and the extent to which the outcome was in no doubt (Heath and Taylor, 1999).5 The apparent certainty of the result was reported by news media using the evidence of opinion polls – another quantitative device. In this way, ‘hard news’ became self-fulfilling.
Weaknesses and criticisms of quantitative methods
These are mainly fivefold. First, the underlying doctrine of positivism is, as already stated in Chapter 3, contestable in its application to the social world. Second, it is too detached, remote and clinical to really understand and explore the complex social and political world. Quantitative research is amoral. Third, its use by the social sciences does not meet the high standards of the natural sciences in which its reputation and claims lie. It is ‘bad science’. The samples, despite claims of randomness, are not statistically reliable. There are too many variables for causality ever to be determined, especially when the measures are superficial. Measures of strength of attitude are fallible: is ‘strongly object’ twice the value of ‘object’? Fourth, quantitative research in Politics research relies on the ability to express concepts as measurable indicators.
But, as already discussed, how can power be measured? Furthermore, official data is often ‘doctored’ to serve government interests. To what extent are records of, say, civil disturbances and arrests really representative of underlying dissent? Can we really weigh an iceberg by measuring its tip? The necessity for measurable concept-indicators means that quantitative research is confined to researching measurable variables rather than more important issues. Lastly, the reliance on observation limits the range and depth of research to what is both observable and measurable.
The stereotypical quantitative researcher is a clean-shaven, male, white-coated, ascetic, Times-reading Dr Strangelove working in an ‘established university’.
Defence and counter-claims
Advocates of quantitative research rebut the criticisms by defending the applicability of positivism and empiricism to most aspects of the social world. It is less imperfect than qualitative research. In purist terms, it is the evil of two lessers. But samples
Qualitative Versus Quantitative Methods: A Relevant Argument? 45 can be devised and statistical analysis applied to produce findings within specified
confidence limits. Questionnaires can be designed with sufficient checks and controls to identify bias by respondents and questioners. Most concepts are capable of being represented by measurable indicators. Subconscious feelings and motivations can be identified by using projective questioning (see Chapter 9). Quantitative method accepts its own fallibility which its users consciously seek to minimise. Most variables are capable of observation by trained researchers using modern technology. At the end of the day, quantitative research is reliable and concrete and has ‘put the science’
into Politics to create political science.
Qualitative methods: strengths
This is the dominant approach adopted in UK Politics research. This choice reflects in part the lack of confidence of many UK researchers in their quantitative skills.
In the main, qualitative method is preferred because it is considered best suited to the study, understanding and explanation of the complexities of social and political life. The strength of this method lies in its unique capacity, through in-depth interviewing and observation, to learn and understand the underlying values of individuals and groups. It better enables theory to be created by induction. By learning the social meanings that the subjects apply to their world, researchers are better able to ‘see the world through the subject’s eyes’. So researchers can identify and understand the interpretative lens that subjects adopt, and, therefore, the dominant powers and institutions that frame the view and tint the lenses.
Essentially, politics takes more than one person: it is an intra- and inter-group activity. Qualitative method enables the focus to be shifted from the individual to the group(s) and to learn (following the theories of Goffman and others) how meanings are negotiated between members and the group dynamics involved.6 It allows comparisons and distinctions to be drawn between what the individual says in the privacy of a personal interview and what they say, or don’t say, in a group. Furthermore, the varied opportunities for participant observation, along a continuum between thresholder and full member, allow degrees of access to the group.
The method also enables minorities to be researched who would otherwise be missed by sample surveys because they are either small in number or might be unwilling to identify themselves. Covert groups include homosexuals, drug-users, gang-members, and terrorists and their supporters. The method avoids distance and objectivity by seeking verstehen (empathetic understanding). Many of the researchers are also (critical) activists. They seek to expose exploitation and to improve the lives of their subjects through policy change. They want to assist their subjects to see their circumstances through other perspectives and lenses, and to develop their own interpretations. They seek to empower their subjects through the research process.
There is therefore a normative dimension to qualitative method that quantitative
46 Research Methods in Politics
method lacks and, for that matter, avoids. The method also offers a greater focus on verbal and other communications and the application of linguistics to the analysis.
Sophisticated (digital) recording devices enable the data to be shared with other researchers. New, miniaturised technology allows fieldwork to be carried out in the homes and workplaces of their subjects in a more surreptitious way. So the subjects are not inhibited by the equipment and contingent variables are less affected.
Criticisms and weaknesses of qualitative methods
The archetypal criticism of qualitative methods is that the data collected is largely anecdotal or exaggerated. The method is also ‘soft’: it lacks the intellectual and operational rigour of quantitative method. The researcher is likely to ‘go native’
(over-identify with the subjects) or, alternatively suffer, from observer drift (where obvious deviances become accepted norms that are under-recorded). So the data and findings are highly unreliable. The claims to induction are spurious: the research question and choice of issues and concepts must be influenced by prior learning.
The researcher is weighed down by bias. By participating with their subjects, the researcher will inevitably contaminate the social field and determine outcomes, or at least influence them in some way, rather than observe them. Furthermore, despite their distaste for concept-indicators, qualitative researchers inevitably adopt concepts which involve some form of proxy to make them intelligible to the subjects. Verstehen is elusive. How can a young, white, middle-class male researcher ‘see the world through the eyes’ of a female with the same background let alone a person from a wholly different culture? But, even if the data is sound, there is the insurmountable problem of interpretation. The researcher must interpret in turn the interpretation by the subject of their world. The scope for misinterpretation is huge. How can the researcher know or show that their interpretation is inaccurate? There is also the ethical danger that, by participating closely in oppressed minorities, the researcher may act as an agent provocateur who leads the group into harm’s way and then retires to the sanctuary of their university.
The stereotypical qualitative researcher is seen by critics as long-haired, earring-wearing, leather and denim-clad, Guardian-reading, smoking, Dr Howard Kirk (The History Man) in a ‘new university’. Or, worse still, he is a woman.
Defence and counter-claims on behalf of qualitative research
Advocates reject the criticisms made as wholly exaggerated. The social sciences are not a soft option. Qualitative research is intellectually demanding and rigorous.
Researchers are fully aware of the weaknesses and dangers of close identification between subject and researcher. Protocols and close supervision are required. Data is not anecdotal. It is no longer accepted on trust: the researcher is required to make
Qualitative Versus Quantitative Methods: A Relevant Argument? 47
Illustration 2 The great divide: Quantitative versus Qualitative Research. The stereotypes
available full transcripts and other records of interviews and group meetings for corroboration. There is a full paper trail available for audit purposes and external examination. The research method provides the only means by which overlooked or concealed minorities can be studied. In this way, their needs for special recognition can be addressed by policy-makers before continued rejection leads to outright dissent. The emphasis on studying subjects in their own habitat means that contingent factors can be identified and the research effect (see Chapter 2) minimised. It recognises that language is never neutral: ‘every word is a bias’ (Nietzsche). The normative opportunities should be welcomed rather than criticised: research should improve the circumstances of its subjects. The method acknowledges the central role of the researcher in the research rather than pretending that this can be eliminated.
Quantitative method does not readily enable group behaviour to be studied. Overall, qualitative research provides high quality data and findings, and deep, meaningful insights into underlying values, fears and motivations of agents and actors in the political world.
Mixed methods
Given the separate strengths and weaknesses of quantitative and qualitative methods, then using them together – mixed methods – would appear to offer the best of
48 Research Methods in Politics
both worlds. But, given their opposing ontology and epistemology (see Chapter 3), the criticism could be made of the lack of scholarship and the inevitable theoretical weaknesses of seeking to mix oil and water. Alternatively, the criticism could be rebutted as being too esoteric and academic and of overlooking the advantages, especially of time and other costs, of bringing the two methods together when the opportunities arise. The justifications cited for a mixed approach could include synergy, corroboration of sources and data triangulation (see Chapter 7).
The received view among research practitioners tends to be that the differences between the methods have been exaggerated. However, there remains strong objection to a ‘mix and match’ approach. Instead, the use of combined methods can be better accepted as appropriate where one method dominates and the other is used in a secondary, supportive way.
Two examples of appropriate combined methods can be given. Both concern research into voting behaviour.
EXAMPLE 1: Election turnout
UK election turnout has fallen since 1951.The research question is: why has turnout fallen? It adopts a positivist, deductive approach in which Crouch’s theory of post-democracy is used and tested. The research seeks to test Crouch’s argument that turnout has fallen because globalisation and the decline of industrial production have loosened the previous sense of antagonistic social identity and class. So the appeal of unions and political parties has weakened and reduced support for active participation. Instead participation has once again become an elite activity by ‘self-referential political class more concerned with forging links with wealthy business interests’ (Crouch, 2004).7In this case, the proxy-indicators of active participation selected are: membership of parties and unions; attendance at political etc.
meetings; holding office; and, voting. The data will be collected from national published records of voting and party, etc., membership since 1951. This population data will be augmented by a national, postal questionnaire sample of a stratified sample of say 2,000 electors (following a pilot survey). It may offer hard, reliable evidence of changes in voting behaviour, class consciousness and party identity and participation. Analysis is likely to show a strong correlation between the decline in participation and loosening of political identity. But it will lack compelling evidence of causal relations. In this case, the data could be supported by interviews with a small sample of activists and non-activists to learn of their personal experiences and to tease out their changing motivations. This qualitative data could either be sought before the questionnaire survey to help frame the questions and alternative answers, or afterwards to explore particular findings. In either event, ‘quotable quotes’ could be incorporated into the research report to illustrate the findings and to create interest. The report is likely to conclude with a discussion of the implications of the findings for ‘democracy’
and electoral reform.
Qualitative Versus Quantitative Methods: A Relevant Argument? 49
EXAMPLE 2: Increasing abstention
The second example also concerns the fall in UK turnout since 1951. The research question remains the same although expressed in a slightly different way: why are more people abstaining from voting? But this time, the research is inductive: there is no hypothesis to be
The second example also concerns the fall in UK turnout since 1951. The research question remains the same although expressed in a slightly different way: why are more people abstaining from voting? But this time, the research is inductive: there is no hypothesis to be