This section provides a critique of existing approaches to coding qualitative research data in the field of political communication with a particular focus upon studies which have sought to identify online deliberation, culminating with an explanation of the reasons that Freelon (2010) was ultimately chosen as the preferred model for the coding and analysis of the research data.
A code, as pertaining to qualitative research has been defined as ‘most often a word or short phrase that symbolically assigns a summative, salient, essence capturing, and/or evocative attribute for a portion of language based or visual data’ (Saldana 2009 p.3) Coding as a research process is the organising of data to be able to effectively conduct research analysis and evaluation, which if done effectively, ought to enhance the likelihood of producing robust results and conclusions. As Strauss (1987 p.27 in Saldana 2009 p.1) states, ‘any researcher who wishes to become proficient at doing qualitative analysis must learn to code well and easily. The excellence of the research rests in large part on the excellence of the coding’. The principal focus of this study is the question of how could you tell if the #indyref was characterised by deliberative democracy or agonistic pluralism. A methodological starting point in doing so is to focus upon existing academic literature which relates to the coding of social media with the purpose of recognising and evaluating, firstly, coding relating to deliberative democracy.
There is a distinct body of coding literature which has developed as social media has increasingly become a recognised arena of political communication. A number of these studies have focused upon coding online political communication for purposes other than identifying the presence of deliberative democracy, as is the focus of this study. These studies do, however, provide insight into a selection of coding methods which can be considered as appropriate methodological designs for research projects focused more broadly upon online political dialogue. Such studies are predominantly focused upon election campaigning and also the utilisation of social media platforms, such as Twitter, by elected politicians.
A number of such studies have built upon Banwart’s (2002) Webstyle content analysis coding scheme, which was developed as a means of assessing gender difference of candidate self-presentation to potential voters through online political advertising. This extensive scheme
Chapter 5 – Methodology 102 includes 58 categories which consist of 212 variables (ibid. p.297-238). Trammell et al. (2006) used an expanded version of the scheme to analyse political blog content, with Wang (2010) again applying the model in assessing online political discourse during Taiwan’s 2008 general election campaign. Other approaches in assessing online dialogue include those at the other end of the spectrum in terms of the level of categorisation and analysis. Grant, Moon and Busby-Grant (2010 p.584) building upon Leavitt et al. (2009) took a broader approach by limiting initial coding categories to those of, ‘broadcast’, ‘broadcast mention’, ‘reply’ and
‘retweet’ in an assessment of Australian politicians’ use of Twitter. In a quantitative study focused upon social media use in the Swiss national election of 2011, Klinger (2013 p.724) again preferred a limited coding scheme with primary cluster categories restricted to
‘information’, ‘mobilisation’, ‘participation’ and ‘other’. Secondary coding in the study then identified sub-clusters of, ‘information’, ‘mass-media references’ and ‘transparency’. Golbeck, Grimes and Rogers (2010 p.1614-1615) used a slightly more targeted approach in assessing Twitter use by US Congress members, with six categories of ‘direct communication’, ‘personal message’, ‘activities’, ‘information’, ‘requesting action’, ‘fundraising’ and ‘unknown'. In a later study of the Twitter feeds of Scottish MPs between 2008 and 2010, Margaretten and Gaber (2012) coded MPs’ tweets as:
Hashtags for evidence of topic diversity; @signs for indications of direct conversations with constituents; RT for retweets, ostensibly to offer additional points of view to the group; URLs for evidence of promoting engagement and mentions for calling attention to another user in the conversation (ibid. p.337).
In a more recent study, again focusing upon politicians in Scotland, though this time analysing MSPs’ use of Twitter, Baxter, Marcella, and O’Shea (2016 p.447) took a more granular approach by initially using four broad categories similar to those already mentioned, but then segmenting data across a further 31 subcategories.
Whilst the works previously mentioned give an insight into broader studies of online political communication, more pertinent to this study are those within a growing body of literature that is unambiguously focused upon coding schemes which can be utilised to identify online deliberation. As early as 1998, Wilhelm devised deliberative coding categories in a study which asked how deliberative online discussion is within Usenet newsgroups. These were based around the principles of Habermasian rational argument (see chapter two), which is also the theoretical basis of a number of the subsequent models which were developed by authors in later studies. Lincoln Dahlberg’s prominent study (2001) coded deliberation in categories of, ‘exchange and critique of reasoned moral-practical validity claims’, ‘reflexivity’,
Chapter 5 – Methodology 103
‘ideal role taking’, ‘sincerity’, ‘discursive inclusion and equality’ and ‘autonomy from state and economic power’. Graham and Witschge (2003), were writing at perhaps the peak of increased optimism regarding the potential for the internet and social media to foster online political deliberation. Their study sought to ‘develop a method for examining the extent to which internet forums meet the normative requirements of rational-critical debate, reciprocity and reflexivity’ (ibid. p173). Trenel (2004) formulated a detailed coding scheme that comprised of, ‘equality’, ‘rationality’, ‘respect’, ‘constructiveness’, ‘interactivity’, ‘personal experience’,
‘emotional balance’ and ‘reflexiveness’. Janssen and Kies (2005 p.326-330), presented an overview of existing research which evaluates the quality of online political discussions with which they aimed to operationalise methods of measuring online deliberation. The coding characteristics they identified, followed what was becoming an emerging pattern, including,
‘reciprocity’, ‘justification’, and ‘reflexivity’. However, they also included codes of, ‘ideal role taking’, ‘sincerity’, ‘inclusion and discursive equality’, and ‘autonomy from state and economic power’.
In an inductive study which sought to depart from deductive notions of deliberation, typically taken from theoretical standpoint of authors such as Habermas, as already mentioned, Mansbridge et al. (2006 p.18-34) identified coding categories of, ‘reason and emotion’, which was a nuanced but important departure from ‘rationality’, alongside ‘common good vs.
common ground’, ‘free flow’ and also three separate facets of equality, those being, ‘extensive and inclusive participation in discussion’, ‘self-facilitation and group control’ and ‘fair representation of views’. Stromer-Galley (2007 p.4-7) devised a coding scheme which was designed to measure deliberation in both face-to-face and online settings. The coding scheme consisted of, ‘reasoned opinion expression’, ‘disagreement’, ‘equality’, ‘topic’ and
‘engagement’. In a significantly more detailed dialogical framework, devised by Graham (2008), a three-phase process was formulated and is detailed in the following diagram:
Chapter 5 – Methodology 104
Table 6 Coding Scheme Overview (Graham 2008)
The three distinct phases of the coding scheme were designed to firstly identify message types, followed by sub-categorisation of ‘reasoned’, ‘non-reasoned’, and ‘non-claim’ replies, with the final phase assessing, ‘communicative empathy’, ‘discursive equality’, ‘discursive freedom’ and ‘sincerity’. Most recently, Halpern and Gibbs (2013 p.12-13) focused upon the likelihood of both Facebook and YouTube as catalysts for online deliberation. In doing so, they included codes of ‘logic and reasoning’, ‘conversational coherence’, ‘equality of participation’, ‘politeness’, ‘civility’ and ‘message length’.
In summary, these different approaches to coding online deliberative democracy tend to be based upon Habermasian concepts of rationality, reciprocity, reflexivity and derivations of equality. In contrast, more granular coding schemes which take a more detailed discursive approach include numerous further codes which deconstruct types of arguments and responses, as well as levels of communicative sincerity. It is important to note, however, that the authors of these studies found some significant difficulties with operationalising such coding categories. For example, a common issue was the difficulty in assessing reflexivity within a discussion as reflection, in most cases, probably takes place outside of the discussion forum as individuals reflect on the points made by others, which potentially renders reflexivity as intangible within the textual data posted in the online forum. Authors such as Given (2008
Chapter 5 – Methodology 105 p.26) sought to overcome this through considering rebuttals and refutes to incorporate a certain level of reflection, this, however, could be classed as a tenuous assumption in terms of genuine rational reflection upon the issue in question. The suggested remedies to the issue by other authors, such as Wilhelm (1998) were the triangulation of online data with research interviews in order to question individuals on their propensity to reflect during such exchanges, whilst Jensen (2003, in Janssen and Kies 2005) asked research subjects to document reflection in supplementary research surveys (other such coding issues are discussed as part of the coding scheme critique in the following section).
On consideration of all of the approaches detailed here, a final decision was made to proceed with a comparative model formulated by Deen G. Freelon titled,- Analyzing online political discussion using three models of democratic communication (2010):
Model of democratic communication Indicative metric
Liberal individualist Monologue
Personal revelation Personal showcase Flaming
Communitarian Ideological fragmentation
Mobilization
Community language Intra-ideological questioning Intra-ideological reciprocity
Deliberative Rational-critical argument
Public issue focus Equality
Discussion topic focus Inter-ideological questioning Inter-ideological reciprocity
Table 7 Models of online democratic communication and their indicative metrics (Freelon 2010)
This model was particularly appealing in that it was conducive to the project given that it followed the less complex dialogic approaches detailed in this section, which whilst appealing in terms of rigour, would have been inappropriate in a relatively longitudinal study of an open forum such as Twitter, where the focus upon discussion topics was particularly wide-ranging.
Chapter 5 – Methodology 106 Also, the opportunity to assess the data which had been collected during the project against the norms of liberal individualism and communitarianism, which are also contained in Freelon’s model, offered additional perspectives to which the Twitter debate during the independence referendum could be benchmarked against in addition to the central model of deliberative democracy. Freelon’s work sought to introduce a new framework of analysis building on existing models proposed by Jürgen Habermas (1996) and Lincoln Dahlberg (2001). The framework is based upon three distinct yet overlapping strands of democracy namely, the liberal, the communitarian, and the deliberative democratic. In so doing Freelon claims that:
This framework enhances the ability of researchers to contextualise disparate online discussion cultures with respect to one another, characterise particular cases in terms of distinct scholarly conceptions of democracy, and testing existing theories of online political communication in new ways (Freelon 2010 p.1173).
Such a framework, the author suggests, is necessary in response to the proliferation of deliberative models (such as those recently discussed) which fail to take into account many of the features of online political expression which exist outside of, or in contrast to, the deliberative model (the three strands of the model are classified by the indicative units in table 7). Freelon’s model was used to code both the Twitter and interview data. This involved taking each separate piece of data and considering which, if any, of the units of analysis it applied to. If it applied to more than one it was designated to how ever many units were appropriate.
At the end of this process each unit of analysis contained a distinct body of evidence which is then discussed under each unit heading in chapter six. This process then, allowed for triangulation of the data in terms of evidence collected from Twitter, tested against the accompanying discussions as taken from the interviews which could in turn be compared with the theoretical assertions identified in the earlier part of the project.
The structured analysis of the data gave a strong indication of the relevance of Freelon’s units of analysis to this particular study. The vast majority of the units of analysis identified recurring themes and issues that will be discussed in the following results chapters of the thesis. This was not, however, the case with all of the indicative units in the table above. There were different reasons for taking these decisions, ranging from the particular nature of the referendum debate rendering the units as inappropriate, such as Freelon’s definition of equality (see the following section for the redefined metric in this specific context) to the absence of evidence of the metric being significant in itself, such as public issue focus, as the lack of evidence specifically defined the nature of the debate. Each metric/unit of analysis is given a heading in the results chapter relating to the primary research question and
Chapter 5 – Methodology 107 explanations are provided if the unit of analysis is omitted or restricted in the volume of evidence which is present. At the end of chapter 6, a critique of Freelon’s model is presented and detail is given as to the positive and negative outcomes of the application of the model to the data collected within the study. The next section further details Freelon’s (2010) model and includes a critique of the application of the model to the data analysis during this project, which is intended to aid future scholars in terms of replicability of the methodology of this study in future projects.
As a recognised, though a generally more contentious model of political communication theory, as part of the review of existing models and coding schemes of deliberative democracy, similar searches were made to identify and analyse coding schemes focused upon agonistic pluralism. This search, however, failed to find any such models in the academic literature. As a result of this, the final coding scheme (which follows in the next section) incorporates the process of considering the current strands of analysis of liberal individualism, communitarianism, and deliberation in relation to the norms of agonistic theory.
Again, in chapter six, which is the first results chapter, the results are further assessed against the norms of agonistic pluralism and the conclusions to chapter six includes the suggested structural basis of such a model, which it is argued would complement existing strands/models of democratic communication.