Department of Telecommunications Indiana University
The job of the political communication scholar is to improve and enhance our understanding of a complex set of communicative dynamics that influence political outcomes, both as means and as ends. The political outcomes studied within the field represent the spectrum of effects, from the most basic matters of exposure and attention (e.g., news use and reliance) to more advanced outcomes like political behaviors (e.g., voting and other forms of participation) or post-behavior cognitive integration (e.g., perceptions of campaign legitimacy, perceptions of vote count accu-racy). The communication theories developed to study a broad range of phenomena continue to expand and evolve, and these theoretical developments are taking place within the context of an ever-changing communication landscape (Bimber, 2003; Campus, Pasquino, & Vaccari, 2008;
Holbert & Benoit, 2009; Prior, 2009; Scheufele, 1999).
The evolving nature of the communication environment has caused the field to engage in a debate about which conceptual frameworks are the most useful in assessing communicative influence in politics (see Bennett & Iyengar, 2008, 2010; Holbert, Garrett, & Gleason, 2010).
This debate has centered on how the field’s research can remain accessible, or “interpretable, cumulative, and socially significant” (Bennett & Iyengar, 2008, p. 709), to relevant audiences.
The long-running debate over how to properly conceptualize the past, present, and future of the effects traditions in political communication is beyond the scope of this Sourcebook (but see Bucy
& D’Angelo, 1999, 2004). However, within this project’s purview is a detailed examination and explication of the advanced methods employed by political communication scholars to analyze theoretical propositions and measure communication dynamics at play at the media–politics interface.
On one level, methods represent a constant within the field of political communication.
Communication is a variable field (Paisley, 1984), so the use of a wide range of methods has been standard practice since the field’s inception (see Chaffee, 1975; Nimmo & Sanders, 1981). In addition, communication straddles the boundary (somewhat artificially constructed at times) between the humanities and the social sciences. Thus, methods commonly employed across both of these much broader epistemological foundations are treated as common within the interdisciplinary field of political communication. In short, methodological diversity has been an integral part of the field for many years and all signs indicate that this will remain the case for the foreseeable future.
However, on another level, the foundational methods commonly employed in political communication (e.g., survey research, experimentation, content analysis) are continually progressing, and it is becoming the informal duty of every political communication scholar to maintain a firm understanding of the latest advancements in method to enable the most rigorous analysis of the core questions driving individual research agendas. Toward this end, this Sourcebook serves as a vehicle by which to introduce political communication researchers to some of the latest advances in methods, measurement, and analytical techniques. The broad range of methods commonly employed in the field are well represented in this volume, but many of the topics covered represent innovations and novel applications of existing techniques that together are pushing and advancing research. For intellectual vibrancy it is important that political com-munication scholars remain ever vigilant in advancing and refining the varied methodological tools at their disposal. Indeed, the advancement of method is a necessary condition enabling the field to progress and remain “interpretable, cumulative, and socially significant” over the long haul.
Survey methods remain a dominant methodological tool for political communication analysis.
A recent content analysis conducted by Evans and Bucy (2010) of peer-reviewed articles appearing in two of political communication’s primary journals, Political Communication and the International Journal of Press/Politics, revealed that more than a quarter of the articles published in these outlets over the past decade retained some type of survey method as their primary analytical tool. As many scholars have observed, the study of communication is about the study of process, and advanced survey designs allow for ever more precise ways to assess communication influences.
The traditional cross-sectional survey design, which claims a long and storied history within the field, retains several distinct disadvantages that prevent political communication researchers from truly understanding processes and effects that unfold over time (Holbert & Stephenson, 2008). The persistent use and deployment of cross-sectional survey designs to this point has left many core causal questions unresolved. Through the use of more advanced longitudinal and panel designs, political communication scholars are beginning to wrestle with some of the field’s most basic causal issues (e.g., Eveland, Hayes, Shah, & Kwak, 2005), of which there are many.
ORGANIZATION OF THE SOURCEBOOK
The first section in this volume examines important developments in survey research. Two chapters in particular address how political communication researchers should approach multistage survey designs that, for the most part, overcome the limitations of cross-sectional designs. In Chapter 2, Eveland and Morey spell out in detail the value of panel designs in addressing empirical questions posed by political communication scholars. In addition, they offer a series of examples to illustrate the appropriate use of panel data. Most importantly, they stress that an advanced methodological design means little without a solid theoretical foundation. For example, these researchers argue that the answers to basic panel design issues (e.g., What time lags should be used between different phases of data collection?) can only be addressed in a valid and reliable way through the use of
theory. Theoretical considerations should guide the decision as to whether the time lag between the data collected at Time 1 and Time 2 should be an hour, a day, a week, a month, or a year. Even the most sophisticated analytical technique can’t make up for a lack of theory or weak conceptualization, and a given methodological tool is best utilized when given solid conceptual grounding. These principles guide the topics covered throughout this Sourcebook.
In Chapter 3, Kenski, Gottfried, and Jamieson summarize the use of the rolling cross-sectional survey design to enhance our understanding of political communication processes within the election context. The rolling cross-sectional design serves as the foundation for the National Annenberg Election Studies (NAES), and Wagner (in Chapter 6, this volume) highlights the key strengths and weakness of the Annenberg approach compared to the American National Election Studies (ANES) surveys, which were long conducted through the University of Michigan’s Institute for Social Research and now involves a collaboration between Stanford University and Michigan. Kenski and colleagues review several of the fundamental issues involved in designing a rolling cross-sectional survey, highlighting the four primary steps to constructing such a design:
determination of the total number of target interviews, creation of replicates, establishment of time intervals between replicates, and equal treatment of all subsamples. In addition, these authors discuss the choices researchers need to make relative to the resources available and how, again, theory should inform key design decisions, including the total number of target interviews or establishment of proper time intervals. The strengths and weaknesses of rolling cross-sectional designs are also compared against the use of other more advanced survey designs, such as repeated cross-sectional designs and panel designs.
Discussion of more advanced survey design issues does not mean that political communi-cation researchers should gloss over other important considerations, including the measurement of core concepts, the use of the “don’t know” response, and minimizing response biases and response errors. In Chapter 4, Hoffman and Young emphasize the need for political communi-cation scholars to remain focused on the basics, even while pushing the boundaries of survey research through the use of more advanced designs. Foundational concepts such as media use, voting, political knowledge, and sophistication are often the most difficult to operationalize in research, and Hoffman and Young perform a valuable service by addressing the strengths and weaknesses of various survey-based approaches used in the literature to measure these constructs.
Even the most sophisticated of survey designs is likely to remain ineffective if proper attention is not given to core concerns, a point which Hoffman and Young make abundantly clear.
Another insight revealed by a systematic review of political communication journals is the prevalence of secondary analysis in the field (Evans & Bucy, 2010). With cross-sectional or
“one-shot” survey designs, secondary analyses are just about on par with the use of primary data by political communication researchers. With panel data, secondary analyses far outweigh the use of primary data. This is understandable given the additional resources of time, money, and research effort needed to collect panel data. Regardless, a reasonable conclusion can be reached that secondary analyses, given their abundance in the field, provide some utility in addressing a broad range of political communication inquiries. Consequently, the field should give secondary analysis its due as a legitimate method, much like the credit it receives in political science, and continue to focus on refining analytical techniques for the analysis of existing data sources.
On this point, Holbert and Hmielowski make an argument in Chapter 5 that secondary analysis should be viewed as a creative act. A common criticism of secondary analysis is that it is not as original or intellectually creative as doing primary research where the researcher is in full control of design, measurement, and analysis (see Kiecolt & Nathan, 1985). This criticism is addressed by Holbert and Hmielowski with a nine-part typology consisting of three stages of secondary analysis (choice of data, variable creation, and data analysis) and three dimensions of
creativity (novelty, effectiveness, and authenticity), as detailed by Averill (2002). Nine studies that reflect a specific mix of secondary analysis stages and creative dimensions are highlighted in Holbert and Hmielowski’s examination in support of the argument that performing secondary analysis represents something original and inventive.
Interdisciplinary Considerations
The field of political communication not only cuts across multiple levels of analysis and a wide range of epistemological foci, but it is also unique in terms of straddling two well-established disciplines, communication and political science. Indeed, discussions of how the two fields of study approach the topic of political communication have been given careful consideration in recent years (see Benoit & Holbert, 2010; Bucy & D’Angelo, 2004; Graber & Smith, 2005). In Chapter 6, Wagner highlights how the widely influential (and older) ANES and important (but newer) NAES embody the differences between the approach political scientists and com-munication scholars take in studying election campaigns. Distinct in design, the ANES and NAES data present some unique opportunities and definitive advantages while maintaining their own idiosyncratic limitations, although they both remain amenable to periodic change and inclusion of new items (e.g., extended panel designs across election cycles, off-year pilot study designs).
Choosing a suitable dataset is a foundational task for those engaging in secondary analysis and Wagner’s chapter provides political communication scholars with an informative framework for deciding how to utilize the unique qualities of both the ANES and NAES.
The field of political communication has reached a point in its development where well-defined bodies of evidence are being amassed in relation to several core questions driving the discipline (e.g., the relationship between news use and knowledge gain, or the influence of political adver-tising on voting intentions). As a result, there is ample opportunity for conducting meta-analyses of existing studies to determine broader patterns of political communication processes and effects.
In Chapter 7, Allen, D’Alessio, and Burrell provide a summary of the unique epistemological value of conducting a meta-analysis. As the field of political communication continues to expand, one would expect meta-analysis to grow in popularity as an analytical technique. And, indeed, it appears that we are beginning to see an increase in the number of published meta-analyses in our journals (e.g., Benoit, Hansen, & Verser, 2003; D’Alessio & Allen, 2007; Hansen & Benoit, 2007). Equally important from a methodological standpoint is for research to continually refine the procedures involved in meta-analysis. With increased utilization, a broader group of scholars will be able to gain a more comprehensive understanding of a core set of relationships deemed important. Allen, D’Alessio, and Burrell go a long way toward aiding scholars in achieving this goal.
At the individual level, political communication has always embraced the use of experimenta-tion (Simon & Iyengar, 1995). Compared to early experimental studies, the designs employed in current research have become significantly more nuanced and complex. In addition, a new level of creativity is being demonstrated in the manner by which experiments are being conducted. In Chapter 8, Iyengar details how political communication scholars can employ new media software and Web-based platforms to better address research questions that call for the causal inference that experimentation allows while at the same time reaching a probability sample of subjects. With hybrid survey-experimental designs and computer software that enables participants to take part in a study within a familiar setting and at their convenience, a new era of experimenta-tion has arrived. As a result, our understanding of political communicaexperimenta-tion processes stands to benefit enormously. As shown by some of Iyengar’s own work conducted within his Political Communication Lab at Stanford (http://pcl.stanford.edu), as well as by pioneering research funded
by the Time-sharing Experiments for the Social Sciences (TESS) program sponsored by the National Science Foundation (http://tess.experimentcentral.org), new media are offering consid-erably expanded opportunities for experimental and hybrid survey-experimental research.
Many of the core research questions driving the study of political communication focus on behaviors. However, there is a clear conceptual distinction between behavorial intention on the one hand and actual behavior on the other. In Chapter 9, Krupnikov and Levine explicate the difference between expressing behavioral preferences (i.e., intentions) and revealing preferences (i.e., performing actual behaviors of interest), detailing how various political communication research projects have focused on the former at the expense of verifying what amounts to behavioral promises.
In addition, these authors review the comparative advantages of various experimental designs suitable for measuring each type of preference. Most importantly, Krupnikov and Levine provide a series of remedies for maximizing the strengths and minimizing the weaknesses of analytical techniques when examining expressed versus revealed preferences in experimental research.
Building on the experimental study of political behavior, Stewart, Salter, and Mehu in Chapter 10 synthesize the interesting and sometimes overlooked experimental literature on nonverbal communication, focusing in particular on televised leader displays. A broad theoretical argument, grounded in ethology, is offered that speaks to the importance of nonverbal communication in different political contexts, including elections, international crises, and governance. A series of advances in the experimental study of viewer responses to televised leader displays is then detailed.
Stewart and colleagues link the study of nonverbal political communication to several core areas of political communication scholarship, including leadership, emotion, source perceptions, and audience responsiveness. It is worth repeating that methodological advances are best employed within the context of a well-grounded theoretical framework. Stewart, Salter, and Mehu illustrate this point by offering a solid conceptual foundation for investigating a series of questions that will require the field to employ innovative methods and pioneering techniques to study political leaders and other news sources as nonverbal communicators.
Scholars of all stripes, not just political communication, commonly associate the use of experimentation with the identification of “short-term” effects. However, more elaborate designs currently allow for the assessment of long-term effects within the controlled environment of the experimental setting. Theoretical arguments that predict the unfolding of a specific effect over time (e.g., a sleeper effect in persuasion, or long-term memory gain in information processing) call for longitudinal experimental designs. In Chapter 11, Hansen and Pfau detail the strengths and pitfalls associated with conducting political communication experiments that involve multiple contacts with participants over periods of time that could span days, weeks, or months. This chapter should be seen as a corollary to the Eveland and Morey contribution on panel designs. Despite addressing different methodologies, Chapters 2 and 11 examine similar opportunities and limitations, with insights offered by each author team that speak to the uniqueness of survey research compared to experimentation. Most importantly, political communication scholars need to step away from associating experimental research with studying only those effects that are immediately measurable. Experimentation has the ability to assess long-term effects, as long as researchers develop an appreciation for more complex design issues and the delayed or follow-up data collections that are necessary to show long-term effects.
Content and Discourse Analysis
The first three sections of this Sourcebook address the central methodologies employed in the study of political communication: survey and experimentation. Another important element of political communication methodology is content analysis. Three chapters are devoted to this method of
determining patterns in message content, beginning with an examination of political visuals. The study of visuals has long presented a conundrum for political communication researchers. Although recognized as influential, political visuals (especially full motion video with audio) have proven difficult to systematically quantify and reliably measure. The field’s ability to nail down specific empirical insights as to how and when visual effects take place, and the magnitude of visual effects in terms of political influence, has been hindered by this lack of reliable measures with which to assess the visual landscape. In Chapter 12, Grabe and Bucy address this need by summarizing how to empirically study visuals within the context of televised election coverage, following their “image bites” approach to broadcast news analysis (see Grabe & Bucy, 2009; Bucy & Grabe, 2007, 2008).
By providing a foundation for the study of political visuals in any televised setting, they open this essential element of political communication research to more systematic study and analysis.
Content analysis as a methodology dovetails with theory in the context of framing, one of the central theoretical pillars of political communication research (along with agenda setting and priming). Much intellectual effort has been expended in recent years explaining how framing as a mode of analysis interfaces with agenda setting, its variants, and related concepts (see Scheufele, 2000; Schuefele & Tewksbury, 2007). As a research team, Chong and Druckman (2007a, 2007b, 2007c) have been doing as much as anyone to advance the study of framing, both in terms of theory development and in terms of methodology. In Chapter 13, on identifying frames in political news, they explicate some of the fundamental methodological issues associated with the study of frames. The methodological principles addressed in this chapter build on themes introduced in their earlier work. For example, Chong and Druckman stress that a frame can only be identified in relation to a given attitude object that is the focus of the frame, whether this object in a political context is a candidate, event, or issue. Their chapter also addresses many of the fundamental assumptions associated with the study of political media frames and how many framing studies work within a competitive framework through an analysis of the influence of competing frames (e.g., episodic versus thematic). The study of framing remains a conceptually diverse area of research and Chong and Druckman bring methodological clarity to this important line of inquiry.
The final chapter in this section, authored by Benoit, provides an overview of the myriad ways in which political communication researchers set about the task of conducting content analyses. The basics of content analysis (e.g., identification of a sampling frame, development of content categories) are spelled out in some detail. In addition, attention is devoted to the matters of reliability and validity, which are vitally important to conducting quality research using this method. Benoit stresses the need for the proper assessment of intercoder reliability in any content analysis and provides classic arguments for why this should be the case. Far too many content analyses in political
The final chapter in this section, authored by Benoit, provides an overview of the myriad ways in which political communication researchers set about the task of conducting content analyses. The basics of content analysis (e.g., identification of a sampling frame, development of content categories) are spelled out in some detail. In addition, attention is devoted to the matters of reliability and validity, which are vitally important to conducting quality research using this method. Benoit stresses the need for the proper assessment of intercoder reliability in any content analysis and provides classic arguments for why this should be the case. Far too many content analyses in political