I use an experimental design to assess the influence of heated, partisan debate on policy attitudes.2 The experiment was administered online to 367 UNC-Chapel Hill undergrad-
uate students using the Qualtrics platform during the fall of 2011. Subjects, who were required to participate in the university subject pool or complete an alternative assignment,
!"#$%&'( )%*'+#&,( -,'.,,#(/,01( +#2(-%00( 3"0%&4( 56'72,( 8#2%9%27+0:*( 82,"0";4(
Figure 2.2: Direct and Indirect Influence of Conflict on Policy Attitudes
were first asked to complete a questionnaire that gathered demographic information as well as information on party identification, political ideology, political knowledge, and interest in politics.3 Participants were then randomly assigned to one of six treatment groups and
asked to read a short vignette (approximately 250 words long) about an education policy. Education was chosen because it was not a topic of national debate while the experiment was in the field, thereby minimizing the influence of real world policy debate on the ex- perimental findings. All of the vignettes were modeled after realNew York Timesarticles about the Obama health care plan (where education was substituted for health care), using as many verbatim statements from the articles as possible. While the real articles attribute
3The use of a student subject pool poses some limitations for the project. First, levels of knowledge about
and interest in politics are higher and less variable among a group of students enrolled in a college-level political science class than they would be among the general public. Nearly 94% of the respondents stated they were somewhat or very interested in politics, leaving just six percent of respondents with low levels of interest. On some measures of political knowledge contained in the pre-treatment questionnaire, more than 91% of participants responded correctly. This creates a hard test of the primary hypothesis, as conflict is expected to have the largest direct impact on the policy attitudes of those with low levels of political knowl- edge. Secondly, students are also fairly uniform in their support for education funding—nearly 71% stated they generally preferred more federal funding for K-12 education, and just under 7% stated that they would like to see the government spend less (with the balance saying the believe federal spending on education is “about right”). This also creates a hard test as those with strong, preformed attitudes are thought to resist the influence of exposure to conflict. Nevertheless, the uniformity of knowledge, interest, and preferences among students makes it impossible to test the ways in which these factors interact with exposure to conflict to influence policy attitudes. For that reason, I do not test hypothesis two here.
quotes to named party leaders (like Nancy Pelosi), no actual members of Congress were referenced by name in the treatments. (The treatments reference speakers with titles, such as “Committee Chairman,” or use made up names.) This was done to prevent attitudes toward specific political figures from influencing participant responses. The vignettes were also designed by a graphic artist to look like articles downloaded from theNew York Times.4 All of the treatments open with a lead paragraph that describes the education bill as one “designed to reform K through 12 education by providing vastly more resources for schools and teachers.” The six treatments—summarized in Table 2.1—differ in that they vary both the tone of debate (heated or civil) as well as the substance of the article. The treatments that highlight partisan conflict use the headline, “Partisan battle on education heats up.” The articles themselves describe the bill as “hotly contested,” note that “the debate has deteriorated into a partisan brawl,” and state that “Democrats will have to close ranks and vote as a bloc to pass the bill without Republican support.” All of the treatments that employ a civil tone describe the bill as a bipartisan one, note that the bill is gaining momentum in the Senate, and describe lawmakers as working to reach a compromise.
Table 2.1: Description of Experimental Treatments
Spending Efficacy Tactics
Heated Tone
Tone: “Partisan brawl,” “hotly contested bill”
Tone: “Partisan brawl,” “hotly contested bill”
Tone: “Partisan brawl,” “hotly contested bill” Substance: Disagreement
over bill’s effect on deficit
Substance: Disagreement over efficacy of provi- sions Substance: Bundle of amendments attached to bill Civil Tone
Tone: “bipartisan bill,” “working toward a com- promise”
Tone: “bipartisan bill,” “working toward a com- promise”
Tone: “bipartisan bill,” “working toward a com- promise”
Substance: Disagreement over bill’s effect on deficit
Substance: Disagreement over efficacy of provi- sions
Substance: Bundle of amendments attached to bill
The three different substantive treatments each focus on a distinct facet of real-world policy debate, as reported by the news media. Those facets are the parliamentary tactics being employed by the respective parties, the program’s cost, and the efficacy of the plan’s provisions. The tactics and procedures treatments provide minimal substantive information about the bill being considered (they note the party affiliation of the bill and provide the de- scription of the bill that is common to all of the articles regarding “vastly more spending”). The article is focused on the attachment of a bundle of amendments to the bill, which are described by the Democrats as “delay tactics” in the heated conflict version of the article, and described as a method of insuring bipartisan support in the civil debate version of the article. The tactics treatments are, therefore, designed to show the impact of partisan debate on policy attitudes in an instance where very little substantive information is available.
The spending and efficacy treatments provide a more moderate level of substantive information about the bill. These treatments give arguments for and against the passage of the legislation, describing the particular elements of the bill about which the parties disagree. In the two “spending” treatments, Republicans reject Democrat’s claims that the proposal will decrease the deficit. In the two “efficacy” treatments, Republicans assert that some of the bill’s provisions might do more harm than good because they are “risky” and “untested,” while Democrats state the plan will increase test scores. All four of these treatments, therefore, describe the substance of the policy debate. The civil and heated versions of the treatments manipulate the tone of that debate.
After reading the article to which they were assigned, all participants were asked to respond to several questions (the order of which was randomized). To assess perceptions of the policymaking process, respondents were asked: “Do you think the education reform process shows more that our policymaking process is working as intended, or more that our policymaking process is broken?” Respondents were also asked whether they support or oppose the policy and where they would place the policy on a seven-point ideological
scale.