Overview & General Instructions
This study is investigating the quality of online user comments posted to online
discussion forums hosted by one of four political news channels: The Los Angeles Times website, the Daily Kos website, The Young Turks YouTube Channel, and the Associated Press YouTube Channel. Data were collected over a two-month period beginning
February 1, 2011 and concluding on March 31, 2011. A gross total of 16,859 comments were archived over the study period and, based upon a partitioning methodology
designed by the researcher, a subset of 5,881 comments require encoding.
As a member of the two-person coding team, you will be responsible for encoding approximately one-half of this subset. The researcher is supplying you with printed copies of all comments you have been assigned to code. You are also being given full access to the digital archive of news content affiliated with these discussions as your assistance is needed in categorizing the collection of articles and segments.
You will receive compensation from the researcher according to the employment contract provided to you. In fulfillment of that agreement, you are required to:
• Utilize the provided coding scheme to the best of your ability and with as uniform a level of focus as possible. Though the data set is large, you are being provided with ample time to complete the coding assignment. As such, be sure to code only when you are well rested and can give the requisite attention to the project.
• Enter all coding data directly into the online GoogleDrive spreadsheet assigned to you. News source and date information have been preloaded into the spreadsheet and correspond to identifying information appearing on the printed sheets. You may open and use the online spreadsheet whenever you wish and as many times as you wish. Contact the researcher if you have any questions about the online spreadsheet or if you encounter any problems accessing it.
• Refrain from communicating with the other member of the coding team in any way until such time as the researcher has received all coding data. Should you have any questions about the coding scheme or any particular data elements, please contact the researcher exclusively.
• Complete all coding by September 1, 2012. Please contact the researcher immediately if, for any reason, you anticipate you will not be able to complete the assignment by this deadline.
Coding Scheme: Measures of the Quality of Deliberation
Each comment is considered discreet and must be coded separately as its own row within the online spreadsheet. Columns correspond to four measures of deliberative quality (civility, politeness, complexity and responsiveness) and are described in detail below.
Civility Measure
Can you answer yes to one or more of the following questions?
a) The discussant verbalizes a generalized or specific threat to democracy,
democratic institutions or democratic values. Democracy is considered to be an amalgamation of the broader concepts of liberty, equality, and the rule of law.
Example Yes: “The U.S. Supreme Court should be disbanded and replaced by a committee of the NRA.”
Example Yes: “Freedom of speech should be eradicated.”
Example No: “When 2012 rolls around, we need to elect a new president.”
Example No: “I cannot believe in this day and age, with so many people below the poverty line, you can still vote Republican.”
b) The discussant verbalizes a threat to individual rights and freedoms. These threats may take the form of statements intending or promising to do physical, psychological or emotional harm with the goal of retarding the utilization of such rights and freedoms. Moreover, these threats may be directed specifically at another discussant or more generally at a citizen or group of citizens.
Example Yes: “If you’re not gainfully employed, you should have no right to vote.”
Example Yes: “I’m going to rip [username’s] tongue out so he can’t yap any more!”
Example No: “Pro-gay advocates need to take to the streets in protest of the Defense of Marriage Act.”
Example No: “If you don’t like it, get off our duff and do something about it!”
c) The discussant employs one or more antagonistic stereotypes – that is, the discussant distinguishes, labels and makes a judgment about a group of individuals – that are intended to antagonize, discredit or harm. These antagonistic stereotypes may be directed specifically at another discussant or more generally at a citizen or group of citizens.
Example Yes: “Cutting the capital gains tax just means more fancy sports cars and private jets for the uber-rich.”
Example Yes: “… [username] is a typical soccer mom who is convinced her kids are little angels that do nothing wrong.”
Example No: “Senior citizens are worried about the future of Social Security – and they should be!”
Example No: “He’s no different than many new immigrants who will work long hours for little pay and then send most of their money back to their home country.”
If you can answer yes to any of the above questions, Civility = 0.
If you cannot answer yes to any of the above questions, Civility = 1.
If the comment has been removed, flagged as spam, is indecipherable (including if it appears in a language other than English), or you simply cannot tell, Civility = 99.
Politeness Measure
Can you answer yes to one or more of the following questions?
a) The discussant engages in name-calling that is clearly intend to offend. These name-callings may be directed specifically at another discussant or more generally at a citizen or group of citizens.
Example Yes: “Senator King is an obnoxious prick.”
Example Yes: “[Username] is a total fraud and shouldn’t even be allowed to post on this discussion board.”
Example No: “Politicians generally do a lousy job in terms of truly understanding the needs of their constituents.”
Example No: “[Username] is totally wrong in her analysis. She’s looking at the
b) The discussant engages ad hominem attacks – that is, the discussant attacks another’s characteristics or authority without addressing the substance of the argument. These attacks may be directed specifically at another discussant or more generally at a citizen or group of citizens.
Example Yes: “I don’t expect you to understand. After all, you’re poor.”
Example Yes: “[Username] has no business lecturing us about tax policy.”
Example No: “President Bush lied to us. He said in the campaign he wouldn’t raise taxes… and then he did.”
Example No: “I can understand why the rich don’t want a national health care program. It will likely lead to higher taxes and nearly all of them have private insurance already.”
c) The discussant uses vulgar language appearing in the dictionary at
www.noswearing.com. These words may be expressed explicitly or clearly implied in the use of abbreviations, grawlixes or other obvious symbols.
Example Yes: “How the f**k are we supposed to do that?”
Example Yes: “Holy @#$%&! You can’t be serious!”
Example No: “Our taxes are going up again?! Jesus H. Christ!”
Example No: “The House of Representatives makes we want to barf.”
If you can answer yes to any of the above questions, Politeness = 0.
If you cannot answer yes to any of the above questions, Politeness = 1.
If the comment has been removed, flagged as spam, is indecipherable (including if it appears in a language other than English), or you simply cannot tell,
Politeness = 99.
Justification Measure
Can you answer yes to one or more of the following questions?
a) The discussant justifies an argument or perspective by providing supporting evidence in the form of cited facts, embedded documents, web links,
quoted/paraphrased comments from experts, etc.
Example Yes: “Senator Johnson took in $190K of PAC money according to the Center for Government Reporting.”
Example Yes: “How can [username] say that? Doesn’t he remember what he posted two days ago?: www.youtube.com/acqwr122”
Example No: “Everyone knows that Republicans favor small government.”
Example No: “President Obama has a war chest of $59 million already!”
b) The discussant justifies an argument by providing supporting evidence in the form of a description of personal experience or a first-hand account
Example Yes: “When I got laid off from my job my unemployment benefits were only $600 per month. How can I survive on that?”
Example No: “The average person receiving unemployment benefits gets about
$600 per month. That’s not very much at all.”
If you can answer yes to any of the above questions, Justification = 0.
If you cannot answer yes to any of the above questions, Justification = 1.
If the comment has been removed, flagged as spam, is indecipherable (including if it appears in a language other than English), or you simply cannot tell,
Justification = 99.
Complexity Measure
Can you answer yes to one or more of the following questions?
a) The discussant incorporates opposing viewpoints into a given post (irrespective of whether he/she agrees with those viewpoints)
Example Yes: “Who can tell who’s right? Conservative say Obamacare will bankrupt the country. Liberals say health care is a right we owe our citizens.”
Example No: “I agree with liberals in saying that access to affordable health care is a right just the same as free speech or worship.”
b) The discussant expresses a viewpoint in more than one way
Example Yes: “The build-up of the military gets a bad rap. Not only does it stimulate large sectors of the economy but it also leads to the development of new technologies that improve every-day quality of life.”
Example No: “The build-up of the military gets a bad rap. It stimulates large sectors of the economy.”
c) The discussant verbalizes his/her lack of certainty
Example Yes: “I’ve wrestled for years with the abortion question and I still can’t decide if I’m pro-choice or pro-life. My values as a citizen are at odds with my values as a Catholic.”
Example No: “The pro-choice position is the only position that’s consistent with the Bill of Rights.”
d) The discussant clearly asks an honest question (and not a rhetorical one) in an attempt to better understand an issue
Example Yes: “How many young people voted in the last election?”
Example No: “Vote Democrat and expect lower taxes? How dumb do you think I am?
If you can answer yes to any of the above questions, Complexity = 0.
If you cannot answer yes to any of the above questions, Complexity = 1.
If the comment has been removed, flagged as spam, is indecipherable (including if it appears in a language other than English), or you simply cannot tell,
Complexity = 99.
Coding Scheme: Categorization of Corresponding Article/Segment
Each comment corresponds to an original printed article or broadcasted audiovisual segment. Two additional columns are included on the online spreadsheet and coders are asked to make appropriate categorizations of each corresponding article/segment.
Geographic Focus
Does the segment or article focus the majority of its attention on (1) political issues, discussions or considerations within the United States; or (2) political issues, discussions or considerations outside of the United States?
If (1), Geographic Focus = Domestic.
If (2), Geographic Focus = International.
If you cannot tell, Geographic Focus = Cannot Tell.
Example Domestic: An article describing the Democratic Party’s opposition to the Defense of Marriage Act.
Example International: A segment describing widespread protests of the G7 summit in Paris.
Topical Focus
Does the segment or article focus the majority of its attention on (1) lawmaking, policymaking or public policy; or (2) on elections, balance of power discussions or political strategy?
If (1), Topical Focus = Governing.
If (2), Topical Focus = Campaigning.
If you cannot tell, Topical Focus = Cannot Tell.
Example Governing: A segment describing a United Nations debate to dispatch humanitarian aid to Syrian refugees.
Example Campaigning: A article detailing the current field of Republican Presidential candidates and their odds of winning the Iowa caucuses.
APPENDIX G