To what extent does the writer:
Mean (SD) Exigency Motivate the reader to keep reading?
Demonstrate the scope and context of the problem? Take on a clear and arguable position about the position/controversy? 3.68 (1.08) Audience Awareness and Appropriateness
Address a specific audience in an appropriate, persuasive way?
Aware of an audience or readers?
2.33 (1.08) Logos Make a clear claim and use reasons and evidence to
support claims? 3.51 (.97) Avoid Certitude and Generalizations
Does the writer come across as fair-minded? Does the writer qualify statements and acknowledge uncertainty in the proposal, or come across as narrow-minded?
3.22 (1.24)
Source
Integration Maintain control of the argument while leveraging sources to support it? (1.24) 3.22 Counterargument State, acknowledge, consider, and fairly represent
the opposition?
2.91 (1.26) Rebuttal Respond to opposing arguments in a clear,
reasonable way that demonstrates understanding?
2.93 (1.25) Organization Are paragraphs organized and organized in a
readable, follow-able, consistent way that is free of tangents?
3.56 (1.03) Source Quality Are references selected from reputable sources such
as the library databases and news sources? (versus general web searches and Wiki)
3.97 (1.08) Number of
Sources
Count of sources cited in paper and/or listed on works cited page.
6.51* (2.8) Paper length Number of pages not including works cited 6.78*
(.83) Clarity and Word
Choice
Is the language appropriate and clear in a way that makes it easy to follow the writer’s ideas?
4.00 (.85) Paper Mean Excludes the “number of sources” category 3.32 (.68) *Number of sources and pages were count scores standardized for subsequent analyses
As illustrated in the table above, the students’ papers were generally clear and readable, with few mechanical, grammatical, or word choice errors that obscured their arguments. Scoring within this clarity component was somewhat restricted because all of
77
the papers were from students who gained admission into the university, thus they all had good high school GPAs, with the majority from Texas graduating in the top 10% of their classes high school, and solid test scores. The skill of writing a readable sentence was a prerequisite for acceptance to the university. Additionally, the narrow range of scores within the clarity category may be due to these papers representing students’ second submissions of their arguments. RW 101 emphasizes multi-drafting, so students had already turned in, and received instructor feedback on, first submissions prior to turning in their final papers.
Other components with relatively large variation included the quality of students’ counterarguments and rebuttals. Reflecting on my own experiences with students’ rhetorical writing in my writing center work as well as in my own classes, I remembered that instructors’ emphasis and instruction on counterarguments and rebuttals is often highly variable. Some instructors, for instance, are satisfied with students simply acknowledging that a counterargument exists. Others push students to articulate the oppositions’ viewpoints and provide a meaningful rebuttal, which can often span multiple pages in their proposal arguments.
In addition to accounting for variation in students’ counterargument and rebuttal scores, instructors likely influenced the number and quality of sources. Although the assignment stated a minimum number of references that students should use, some instructors chose to increase the quantity of required sources to as high as 10. Similarly, instructors likely held different standards regarding the quality of sources that students were permitted to use in their papers. In my work with this course and its instructors, I have noticed that some assignments allow students to use sources from general web
78
searches and sites like Wikipedia, whereas other instructors restrict students to use only sources through the library databases and/or .edu or .gov sites. Thus, instructors’
standards as stated in the assignment prompt and through feedback on the students’ initial (first) paper submissions likely contributed to variance on number and quality of sources. In my analyses, presented in the following section, I sought quantitative trends in component scores, as well as potential links between these scores and students’
epistemological and writing beliefs.
Relationships between paper components. My focus during these steps of the
analyses was to establish a qualitative understanding of rhetorical and clarity components of the students’ proposal arguments. However, as I coded and re-coded the papers, I noticed relationships between the paper categories, which prompted me to consider collapsing the 11 categories. Therefore, in order to organize my analyses of the paper categories, I first present a correlation table to demonstrate the way components of the papers may be related:
79
TABLE 16 | CORRELATIONS BETWEEN PAPER COMPONENTS Exig Aud Aware Logos Avoid Cert. Source Integ. Count Arg Re- but Org Source Quality Clarity Exigency and Problem Statement - Audience Awareness .45 ** - Logos .41** .46** - Avoid Certitude .34 ** .34** .61** - Source Integration and Approp- riation .30** .30** .42** .43** - Counter- argument .26 * .34** .41** .43** .33** - Rebuttal .29** .31** .42** .46** .36** .96** - Organization .37** .14 .44** .36** .40** .27** .26* - Source Quality .03 .14 .23 * .26* .15 .05 -.01 .21* - Word Choice and Clarity .21 * .09 .48** .34** .29** .26* .27* .47** -.13 - Number of Sources -.05 -.01 -.08 -.01 .09 -.10 -.10 -.01 .45 ** -.28* *p<.05; **p<.01
Prior to calculating correlations between the paper elements, I first standardized their scores in order to allow for comparison of the “number of sources” component, which was not scored on a 1-5 scale. As indicated in the correlation table, many components of the paper were significantly related to others. For instance, papers that were more exigent also tended to be more audience aware and have a clear statement of
80
thesis (logos). The quality of source integration was also related to the logos component in addition to the writers’ tendencies to avoid certainty and broad generalizations. Also notable is the negative relationship between the quantity of sources students included in their papers and the clarity of the paper. Thus, the more sources that students referenced in their papers, the less likely their paper scored high on word choice and readability.
Data reduction procedures for paper components. In the following step, I
worked to reduce the 11 paper components into several broad categories to facilitate subsequent analyses. To do this, I conducted exploratory factor analysis on the 11 paper components. The categories of source quality, counterargument, and rebuttal were interfering with a clean grouping of factors, thus I removed “source quality” and re-ran the analysis. Then, as the counterargument and rebuttal categories continued to disrupt a clean factor structure, I removed them from the analysis and opted to include them as a category on their own. The resulting scree plot suggested a three-factor solution, so I confined the results to three factors which, taken together, accounted for 72% of the variance in scores across these components (Principal Component Analysis; Varimax rotation). Factor loadings are listed in the table below: