Study Design and Methods

The data for this study were collected from three pieces of assigned coursework over the span of one semester in an undergraduate environmental economics course. All three of these assessments are 750-word blog posts set on an environmental issue of the students’ choosing. The required topics for the three assignments are summarized as follows:

• Blog Post #1: Discuss the physical origins of the environmental issue you have chosen, including a discussion of the environmental impacts at the local, regional, or global level.

• Blog Post #2: Discuss the economic origins of the environmental issue you have chosen, focusing on a fundamental discussion of price and quantities. For instance, is this good a non-market good, in which a well-defined price doesn’t exist? Or, is it a public-good issue, in which free riding incentivizes overconsumption? Include at least one graph to help in your discussion.

• Blog Post #3: Discuss the role of policy in helping to eliminate/mitigate the environmental issue that you chose. Here, I want you to choose one policy discussed in class and describe how this policy could eliminate your environmental issue. Make sure the policy you chose is a feasible option for addressing your chosen issue. I’d like you to discuss in-depth the policy you chose, how it works, and why you chose it for your particular issue. Include at least one graph to help in your discussion.

These blog posts were graded according to the detailed rubric and grading criteria presented in Appendix 3B and written feedback was provided in the form of comments under the headings ‘Key Strengths’ and ‘Key Areas for Improvement’. An open-response style of assessment is used instead of fixed response multiple choice questions to capture deeper and more multifaceted measures of learning. An advanced undergraduate trained in assessing writing was the sole assessor to eliminate issues of inter-marker reliability (Meadows and Billington, 2005; Newstead, 2004; Bloxham, 2009).

Written coursework in the form of blog posts were chosen to assess self-evaluation as it is thought that students would be better able to judge the quality of their work in a less time constrained environment. Also, to the best of the authors’ knowledge, only one previous study involving economics students has used written coursework to assess accuracy of self-evaluation (Ward, et al., 2002) and no previous studies in economics have examined these changes within course. The self-evaluation data will be collected by asking the students the following question

after they have submitted the required assignment, in which the student provides an overall percentage grade:

“What do you consider would be an appropriate grade for the blog post you have submitted?” Ward, et al. (2002) develop a number of key issues to be addressed when collecting the data for the purpose of estimating self-evaluation accuracy and which will be addressed in this study. First, sample selection issues arise when submission of the self-evaluation scores is voluntary. Here, it can be thought that the most motivated students will participate in the self- evaluation questions, and that high motivation might be correlated with other factors also affecting self-evaluation accuracy. In this study, the self-evaluation exercise is mandatory and if the student fails to submit the self-evaluation exercise, their work will remain ungraded and they will receive a zero for the assignment.

Another issue is the extent to which the self-evaluation scores accurately reflect the students’ true beliefs about the quality of their work. Research from the pedagogy literature suggests that students find understanding and applying assessment criteria cognitively demanding, so if the students perceive limited marginal gain from the self-assessment, the cognitive burden may be too high to complete the question. (Nicol and MacFarlane-Dick, 2006) To incentivize accurate completion of the self-evaluation questions, this study will follow Guest and Riegler (2017) and provide a ‘grade incentive’, in which the student will be awarded additional points on the assignment if the self-evaluation estimate was within a certain threshold of the grade assigned. A final issue with the accuracy of self-evaluation scores are concerns on behalf of the student of how it might influence the person grading their work. Lew and Schmidt (2007) find that students who believe they have submitted a low-quality piece of coursework may not wish to communicate these beliefs to the grader and in response may overrate their work in the hope of

positively influencing the grader. Conversely, for those students who believe they have produced a high-quality piece of work, concerns of ‘showing-off’ might push them to deliberately under report their self-evaluation score. This problem is most likely to occur in situations when the self- evaluation score is submitted with the assessment. To overcome this problem, the students will submit completed self-evaluation questions separately from the assignment and will be reminded that the grader will not see their self-reported scores.