Zurich Open Repository and Archive University of Zurich Main Library Strickhofstrasse 39 CH-8057 Zurich www.zora.uzh.ch Year: 2016
Do we measure overconfidence? A closer look at the interval production task Langnickel, Ferdinand ; Zeisberger, Stefan
Abstract: The most common test for overconfidence in the form of miscalibration—the Interval Pro-duction task (IP)—is based on the assumption that people internalize requested confidence levels. We demonstrate experimentally that decision makers’ perceived confidence is, however, unaffected by varia-tions in the requested confidence level. In addition, we find large heterogeneity in perceived confidence that the traditional IP measure fails to account for. We show that the alternative measure based on decision makers’ perceived confidence by contrast yields coherent, moderate overconfidence levels. Our evidence suggests that the consistency of the two measures is limited and that they are related to different individual characteristics.
DOI: https://doi.org/10.1016/j.jebo.2016.04.019
Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-124198
Journal Article Accepted Version Originally published at:
Langnickel, Ferdinand; Zeisberger, Stefan (2016). Do we measure overconfidence? A closer look at the interval production task. Journal of Economic Behavior Organization, 128:121-133.
Accepted Manuscript
Title: Do We Measure Overconfidence? A Closer Look at the Interval Production Task
Author: Ferdinand Langnickel Stefan Zeisberger
PII: S0167-2681(16)30076-2
DOI: http://dx.doi.org/doi:10.1016/j.jebo.2016.04.019
Reference: JEBO 3794
To appear in: Journal of Economic Behavior & Organization
Received date: 7-10-2015
Revised date: 23-4-2016
Accepted date: 28-4-2016
Please cite this article as: Ferdinand Langnickel, Stefan Zeisberger, Do We Measure Overconfidence? A Closer Look at the Interval Production Task, <![CDATA[Journal of Economic Behavior and Organization]]> (2016), http://dx.doi.org/10.1016/j.jebo.2016.04.019
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Accepted Manuscript
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60❉♦ ❲❡ ▼❡❛s✉r❡ ❖✈❡r❝♦♥✜❞❡♥❝❡❄ ❆ ❈❧♦s❡r ▲♦♦❦ ❛t t❤❡
■♥t❡r✈❛❧ Pr♦❞✉❝t✐♦♥ ❚❛s❦
∗
❋❡r❞✐♥❛♥❞ ▲❛♥❣♥✐❝❦❡❧
❛❛♥❞ ❙t❡❢❛♥ ❩❡✐s❜❡r❣❡r
†❜ ❛❉❡♣❛rt♠❡♥t ♦❢ ❇❛♥❦✐♥❣ ❛♥❞ ❋✐♥❛♥❝❡✱ ❯♥✐✈❡rs✐t② ♦❢ ❩✉r✐❝❤✱ P❧❛tt❡♥str❛ss❡ ✸✷✱ ✽✵✸✷ ❩✉r✐❝❤✱ ❙✇✐t③❡r❧❛♥❞ ❜❈♦❧❧❡❣❡ ♦❢ ❇✉s✐♥❡ss✱ ❙t♦♥② ❇r♦♦❦ ❯♥✐✈❡rs✐t②✱ ❙t♦♥② ❇r♦♦❦✱ ◆❨ ✶✶✼✾✹❆♣r✐❧ ✷✷✱ ✷✵✶✻
❆❜str❛❝t ❚❤❡ ♠♦st ❝♦♠♠♦♥ t❡st ❢♦r ♦✈❡r❝♦♥✜❞❡♥❝❡ ✐♥ t❤❡ ❢♦r♠ ♦❢ ♠✐s❝❛❧✐❜r❛t✐♦♥✖t❤❡ ■♥✲ t❡r✈❛❧ Pr♦❞✉❝t✐♦♥ t❛s❦ ✭■P✮✖✐s ❜❛s❡❞ ♦♥ t❤❡ ❛ss✉♠♣t✐♦♥ t❤❛t ♣❡♦♣❧❡ ✐♥t❡r♥❛❧✐③❡ r❡✲ q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s✳ ❲❡ ❞❡♠♦♥str❛t❡ ❡①♣❡r✐♠❡♥t❛❧❧② t❤❛t ❞❡❝✐s✐♦♥ ♠❛❦❡rs✬ ♣❡r✲ ❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡ ✐s✱ ❤♦✇❡✈❡r✱ ✉♥❛✛❡❝t❡❞ ❜② ✈❛r✐❛t✐♦♥s ✐♥ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ■♥ ❛❞❞✐t✐♦♥✱ ✇❡ ✜♥❞ ❧❛r❣❡ ❤❡t❡r♦❣❡♥❡✐t② ✐♥ ♣❡r❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡ t❤❛t t❤❡ tr❛❞✐✲ t✐♦♥❛❧ ■P ♠❡❛s✉r❡ ❢❛✐❧s t♦ ❛❝❝♦✉♥t ❢♦r✳ ❲❡ s❤♦✇ t❤❛t t❤❡ ❛❧t❡r♥❛t✐✈❡ ♠❡❛s✉r❡ ❜❛s❡❞ ♦♥ ❞❡❝✐s✐♦♥ ♠❛❦❡rs✬ ♣❡r❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡ ❜② ❝♦♥tr❛st ②✐❡❧❞s ❝♦❤❡r❡♥t✱ ♠♦❞❡r❛t❡ ♦✈❡r✲ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s✳ ❖✉r ❡✈✐❞❡♥❝❡ s✉❣❣❡sts t❤❛t t❤❡ ❝♦♥s✐st❡♥❝② ♦❢ t❤❡ t✇♦ ♠❡❛s✉r❡s ✐s ❧✐♠✐t❡❞ ❛♥❞ t❤❛t t❤❡② ❛r❡ r❡❧❛t❡❞ t♦ ❞✐✛❡r❡♥t ✐♥❞✐✈✐❞✉❛❧ ❝❤❛r❛❝t❡r✐st✐❝s✳ ❑❡②✇♦r❞s✿ ♦✈❡r❝♦♥✜❞❡♥❝❡✱ ♠✐s❝❛❧✐❜r❛t✐♦♥✱ ♠❡t❤♦❞♦❧♦❣②✱ ❡①♣❡r✐♠❡♥t❛❧ ❡❝♦♥♦♠✐❝s✱ ❡①✲ ♣❡r✐♠❡♥t❛❧ ✜♥❛♥❝❡ ❏❊▲ ❝❧❛ss✐✜❝❛t✐♦♥✿ ❉✽✸✱ ●✵✷✱ ●✶✷ ∗❲❡ t❤❛♥❦ ❉❛✈✐❞ ❇✉❞❡s❝✉✱ ❚❤♦rst❡♥ ❍❡♥s✱ ❚❤♦♠❛s ▲❛♥❣❡r✱ ❆❧❡①❛♥❞r❛ ◆✐❡ss❡♥✲❘ü♥③✐✱ ❯❧r✐❝❤ ❙❝❤♠✐❞t✱ ❑❛r❧ ❍❛❧✈♦r ❚❡✐❣❡♥✱ ▲❡♦♥❛r❞ ❲❛❧❦✱ ▼❛rt✐♥ ❲❡❜❡r✱ ❛♥❞ ♣❛rt✐❝✐♣❛♥ts ♦❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s❡♠✐♥❛rs ❛♥❞ ❝♦♥❢❡r✲ ❡♥❝❡s✿ ❊①♣❡r✐♠❡♥t❛❧ ❋✐♥❛♥❝❡ ✷✵✶✺ ✐♥ ◆✐❥♠❡❣❡♥✱ ❚❤❡ ◆❡t❤❡r❧❛♥❞s✱ ❙P❯❉▼ ✷✵✶✺ ✐♥ ❇✉❞❛♣❡st✱ ❍✉♥❣❛r②✱ ❋✐♥❛♥❝✐❛❧ ❊❝♦♥♦♠✐❝s ❙❡♠✐♥❛r ❛t ❯♥✐✈❡rs✐t② ♦❢ ❩✉r✐❝❤✱ ❙✇✐t③❡r❧❛♥❞✳ ❚❤✐s r❡s❡❛r❝❤ ✇❛s ❣r❛t❡❢✉❧❧② s✉♣♣♦rt❡❞ ❜② t❤❡ ❙✇✐ss ◆❛t✐♦♥❛❧ ❙❝✐❡♥❝❡ ❋♦✉♥❞❛t✐♦♥ ✭✶✵✵✵✶✽✲✶✹✾✾✸✹✮✳ †❈♦rr❡s♣♦♥❞✐♥❣ ❛✉t❤♦r❀ ❡♠❛✐❧✿ st❡❢❛♥✳③❡✐s❜❡r❣❡r❅st♦♥②❜r♦♦❦✳❡❞✉ ✶Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60✶ ■♥tr♦❞✉❝t✐♦♥
◆✉♠❡r♦✉s st✉❞✐❡s ❤❛✈❡ ❞❡♠♦♥str❛t❡❞ t❤❛t ♦✈❡r❝♦♥✜❞❡♥❝❡ ❝❛♥ ❧❡❛❞ t♦ s✉❜♦♣t✐♠❛❧ ❞❡✲ ❝✐s✐♦♥s ✐♥ ❛ ✈❛r✐❡t② ♦❢ ❞♦♠❛✐♥s ✭❈❛♠❡r❡r ❛♥❞ ▲♦✈❛❧❧♦✱ ✶✾✾✾❀ ❏♦❤♥s♦♥ ❛♥❞ ❚✐❡r♥❡②✱ ✷✵✶✶❀ ▼❛❧♠❡♥❞✐❡r ❛♥❞ ❚❛t❡✱ ✷✵✵✺❀ ❖❞❡❛♥✱ ✶✾✾✽✮✳ ❚❤❡ ♠♦st r♦❜✉st ❞♦❝✉♠❡♥t❛t✐♦♥ ♦❢ ♦✈❡r❝♦♥✲ ✜❞❡♥❝❡ ✐s t❤❛t ♣❡♦♣❧❡ ♦✈❡r❡st✐♠❛t❡ t❤❡ ♣r❡❝✐s✐♦♥ ♦❢ t❤❡✐r ♦✇♥ ❥✉❞❣♠❡♥t ✭❢♦r ❛ r❡✈✐❡✇✱ s❡❡ ▲✐❝❤t❡♥st❡✐♥ ❡t ❛❧✳✱ ✶✾✽✷❀ ▼♦♦r❡ ❡t ❛❧✳✱ ✷✵✶✹✮✳ ❚❤✐s ❢♦r♠ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡✱ ❝❛❧❧❡❞ ♠✐s❝❛❧✐✲ ❜r❛t✐♦♥✱ ♦✈❡r♣r❡❝✐s✐♦♥✱ ♦r ❥✉❞❣♠❡♥t❛❧ ♦✈❡r❝♦♥✜❞❡♥❝❡✱ ✐♥❞✉❝❡s ♣❡♦♣❧❡ t♦ r❡❧② t♦♦ ♠✉❝❤ ♦♥ t❤❡✐r ♦✇♥ ✭❜✐❛s❡❞✮ ❥✉❞❣♠❡♥t ❛♥❞ t❤✉s ✐s ❛♥ ✐♥t❡❣r❛❧ ♣❛rt ♦❢ ❜✐❛s❡❞ ❞❡❝✐s✐♦♥ ♠❛❦✐♥❣ ✐♥ ❣❡♥✲ ❡r❛❧ ✭❇❛③❡r♠❛♥ ❛♥❞ ▼♦♦r❡✱ ✷✵✶✸✮✳✶ ❚❤❡ ♠❛✐♥❧② ✉s❡❞ t❡st ❢♦r ✐t ✐s t❤❡ ■♥t❡r✈❛❧ Pr♦❞✉❝t✐♦♥ t❛s❦ ✭■P✮✳✷ ■♥ t❤❡ ■P✱ ❞❡❝✐s✐♦♥ ♠❛❦❡rs ❛r❡ ❛s❦❡❞ t♦ ♣r♦✈✐❞❡ ❧♦✇❡r ❛♥❞ ✉♣♣❡r ❜♦✉♥❞ ❡st✐♠❛t❡s ✭✐♥t❡r✈❛❧s✮ ❢♦r ❛ s❡t ♦❢ q✉❡st✐♦♥s ❧✐❦❡ ✏❍♦✇ ❧♦♥❣ ✐s t❤❡ ◆✐❧❡ r✐✈❡r❄✑✳ ❙✉❜❥❡❝ts ❛r❡ ✐♥str✉❝t❡❞ t♦ st❛t❡ ✐♥t❡r✈❛❧s s✉❝❤ t❤❛t t❤❡✐r ♦✇♥ ❝♦♥✜❞❡♥❝❡✱ t❤❛t t❤❡ tr✉❡✱ ✉♥❦♥♦✇♥ ✈❛❧✉❡ ❢❛❧❧s ❜❡t✇❡❡♥ t❤❡s❡ st❛t❡❞ ❜♦✉♥❞s✱ ❡q✉❛❧s ❛ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ t❤❛t ✐s r❡q✉❡st❡❞ ❜② t❤❡ ❡①♣❡r✐♠❡♥t❡r✱ ❢♦r ❡①❛♠♣❧❡ ✾✵✪✳ ❖♥ ❛✈❡r❛❣❡✱ t❤❡ r❛t✐♦ ♦❢ tr✉❡ ✈❛❧✉❡s t❤❛t ❢❛❧❧ ✐♥t♦ ❞❡❝✐s✐♦♥ ♠❛❦❡rs✬ ✐♥t❡r✈❛❧ ❡st✐♠❛t❡s✱ t❤❡ ✏❤✐t r❛t❡✑✱ s❤♦✉❧❞ ❝♦rr❡s♣♦♥❞ t♦ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ✭✐♥ t❤✐s ❝❛s❡ ✾✵✪✮✳ ❍♦✇❡✈❡r✱ ❝♦♠♠♦♥❧② ♣❡♦♣❧❡ ❛r❡ ❢♦✉♥❞ t♦ ❤❛✈❡ ♠✉❝❤ ❧♦✇❡r ❤✐t r❛t❡s✱ s♦ t❤❛t t❤❡② ❛r❡ ❝❧❛ss✐✜❡❞ ❛s ♦✈❡r❝♦♥✜❞❡♥t ✭❆❧♣❡rt ❛♥❞ ❘❛✐✛❛✱ ✶✾✽✷❀ ❘✉ss♦ ❛♥❞ ❙❝❤♦❡♠❛❦❡r✱ ✶✾✾✷✮✳ ❚❤✐s ♣❛♣❡r ❛✐♠s t♦ ❝r✐t✐❝❛❧❧② ❛ss❡ss t❤❡ s✉✐t❛❜✐❧✐t② ♦❢ t❤❡ ■P t♦ ♠❡❛s✉r❡ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ ❘❡❝❡♥t st✉❞✐❡s ❤❛✈❡ ❛❧r❡❛❞② s✉❣❣❡st❡❞ t❤❛t t❤❡ ■P ♠❡❛s✉r❡ ♠❛② ♥♦t ❢✉♥❝t✐♦♥ ❛s ♣r❡s✉♠❡❞✳ ■♥ ♣❛rt✐❝✉❧❛r✱ ✐t ❤❛s ❜❡❡♥ s❤♦✇♥ t❤❛t ❣r♦✉♣s ✇✐t❤ ❞✐✛❡r❡♥t r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s ❛❝❤✐❡✈❡ t❤❡ s❛♠❡ ❛✈❡r❛❣❡ ❤✐t r❛t❡ ❜❡❝❛✉s❡ t❤❡② ❞♦ ♥♦t ❛❞❥✉st t❤❡ ✇✐❞t❤ ♦❢ t❤❡✐r ✐♥t❡r✈❛❧ ❡st✐♠❛t❡s ✭❚❡✐❣❡♥ ❛♥❞ ❏ør❣❡♥s❡♥✱ ✷✵✵✺✮✳✸ ■♥ ❛❞❞✐t✐♦♥✱ ✇❤❡♥ ❡st✐♠❛t✐♥❣ t❤❡✐r ♦✇♥ ♥✉♠❜❡r ♦❢ ❤✐ts ✭✏❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts✑✮ ♣❡♦♣❧❡ t❡♥❞ t♦ ❜❡ ♦♥❧② ❝♦♥s❡r✈❛t✐✈❡❧② ❝♦♥✜❞❡♥t ✐♥ t❤❡✐r ♦✇♥ ✐♥t❡r✈❛❧s ❝♦♠♣❛r❡❞ t♦ t❤❡ t②♣✐❝❛❧❧② ❤✐❣❤ ❧❡✈❡❧ ♦❢ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ✭❈❡s❛r✐♥✐ ❡t ❛❧✳✱ ✷✵✵✻❀ ❲✐♥♠❛♥ ❡t ❛❧✳✱ ✷✵✵✹✮✳ ❲❤❛t r❡♠❛✐♥s ✉♥s♦❧✈❡❞✱ ❤♦✇❡✈❡r✱ ✐s t❤❡ ❛❝t✉❛❧ ❡✛❡❝t ♦❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ♦♥ t❤❡ ♣❡r❝❡✐✈❡❞ ❧❡✈❡❧ ♦❢ ❝♦♥✜❞❡♥❝❡✳ ❲✐❧❧ ♣❡♦♣❧❡✱ ❢♦r ❡①❛♠♣❧❡✱ st✐❧❧ ♣r♦✈✐❞❡ ❝♦♥s❡r✈❛t✐✈❡ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ✐❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ✐s ❧♦✇❄ ❆♥ ❡①♣❡r✐♠❡♥t❛❧ ❡❝♦♥♦♠✐st ✇❤♦ st✉❞✐❡s t❤❡ r❡❧❛t✐♦♥ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛♥❞ ❡❝♦♥♦♠✐❝ ❜❡❤❛✈✐♦r ♥♦♥❡t❤❡❧❡ss ❝❛♥ ❜❡ ❛❣♥♦st✐❝ ❛❜♦✉t t❤❡s❡ ♣♦✐♥ts ♦❢ ❝r✐t✐q✉❡ ❛s t❤❡② ❛❞❞r❡ss t❤❡ ✶■♥ t❤❡ ❢♦❧❧♦✇✐♥❣✱ ✇❡ ✇✐❧❧ ✉s❡ ♦♥❧② t❤❡ t❡r♠ ♦✈❡r❝♦♥✜❞❡♥❝❡ t♦ r❡❢❡r t♦ ❥✉❞❣♠❡♥t❛❧ ♠✐s❝❛❧✐❜r❛t✐♦♥✱ ✉♥❧❡ss st❛t❡❞ ♦t❤❡r✇✐s❡✳ ✷❙❡❡✱ ❡✳❣✳ ❆❧♣❡rt ❛♥❞ ❘❛✐✛❛ ✭✶✾✽✷✮❀ ❇❡♥✲❉❛✈✐❞ ❡t ❛❧✳ ✭✷✵✶✸✮❀ ❇✐❛✐s ❡t ❛❧✳ ✭✷✵✵✺✮❀ ❘✉ss♦ ❛♥❞ ❙❝❤♦❡♠❛❦❡r ✭✶✾✾✷✮✳ ✸❚❤❡s❡ ✜♥❞✐♥❣s ❛r❡ ❜❛s❡❞ ♦♥ ❜❡t✇❡❡♥✲s✉❜❥❡❝t ✈❛r✐❛t✐♦♥ ♦❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ■♥ r❡s♣♦♥s❡ t♦ ✇✐t❤✐♥✲s✉❜❥❡❝t ✈❛r✐❛t✐♦♥ ♦❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❞❡❝✐s✐♦♥ ♠❛❦❡rs ❤❛✈❡ ❜❡❡♥ s❤♦✇♥ t♦ ❛❞❥✉st t❤❡✐r ✐♥t❡r✈❛❧ ❡st✐♠❛t❡s ✭s❡❡✱ ❡✳❣✳ ❆❧♣❡rt ❛♥❞ ❘❛✐✛❛✱ ✶✾✽✷❀ ❇✉❞❡s❝✉ ❛♥❞ ❉✉✱ ✷✵✵✼✮✳ ✷Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ❛❣❣r❡❣❛t❡✱ ♥♦t t❤❡ ✐♥❞✐✈✐❞✉❛❧ ❧❡✈❡❧✳ ❚❤❡r❡❢♦r❡✱ ✐t ✐s ♥♦t s✉r♣r✐s✐♥❣ t❤❛t t❤❡ ■P ❤❛s ❜❡❡♥ ✉s❡❞ ✐♥ ♥✉♠❡r♦✉s ❡①♣❡r✐♠❡♥t❛❧ st✉❞✐❡s t♦ ❡❧✐❝✐t ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛t t❤❡ ✐♥❞✐✈✐❞✉❛❧ ❧❡✈❡❧ ❛♥❞ ❧✐♥❦ ✐t t♦ ❡❝♦♥♦♠✐❝ ❜❡❤❛✈✐♦r ✭❢♦r ♠♦st r❡❝❡♥t st✉❞✐❡s✱ s❡❡ ❆❝❦❡rt ❡t ❛❧✳✱ ✷✵✶✺❀ ❇❡♥✲❉❛✈✐❞ ❡t ❛❧✳✱ ✷✵✶✸❀ ❋❡❧❧♥❡r✲❘ö❤❧✐♥❣ ❛♥❞ ❑rü❣❡❧✱ ✷✵✶✹❀ ❍❡r③ ❡t ❛❧✳✱ ✷✵✶✹✮✱ ❜✉t t❤❡ r❡s✉❧ts ❛r❡ st✐❧❧ s✉r♣r✐s✐♥❣❧② ✐♥❝♦♥❝❧✉s✐✈❡ ♦✈❡r❛❧❧✳ ❆❣❛✐♥st t❤✐s ❜❛❝❦❣r♦✉♥❞✱ ✇❡ ❛❞❞r❡ss t❤r❡❡ ♦♣❡♥ ✐ss✉❡s t♦ ❤❡❧♣ ✉♥❞❡rst❛♥❞✐♥❣ t❤❡ ♠❡❛✲ s✉r❡♠❡♥t ♦❢ ❛❣❣r❡❣❛t❡ ❛♥❞ ✐♥❞✐✈✐❞✉❛❧ ♦✈❡r❝♦♥✜❞❡♥❝❡ ✐♥ t❤❡ ■P ♣❛r❛❞✐❣♠✳ ❋✐rst✱ ♦♥❡ ♠✐❣❤t ✇♦♥❞❡r ✇❤② ❞❡❝✐s✐♦♥ ♠❛❦❡rs ❛r❡ ✉♥❛❜❧❡ t♦ ❛❞❛♣t t❤❡✐r ✐♥t❡r✈❛❧s ✇✐❞t❤s ✐♥ t❤❡ ■P✳ ❚♦ ♣r♦✲ ✈✐❞❡ ❛♥ ❛♥s✇❡r ✇❡ t❛❦❡ ❛ st❡♣ ❜❛❝❦ ❛♥❞ t❡st ✇❤❡t❤❡r t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❤❛s ❛♥ ❡✛❡❝t ♦♥ ♣❡♦♣❧❡✬s ♣❡r❝❡✐✈❡❞ ❧❡✈❡❧ ♦❢ ❝♦♥✜❞❡♥❝❡ ✐♥ t❤❡ ✜rst ♣❧❛❝❡✳ ❙❡❝♦♥❞✱ ❣✐✈❡♥ t❤❡ ♦❢t❡♥ ✉♥s✉❝❝❡ss❢✉❧ ❡①♣❡r✐♠❡♥t❛❧ ❛tt❡♠♣ts t♦ ❧✐♥❦ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛♥❞ ❡❝♦♥♦♠✐❝ ❜❡❤❛✈✐♦r ❛t t❤❡ ✐♥✲ ❞✐✈✐❞✉❛❧ ❧❡✈❡❧✱ ❛♥♦t❤❡r ✐♠♣♦rt❛♥t ♦♣❡♥ q✉❡st✐♦♥ ✐s ✇❤❡t❤❡r t❤❡ ✉s❡ ♦❢ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❝❤❛♥❣❡s t❤❡ r❡❧❛t✐✈❡ r❛♥❦✐♥❣ ✐♥ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❝♦♠♣❛r❡❞ t♦ t❤❡ ❝❧❛ss✐❝ ■P ♠❡❛s✉r❡✳ ▲❛st❧②✱ t❤❡ t✇♦ ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡s ♠❛② ❜❡ ❡①♣❧❛✐♥❡❞ ❜② ✐♥❞✐✈✐❞✉❛❧ ✏❜❛❝❦❣r♦✉♥❞✑ ❝❤❛r❛❝t❡r✲ ✐st✐❝s s✉❝❤ ❛s ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s ♦r t❤❡ ❛✈❡rs✐♦♥ ♦❢ ✇✐❞❡ ✐♥t❡r✈❛❧s ✇❤✐❝❤ ♣♦ss✐❜❧② ❡①♣❧❛✐♥s s♦♠❡ ❡①♣❡r✐♠❡♥t❛❧ r❡s✉❧ts✳ ❚❤❡ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡ r❡sts ♦♥ t❤❡ ❛ss✉♠♣t✐♦♥ t❤❛t ♣❡♦♣❧❡ ❛❞♦♣t t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❛♥❞ ❛❞❥✉st t❤❡✐r ✐♥t❡r✈❛❧ ❡st✐♠❛t❡s ❛❝❝♦r❞✐♥❣❧②✳ ❯s✐♥❣ t❤❡ ❝❧❛ss✐❝❛❧ ■P ♣❛r❛❞✐❣♠ ✇❡ r✉♥ ❛♥ ❡①♣❡r✐♠❡♥t ✐♥ ✇❤✐❝❤ ✇❡ ❡❧✐❝✐t ♣❡r❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s ✐♥ t❤❡ ❢♦r♠ ♦❢ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts✱ ✐✳❡✳ ❡①✲♣♦st ❡st✐♠❛t❡s ♦❢ t❤❡ ♥✉♠❜❡r ♦❢ ❤✐ts ✐♥ t❤❡ ■P✱ ❛♥❞ ✇❡ ✈❛r② t❤❡ ❞❡❣r❡❡ ♦❢ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡✳ ■♥ t✇♦ ✐♥❞❡♣❡♥❞❡♥t s✉r✈❡②s ✐♥ ❙✇✐t③❡r❧❛♥❞ ❛♥❞ t❤❡ ❯✳❙✳✱ ✇❡ ❡♠♣❧♦② ❛ ❜❡t✇❡❡♥✲s✉❜❥❡❝t ❞❡s✐❣♥ ✐♥ ✇❤✐❝❤ ♣❛rt✐❝✐♣❛♥ts ❛r❡ r❛♥❞♦♠❧② ❛ss✐❣♥❡❞ t♦ ♦♥❡ ♦❢ t❤r❡❡ tr❡❛t♠❡♥ts ✇✐t❤ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s ♦❢ 30%,60% ♦r 90%✳ ❲❡ ❝♦♥✜r♠ ✇❡❛❦♥❡ss❡s ♦❢ t❤❡ ■P ♠❡❛s✉r❡ ♣r❡s❡♥t❡❞ ✐♥ ❚❡✐❣❡♥ ❛♥❞ ❏ør❣❡♥s❡♥ ✭✷✵✵✺✮ ❛♥❞ s❤♦✇ t❤❛t ❞❡❝✐s✐♦♥ ♠❛❦❡rs ♥♦t ❡✈❡♥ ❛❞❥✉st t❤❡✐r ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts t♦ ❞✐✛❡r❡♥t ❧❡✈❡❧s ♦❢ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡✳ ❯s✐♥❣ ❞❡❝✐s✐♦♥ ♠❛❦❡rs✬ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts✱ ✇❡ ✜♥❞ ❡✈✐❞❡♥❝❡ t❤❛t ♣❡♦♣❧❡ r❡s♣♦♥❞ t♦ ❛♥ ✐♥❞✐✈✐❞✉❛❧ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ t❤❛t ✐s ✉♥❛✛❡❝t❡❞ ❜② t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ■♥ ❛❧❧ tr❡❛t♠❡♥ts✱ ✇❡ ♦❜s❡r✈❡ ❧❛r❣❡ ✈❛r✐❛t✐♦♥s ✐♥ ✐♥❞✐✈✐❞✉❛❧ ❝♦♥✜❞❡♥❝❡ t❤❛t t❤❡ ■P ♠❡❛s✉r❡ ❞♦❡s ♥♦t t❛❦❡ ✐♥t♦ ❛❝❝♦✉♥t✳ ❆s ❛ ❝♦♥s❡q✉❡♥❝❡✱ t❤❡ ❝♦♥s✐st❡♥❝② ♦❢ t❤❡ t✇♦ ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡s ✐s ❧✐♠✐t❡❞✳ P❡♦♣❧❡ ♠✐❣❤t ❛♣♣❡❛r ✈❡r② ♦✈❡r❝♦♥✜❞❡♥t ✐♥ t❤❡ ■P s✐♠♣❧② ❜❡❝❛✉s❡ t❤❡✐r tr✉❡ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ✐s ♦✈❡r❡st✐♠❛t❡❞ ❜② t❤❡ r❡q✉❡st❡❞ ❧❡✈❡❧ ❛♥❞ ✈✐❝❡ ✈❡rs❛✳ ❲❡ ❝♦♥❝❧✉❞❡ t❤❛t t❤❡ ■P ❛s ✐t ✐s ❝✉rr❡♥t❧② ✉s❡❞✱ ✐✳❡✳ ❝♦♠♣❛r✐♥❣ ❤✐t r❛t❡s ✇✐t❤ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s✱ ❤❛s ♠❛❥♦r s❤♦rt❝♦♠✐♥❣s ❢♦r ♠❡❛s✉r✐♥❣ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛t ❜♦t❤ t❤❡ ❛❣❣r❡❣❛t❡ ❛♥❞ ✐♥❞✐✈✐❞✉❛❧ ❧❡✈❡❧✳ ❆s ❛♥ ❛❧t❡r♥❛t✐✈❡✱ ✇❡ ♣r♦♣♦s❡ t♦ ✉s❡ ♣❡♦♣❧❡✬s ❢r❡q✉❡♥❝② ✸Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ❥✉❞❣♠❡♥ts t♦ ♠❡❛s✉r❡ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ ❲❡ ✇♦✉❧❞ ❧✐❦❡ t♦ ♠❡♥t✐♦♥ t❤❛t ✇❡ ❛r❡ ♥♦t t❤❡ ✜rst t♦ q✉❡st✐♦♥ t❤❡ r❡❧✐❛❜✐❧✐t② ♦❢ ♠❡❛s✉r✐♥❣ ♦✈❡r❝♦♥✜❞❡♥❝❡ ✇✐t❤ t❤❡ ■P ♦♥ t❤❡ ❛❣❣r❡❣❛t❡ ❧❡✈❡❧✳ ◆❡①t t♦ t❤❡ ❛❢♦r❡♠❡♥t✐♦♥❡❞ st✉❞✐❡s ✭❚❡✐❣❡♥ ❛♥❞ ❏ør❣❡♥s❡♥✱ ✷✵✵✺❀ ❈❡s❛r✐♥✐ ❡t ❛❧✳✱ ✷✵✵✻❀ ❲✐♥♠❛♥ ❡t ❛❧✳✱ ✷✵✵✹✮✱ ♠❛♥② ♣r❡✈✐♦✉s st✉❞✐❡s ❤❛✈❡ r❛✐s❡❞ s♣❡❝✐✜❝ ♠❡t❤♦❞♦❧♦❣✐❝❛❧ ❝♦♥❝❡r♥s ✇✐t❤ ❝♦♥s❡q✉❡♥❝❡s ❢♦r t❤❡ ❛❣❣r❡❣❛t❡ ❧❡✈❡❧ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ ■t ❤❛s ❜❡❡♥ s❤♦✇♥✱ ❢♦r ❡①❛♠♣❧❡✱ t❤❛t t❤❡ ❛❧t❡r♥❛t✐✈❡ t✇♦✲❝❤♦✐❝❡ q✉❡st✐♦♥ ❢♦r♠❛t ✭❡✳❣✳✱ ❑♦r✐❛t ❡t ❛❧✳✱ ✶✾✽✵✮ ②✐❡❧❞s ❧♦✇❡r ❧❡✈❡❧s ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ t❤❛♥ t❤❡ ■P ✭❑❧❛②♠❛♥ ❡t ❛❧✳✱ ✶✾✾✾✮✳ ❙✐♠✐❧❛r❧②✱ ❡①❝❧✉s✐♦♥ ✐♥str✉❝t✐♦♥s t❤❛t ✐♥❞✉❝❡ ♣❡♦♣❧❡ t♦ t❤✐♥❦ ❛❜♦✉t t❤❡ ✈❛❧✉❡s t❤❛t ❧✐❡ ♦✉ts✐❞❡ ♦❢ t❤❡✐r ♣r♦✈✐❞❡❞ ✐♥t❡r✈❛❧s ✐♥ t❤❡ ■P ②✐❡❧❞ s♠❛❧❧❡r ❧❡✈❡❧s ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ t❤❛♥ ✐♥❝❧✉s✐♦♥ q✉❡st✐♦♥s ✭❙♦❧❧ ❛♥❞ ❑❧❛②♠❛♥✱ ✷✵✵✹❀ ❚❡✐❣❡♥ ❛♥❞ ❏ør❣❡♥s❡♥✱ ✷✵✵✺❀ ❨❛♥✐✈ ❛♥❞ ❙❝❤✉❧✱ ✶✾✾✼✮✳ ■♥t❡r❡st✐♥❣❧②✱ ♦t❤❡r ❢♦r♠s ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ s✉❝❤ ❛s ❜❡tt❡r✲ t❤❛♥✲❛✈❡r❛❣❡ ❜❡❧✐❡❢s ✭❙✈❡♥s♦♥✱ ✶✾✽✶✮ ♦r t❤❡ ✐❧❧✉s✐♦♥ ♦❢ ❝♦♥tr♦❧ ✭▲❛♥❣❡r✱ ✶✾✼✺✮ ✇❡r❡ ❢♦✉♥❞ t♦ ❜❡ ✐♥❝♦♥s✐st❡♥t ✇✐t❤ t❤❡ ■P ♠❡❛s✉r❡ ✭❉❡❛✈❡s ❡t ❛❧✳✱ ✷✵✵✽❀ ●❧❛s❡r ❛♥❞ ❲❡❜❡r✱ ✷✵✵✼❀ ❍✐❧t♦♥ ❡t ❛❧✳✱ ✷✵✶✶❀ ▼❡♥❦❤♦✛ ❡t ❛❧✳✱ ✷✵✵✻❀ ▼♦♦r❡ ❛♥❞ ❍❡❛❧②✱ ✷✵✵✽✮✳ ▼♦r❡♦✈❡r✱ t❤❡ ❧❡✈❡❧ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❤❛s ❜❡❡♥ ❢♦✉♥❞ t♦ ✈❛r② ✇✐t❤ t❤❡ ❞✐✣❝✉❧t② ✭▲✐❝❤t❡♥st❡✐♥ ❡t ❛❧✳✱ ✶✾✽✷✮ ❛♥❞ ❞♦♠❛✐♥ ✭❑❧❛②♠❛♥ ❡t ❛❧✳✱ ✶✾✾✾✮ ♦❢ t❤❡ q✉❡st✐♦♥ s❡t✳ ❖t❤❡r st✉❞✐❡s ❛ttr✐❜✉t❡ ♦✈❡r❝♦♥✜❞❡♥❝❡ t♦ ♠❡❛s✉r❡♠❡♥t ❡rr♦rs ❛♥❞ s❛♠♣❧✐♥❣ ❡✛❡❝ts ♦❢ t❤❡ q✉❡st✐♦♥ s❡t t❤❛t ✐♠♣❛✐r t❤❡ ✈❛❧✐❞✐t② ♦❢ ♦t❤❡r✇✐s❡ ✇❡❧❧✲✇♦r❦✐♥❣ ❝✉❡s ✭●✐❣❡r❡♥③❡r ❡t ❛❧✳✱ ✶✾✾✶❀ ❏✉s❧✐♥✱ ✶✾✾✸✱ ✶✾✾✹❀ ❙♦❧❧✱ ✶✾✾✻✮✳ ❖✉r ✇♦r❦ ❝♦♥tr✐❜✉t❡s t♦ t❤❡ ✉♥❞❡rst❛♥❞✐♥❣ ♦❢ ❤♦✇ ❛♣♣r♦♣r✐❛t❡ t❤❡ ■P ♠❡❛s✉r❡ ✐s ♦♥ t❤❡ ❛❣❣r❡❣❛t❡ ❧❡✈❡❧✳ ❆❞❞✐t✐♦♥❛❧❧②✱ ♦✉r ✜♥❞✐♥❣s ❝❛♥ ❤❡❧♣ t♦ ❡①♣❧❛✐♥ ♣r❡✈✐♦✉s ♣✉③③❧✐♥❣ r❡s✉❧ts ♦♥ t❤❡ ✐♥❞✐✈✐❞✉❛❧ ❧❡✈❡❧✳ ❲❤✐❧❡ t❤❡ ■P ❤❛s ❜❡❡♥ ❢♦✉♥❞ t♦ ♣r❡❞✐❝t s♦♠❡ ❜❡❤❛✈✐♦r ❧✐❦❡ ✐♥♥♦✈❛t✐✈❡ ❛❝t✐✈✐t②✱ r✐s❦✲ t❛❦✐♥❣ ❛♥❞ ♦r❞❡r✐♥❣ ❞❡❝✐s✐♦♥s ♦❢ ♠❛♥❛❣❡rs ✭❇❡♥✲❉❛✈✐❞ ❡t ❛❧✳✱ ✷✵✶✸❀ ❍❡r③ ❡t ❛❧✳✱ ✷✵✶✹❀ ❘❡♥ ❛♥❞ ❈r♦s♦♥✱ ✷✵✶✸✮✱ ❛ ✇✐❞❡ r❛♥❣❡ ♦❢ st✉❞✐❡s ✐s ♥♦t ❛❜❧❡ t♦ ❡①♣❧❛✐♥ ❡❝♦♥♦♠✐❝ ❜❡❤❛✈✐♦r ✇✐t❤ t❤❡ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡✳ ■♥ ❛ r❡❝❡♥t st✉❞②✱ ❢♦r ❡①❛♠♣❧❡✱ ❆❝❦❡rt ❡t ❛❧✳ ✭✷✵✶✺✮ ✜♥❞ t❤❛t ✐♥✈❡st♦rs ✇❤♦ ❞✐✈❡rs✐❢② ♠♦r❡ ✐♥ ❛♥ ❡①♣❡r✐♠❡♥t❛❧ ❛ss❡t ♠❛r❦❡t ❛♣♣❡❛r ♠♦r❡ ♦✈❡r❝♦♥✜✲ ❞❡♥t ✐♥ t❤❡ ■P ✇❤✐❝❤ ✐s ❛t ♦❞❞s ✇✐t❤ ♣r❡✈✐♦✉s ✜♥❞✐♥❣s ✭❡✳❣✳ ●♦❡t③♠❛♥♥ ❛♥❞ ❑✉♠❛r✱ ✷✵✵✽✮✳ ❙✐♠✐❧❛r❧②✱ ❞❡s♣✐t❡ s❤♦✇✐♥❣ ❛ ♥❡❣❛t✐✈❡ r❡❧❛t✐♦♥ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛♥❞ tr❛❞✐♥❣ ♣❡r❢♦r♠❛♥❝❡ ✭❇✐❛✐s ❡t ❛❧✳✱ ✷✵✵✺❀ ❉❡❛✈❡s ❡t ❛❧✳✱ ✷✵✵✽❀ ❑✐r❝❤❧❡r✱ ✷✵✵✷✮ ❡①♣❡r✐♠❡♥t❛❧ st✉❞✐❡s ♠♦st❧② ❢❛✐❧ t♦ ❝♦♥✜r♠ t❤❡ ❡♠♣✐r✐❝❛❧❧② ❤②♣♦t❤❡s✐③❡❞ ❧✐♥❦ ❜❡t✇❡❡♥ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛♥❞ tr❛❞✐♥❣ ✈♦❧✉♠❡ ✭❇✐✲ ❛✐s ❡t ❛❧✳✱ ✷✵✵✺❀ ❋❡❧❧♥❡r✲❘ö❤❧✐♥❣ ❛♥❞ ❑rü❣❡❧✱ ✷✵✶✹❀ ●❧❛s❡r ❛♥❞ ❲❡❜❡r✱ ✷✵✵✼❀ ❑✐r❝❤❧❡r✱ ✷✵✵✷✮❀ ❢♦r ❡♠♣✐r✐❝❛❧ st✉❞✐❡s s❡❡ ✭❇❛r❜❡r ❛♥❞ ❖❞❡❛♥✱ ✷✵✵✵✱ ✷✵✵✶❀ ❙t❛t♠❛♥ ❡t ❛❧✳✱ ✷✵✵✻✮✳✹ ❚❤❡s❡ ✹❉❡❛✈❡s ❡t ❛❧✳ ✭✷✵✵✽✮ ❞♦ ✜♥❞ ❛ ♣♦s✐t✐✈❡ ❝♦rr❡❧❛t✐♦♥ ❜❡t✇❡❡♥ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛♥❞ tr❛❞✐♥❣ ✈♦❧✉♠❡✳ ❍♦✇❡✈❡r✱ ❞✉❡ t♦ t❤❡ s♣❡❝✐✜❝ ❡①♣❡r✐♠❡♥t❛❧ ❞❡s✐❣♥ t❤❡② ❝❛♥♥♦t ❡①❝❧✉❞❡ t❤❛t t❤❡ r❡s✉❧ts ❛r❡ ❞r✐✈❡♥ ❜② ❜❡tt❡r✲t❤❛♥✲ ❛✈❡r❛❣❡ ❜❡❧✐❡❢s✳ ❙✐♠✐❧❛r❧②✱ ▼✐❝❤❛✐❧♦✈❛ ❛♥❞ ❙❝❤♠✐❞t ✭✷✵✶✶✮ ✜♥❞ ❧❛r❣❡r tr❛❞✐♥❣ ✈♦❧✉♠❡s ✐♥ ♠❛r❦❡ts ❝♦♥s✐st✐♥❣ ✹Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ♣✉③③❧✐♥❣ ✜♥❞✐♥❣s ❧❡❞ ❇❛r❜❡r ❛♥❞ ❖❞❡❛♥ ✭✷✵✶✸✮ t♦ ❤②♣♦t❤❡s✐③❡ t❤❛t ✏t❤✐s ✇❡❛❦ ❧✐♥❦ ♠✐❣❤t ❜❡ ♣❛rt✐❛❧❧② ❡①♣❧❛✐♥❡❞ ❜② t❤❡ ❝✉rr❡♥t ✐♥❛❜✐❧✐t② t♦ ♠❡❛s✉r❡ ♠✐s❝❛❧✐❜r❛t✐♦♥ ✇❡❧❧✳✑ ❖✈❡r❛❧❧✱ ♦✉r r❡s✉❧ts s❤♦✇ t❤❛t t❤❡r❡ ✐s q✉✐t❡ s♦♠❡ tr✉t❤ ✐♥ t❤✐s ❤②♣♦t❤❡s✐s✳✷ ❊①♣❡r✐♠❡♥t❛❧ ❞❡s✐❣♥
✷✳✶ ❊①♣❡r✐♠❡♥t❛❧ Pr♦❝❡❞✉r❡
❖✉r ❡①♣❡r✐♠❡♥t❛❧ ❞❡s✐❣♥ ❝♦♥s✐sts ♦❢ t✇♦ ♠❛✐♥ ♣❛rts✿ ❚❤❡ ✜rst ♣❛rt ✐♥✈♦❧✈❡s t❤❡ ■P ✉s✐♥❣ t❡♥ ❣❡♥❡r❛❧ ❦♥♦✇❧❡❞❣❡ ✐t❡♠s ✭s❡❡ ❚❛❜❧❡ ❆✳✶ ✐♥ ❆♣♣❡♥❞✐① ❇✮ ❛s ✉s✉❛❧❧② ❡♠♣❧♦②❡❞ ✐♥ t❤❡ ❧✐t❡r❛t✉r❡✳ ❚♦ ❡st❛❜❧✐s❤ ❝♦♠♣❛r❛❜✐❧✐t② ♦❢ ♦✉r r❡s✉❧ts✱ ✇❡ ✉s❡ t❤❡ s❛♠❡ t❡st ❛s ❇✐❛✐s ❡t ❛❧✳ ✭✷✵✵✺✮ ❛♥❞ ❍✐❧t♦♥ ❡t ❛❧✳ ✭✷✵✶✶✮✳ P❛rt✐❝✐♣❛♥ts ❛r❡ ✐♥str✉❝t❡❞ t♦ ❡st✐♠❛t❡ ❜♦✉♥❞❛r✐❡s ❢♦r t❤❡s❡ t❡♥ ❣❡♥❡r❛❧ ❦♥♦✇❧❡❞❣❡ q✉❡st✐♦♥s s♦ t❤❛t t❤❡✐r ❝❡rt❛✐♥t② ❡q✉❛❧s t❤❡ r❡q✉❡st❡❞ ❧❡✈❡❧ ♦❢ ❝♦♥✜❞❡♥❝❡✳ ❲❡ ✉s❡ t❤r❡❡ tr❡❛t♠❡♥ts ✇✐t❤ ❞✐✛❡r❡♥t ❧❡✈❡❧s ♦❢ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡✿ ✸✵✪✱ ✻✵✪ ❛♥❞ ✾✵✪✳ ❚♦ s✉♣♣♦rt ♣❛rt✐❝✐♣❛♥ts✬ ❛tt❡♥t✐♦♥ t♦ t❤❡✐r r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❛♥❞ t❤❡✐r ✉♥❞❡rst❛♥❞✐♥❣ ♦❢ ✐ts ✐♠♣❧✐❝❛t✐♦♥✱ ✇❡ t❛❦❡ s❡✈❡r❛❧ ♣r❡❝❛✉t✐♦♥s✳ ❋✐rst✱ ✇❡ ✐♥❝r❡❛s❡ t❤❡ s❛❧✐❡♥❝❡ ♦❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✱ ❡①♣r❡ss❡❞ ✐♥ ♣❡r❝❡♥t❛❣❡✱ ❜② ❝❧❡❛r❧② ❤✐❣❤❧✐❣❤t✐♥❣ t❤❡ r❡❧❡✈❛♥t t❡①t ❢r♦♠ t❤❡ r❡♠❛✐♥✐♥❣ ✐♥str✉❝t✐♦♥s✳ ❙❡❝♦♥❞✱ ✇❡ ❡①♣❧✐❝✐t❧② st❛t❡ ❤♦✇ ♠❛♥② ♦❢ t❤❡ t❡♥ tr✉❡ ✈❛❧✉❡s s❤♦✉❧❞ ♦♥ ❛✈❡r❛❣❡ ❢❛❧❧ ✐♥s✐❞❡ ♦❢ t❤❡ ♣r♦✈✐❞❡❞ ✐♥t❡r✈❛❧s ❛s ♣❡♦♣❧❡ ♠❛② str✉❣❣❧❡ ✇✐t❤ ♣❡r❝❡♥t❛❣❡s ✭●✐❣❡r❡♥③❡r ❛♥❞ ❍♦✛r❛❣❡✱ ✶✾✾✺✮✳ ❋✐♥❛❧❧②✱ ✇❡ ❡①♣❧❛✐♥ t❤❡ ✐♥t✉✐t✐♦♥ ♦❢ t❤❡ ❛ss✐❣♥♠❡♥t ❜② ❛ ❝♦♥❝r❡t❡ ❡①❛♠♣❧❡✳ ❈♦♠♣❧❡t❡ ✐♥str✉❝t✐♦♥s ♦❢ t❤❡ ❡①♣❡r✐♠❡♥t ❝❛♥ ❜❡ ❢♦✉♥❞ ✐♥ ❆♣♣❡♥❞✐① ❇✳ ❆❢t❡r t❤❡ st❛♥❞❛r❞ ■P ♣r♦❝❡❞✉r❡✱ ✇❡ ❡❧✐❝✐t ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❜② r❡q✉❡st✐♥❣ ♣❛rt✐❝✐♣❛♥ts t♦ ❡st✐♠❛t❡ ❤♦✇ ♠❛♥② tr✉❡ ✐t❡♠s ❢❛❧❧ ✐♥s✐❞❡ t❤❡✐r ✐♥t❡r✈❛❧s✳✺ ❚❤❡s❡ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ♣r♦✈✐❞❡ ❛ s❡❧❢✲r❡♣♦rt❡❞ ♣r♦①② ❢♦r t❤❡ ✐♥❞✐✈✐❞✉❛❧ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❛♥❞ ❛❧❧♦✇ ✉s t♦ ❝♦♠♣✉t❡ ❛♥ ❛❧t❡r♥❛t✐✈❡ ♠❡❛s✉r❡ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ t❤❛t ❛❝❝♦✉♥ts ❢♦r ✐♥❞✐✈✐❞✉❛❧ ❞✐✛❡r❡♥❝❡s ✐♥ ❝♦♥✜❞❡♥❝❡✳ ■♥ ❛❞❞✐t✐♦♥✱ ✇❡ ❡❧✐❝✐t t❤❡ ♣❡r❝❡✐✈❡❞ ❞❡❣r❡❡ ♦❢ ❞✐✣❝✉❧t② ♦♥ ❛ ✜✈❡ ♣♦✐♥t ▲✐❦❡rt s❝❛❧❡ ✭✶❂✈❡r② ❡❛s②✱ ✺❂✈❡r② ❞✐✣❝✉❧t✮ t♦ ❝♦♥tr♦❧ ❢♦r t❤❡ ❤❛r❞✲❡❛s② ❡✛❡❝t ✭●✐❣❡r❡♥③❡r ❡t ❛❧✳✱ ✶✾✾✶✮✳ ❋✐♥❛❧❧②✱ ✇❡ ❛✐♠ t♦ s❤❡❞ ❧✐❣❤t ♦♥ t❤❡ ❝♦♥str✉❝t✐♦♥ ♦❢ ✐♥t❡r✈❛❧s ❛♥❞ ❛s❦ ♦✉r ♣❛rt✐❝✐♣❛♥ts t♦ ♦❢ ♦✈❡r❝♦♥✜❞❡♥t tr❛❞❡rs ❝♦♠♣❛r❡❞ t♦ ♠❛r❦❡ts ✇✐t❤ ❧❡ss ♦✈❡r❝♦♥✜❞❡♥t tr❛❞❡rs ✉s✐♥❣ ❛♥ ❛❧t❡r♥❛t✐✈❡ ♠❡❛s✉r❡ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❜❛s❡❞ ♦♥ ♣❡r❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡✳ ✺■♥ ♣r✐♥❝✐♣❧❡✱ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❝❛♥ ❜❡ ✐♥❝❡♥t✐✈✐③❡❞ ❜② ♠❡❛♥s ♦❢ ❝♦♠♠♦♥ ✐♥❝❡♥t✐✈❡ ❝♦♠♣❛t✐❜❧❡ ♠❡❝❤❛♥✐s♠s✳ ❈❡s❛r✐♥✐ ❡t ❛❧✳ ✭✷✵✵✻✮ ❝♦♠♣❛r❡❞ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❜❡t✇❡❡♥ t✇♦ ❣r♦✉♣s ♦❢ ✇❤✐❝❤ ♦♥❧② ♦♥❡ ✇❛s ✐♥❝❡♥t✐✈✐③❡❞ ❛♥❞ ❞✐❞ ♥♦t ✜♥❞ s✐❣♥✐✜❝❛♥t ❞✐✛❡r❡♥❝❡s✳ ■♥ ❧✐❣❤t ♦❢ t❤✐s ✇❡❛❦ ❡✈✐❞❡♥❝❡ ♦❢ t❤❡ ✐♠♣❛❝t ♦❢ ♠♦♥❡t❛r② ✐♥❝❡♥t✐✈❡s ✇❡ r❡❢r❛✐♥❡❞ ❢r♦♠ ✉s✐♥❣ ✐♥❝❡♥t✐✈✐③❡❞ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts✳ ✺Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ❡①♣❧❛✐♥ ❤♦✇ t❤❡② ❣❡♥❡r❛t❡❞ t❤❡✐r ✐♥t❡r✈❛❧ ❡st✐♠❛t❡s ✇✐t❤ ❛ ❢r❡❡ t❡①t q✉❡st✐♦♥✳✻ ■♥ t❤❡ s❡❝♦♥❞ ♣❛rt ✇❡ ❡❧✐❝✐t s❡✈❡r❛❧ ❜❡❤❛✈✐♦r❛❧ ❝❤❛r❛❝t❡r✐st✐❝s t♦ ❛♥❛❧②③❡ ✇❤❡t❤❡r t❤❡② ❝❛♥ ❡①♣❧❛✐♥ ✐♥❞✐✈✐❞✉❛❧ ❞✐✛❡r❡♥❝❡s ✐♥ t❤❡ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡✳ ❲❡ ♠❡❛s✉r❡ r✐s❦✲ ❛♥❞ ❧♦ss ❛✈❡rs✐♦♥ ♣❛r❛♠❡t❡rs ♦❢ ♣❛rt✐❝✐♣❛♥ts ✉s✐♥❣ ✐♥❝❡♥t✐✈✐③❡❞ ❜✐♥❛r② ❝❤♦✐❝❡ q✉❡st✐♦♥s s✐♠✐❧❛r t♦ ❆❜❞❡❧❧❛♦✉✐ ❡t ❛❧✳ ✭✷✵✵✽✮✳ ❚♦ t❡st ❝♦❣♥✐t✐✈❡ s❦✐❧❧s ✇❡ ❧❡t ♣❛rt✐❝✐♣❛♥ts ♣❡r❢♦r♠ ❛♥ ❛❞❛♣t❡❞ ✈❡rs✐♦♥ ♦❢ t❤❡ ❈♦❣♥✐t✐✈❡ ❘❡✢❡❝t✐♦♥ ❚❡st ✭❈❘❚✮ ♦❢ ❋r❡❞❡r✐❝❦ ✭✷✵✵✺✮✳ ❙✐♥❝❡ t❤❡ ■P r❡q✉✐r❡s t❤❛t ♣❛rt✐❝✐♣❛♥ts r❡♣♦rt ✐♥t❡r✈❛❧s ♦❢ t❤❡✐r s✉❜❥❡❝t✐✈❡ ♣r♦❜❛❜✐❧✐t② ❞✐str✐❜✉t✐♦♥s ❢♦r ♥♦♥✲r❛♥❞♦♠ ✈❛❧✉❡s✱ ✇❡ ❢✉rt❤❡r ❡❧✐❝✐t r✐s❦ ❧✐t❡r❛❝② ✇✐t❤ t❤❡ ❇❡r❧✐♥ ◆✉♠❡r❛❝② ❚❡st ✭❇◆❚✮ ✭❈♦❦❡❧② ❡t ❛❧✳✱ ✷✵✶✷✮✳ ❋✐♥❛❧❧②✱ ✇❡ ❛ss❡ss t❤❡ ❞❡❣r❡❡ t♦ ✇❤✐❝❤ ♣❛rt✐❝✐♣❛♥ts ❛✈♦✐❞ t❤❡ ✉s❡ ♦❢ ✇✐❞❡ ✐♥t❡r✈❛❧s ✉s✐♥❣ ❛ ♠❡t❤♦❞♦❧♦❣② ❜♦rr♦✇❡❞ ❢r♦♠ ❨❛♥✐✈ ❛♥❞ ❋♦st❡r ✭✶✾✾✺✮✳ ❆ ❞❡t❛✐❧❡❞ ❞❡s❝r✐♣t✐♦♥ ♦❢ t❤❡ ❡❧✐❝✐t❛t✐♦♥ ♠❡t❤♦❞♦❧♦❣✐❡s ✐s ♣r♦✈✐❞❡❞ ✐♥ ❆♣♣❡♥❞✐① ❇✳✷✳✷ P❛rt✐❝✐♣❛♥ts ❛♥❞ ■♥❝❡♥t✐✈❡s
❲❡ ❝♦♥❞✉❝t❡❞ t✇♦ ♦♥❧✐♥❡ s✉r✈❡②s ✐♥ ❉❡❝❡♠❜❡r ✷✵✶✹ ❛♥❞ ❏❛♥✉❛r② ✷✵✶✺ ✇✐t❤ ❛ t♦t❛❧ ♦❢ ✸✵✵ ♣❛rt✐❝✐♣❛♥ts✳ ❖♥❡ s✉r✈❡② ✐♥✈♦❧✈❡s ✶✺✶ ♣❛rt✐❝✐♣❛♥ts ♦♥ t❤❡ ❝r♦✇❞✲s♦✉r❝✐♥❣ ♣❧❛t❢♦r♠ ❆♠❛③♦♥ ▼❡❝❤❛♥✐❝❛❧ ❚✉r❦ ✭▼❚✉r❦✮✳ ❚❤❡ s✉r✈❡② ✐♥str✉❝t✐♦♥s ✇❡r❡ ✐♥ ❊♥❣❧✐s❤ ✉s✐♥❣ ✐♠♣❡r✐❛❧ ✉♥✐ts ❢♦r t❤❡ ■P q✉❡st✐♦♥s✳ ▼❚✉r❦ ♣❛rt✐❝✐♣❛♥ts r❡❝❡✐✈❡❞ ❛ ♣❛rt✐❝✐♣❛t✐♦♥ ❢❡❡ ♦❢ ✩✶✳✽✵✳ ❚♦ ♣r♦✈✐❞❡ ❛❞❞✐t✐♦♥❛❧ r♦❜✉st♥❡ss ❢♦r ♦✉r r❡s✉❧ts✱ ✇❡ r❡❝r✉✐t❡❞ ✶✹✾ ♣❛rt✐❝✐♣❛♥ts ❢r♦♠ t❤❡ ❯♥✐✈❡rs✐t② ♦❢ ❩✉r✐❝❤ ✐♥ ❛ s❡❝♦♥❞ s✉r✈❡②✳ ■♥✈✐t❛t✐♦♥s ❢♦r t❤✐s s✉r✈❡② ✇❡r❡ s❡♥t ✈✐❛ ❡♠❛✐❧ t♦ ❛ ❜❛❝❤❡❧♦r ❧❡✈❡❧ ✜♥❛♥❝❡ ❝❧❛ss ❛♥❞ t✇♦ ♠❛st❡r ❧❡✈❡❧ ✜♥❛♥❝❡ ❛♥❞ ❡❝♦♥♦♠✐❝s ❝❧❛ss❡s✳ ❚❤❡ s✉r✈❡② ✐♥str✉❝t✐♦♥s ✇❡r❡ ❡✐t❤❡r ✐♥ ❊♥❣❧✐s❤ ♦r ●❡r♠❛♥ ❞❡♣❡♥❞✐♥❣ ♦♥ t❤❡ ❧❛♥❣✉❛❣❡ ♦❢ t❤❡ ❝♦✉rs❡ t❤❛t ✇❡ r❡❝r✉✐t❡❞ ❢r♦♠✳ ❙t✉❞❡♥ts ✇❡r❡ ❛s❦❡❞ t♦ ♣r♦✈✐❞❡ ■P ✐t❡♠s ✐♥ ♠❡tr✐❝ ✉♥✐ts✳ ■♥ ❡❛❝❤ s✉r✈❡②✱ ✜✈❡ ♣❛rt✐❝✐♣❛♥ts ✇❡r❡ r❛♥❞♦♠❧② ❝❤♦s❡♥ ❢♦r ❛ ❜♦♥✉s ♣❛②♠❡♥t t♦ ✐♥❝❡♥t✐✈✐③❡ t❤❡✐r r✐s❦② ❝❤♦✐❝❡s ✐♥ P❛rt ✷ ♦❢ t❤❡ s✉r✈❡②✳✼ ❚♦ ❡♥s✉r❡ t❤❛t ♦✉r ❞❛t❛ ❝♦♠❡ ❢r♦♠ ♣❛rt✐❝✐♣❛♥ts ✇❤♦ ❝♦♠♣❧❡t❡❞ ❛❧❧ t❛s❦s ♣r♦♣❡r❧② ❛♥❞ ✇✐t❤♦✉t ✐♥t❡rr✉♣t✐♦♥s✱ ✇❡ ♦♥❧② ✐♥❝❧✉❞❡ t❤♦s❡ ♦❜s❡r✈❛t✐♦♥s ❢♦r ✇❤✐❝❤ t❤❡ ❝♦♠♣❧❡t✐♦♥ t✐♠❡ ✇❛s ♠♦r❡ t❤❛♥ ✜✈❡ ❛♥❞ ❧❡ss t❤❛♥ s✐①t② ♠✐♥✉t❡s✳✽ ❚❤❡s❡ ❝♦♥str❛✐♥ts r❡❞✉❝❡ t❤❡ ♥✉♠❜❡r ♦❢ ✻❆❞❞✐t✐♦♥❛❧❧②✱ t❤❡ s✉r✈❡② ✐♥❝❧✉❞❡s t✇♦ q✉❡st✐♦♥s ❢♦r ✇❤✐❝❤ ✇❡ ❞♦ ♥♦t r❡♣♦rt t❤❡ r❡s✉❧ts ✐♥ t❤✐s st✉❞②✳ ❲❡ ❛s❦ ♣❛rt✐❝✐♣❛♥ts t♦ ❡st✐♠❛t❡ t❤❡ ♥✉♠❜❡r ♦❢ ❤✐ts ♦❢ t❤❡ ❛✈❡r❛❣❡ ♣❛rt✐❝✐♣❛♥t ❛♥❞ t♦ ♣r♦✈✐❞❡ ♦♥❡✲②❡❛r ✐♥t❡r✈❛❧ ❢♦r❡❝❛sts ❢♦r t❤❡ ❧❡✈❡❧ ♦❢ ❡✐t❤❡r t❤❡ ❉♦✇ ❏♦♥❡s ■♥❞✉str✐❛❧ ❆✈❡r❛❣❡ ♦r t❤❡ ❙✇✐ss ▼❛r❦❡t ■♥❞❡① ❞❡♣❡♥❞✐♥❣ ♦♥ t❤❡ ♣❛rt✐❝✐♣❛♥t s❛♠♣❧❡✳ ✼❚❤❡ ❛✈❡r❛❣❡ ♣❛②♠❡♥ts ❛♠♦✉♥t❡❞ t♦ ✹✻✳✽ ❈❍❋ ❛♥❞ ✩ ✷✷✳✹ ✐♥ t❤❡ ❙t✉❞❡♥t ❛♥❞ ▼❚✉r❦ s❛♠♣❧❡✱ r❡s♣❡❝✲ t✐✈❡❧②✳ ❲❡ ❞❡❧✐❜❡r❛t❡❧② ❝❤♦s❡ ❧❛r❣❡r st❛❦❡s ✐♥ t❤❡ ❙t✉❞❡♥t s❛♠♣❧❡ s✐♥❝❡ ❛ ♣✐❧♦t st✉❞② ✐♥❞✐❝❛t❡❞ ❧♦♥❣❡r ❝♦♠♣❧❡t✐♦♥ t✐♠❡s ❢♦r st✉❞❡♥ts✳ ❆t t❤❡ t✐♠❡ ♦❢ t❤❡ st✉❞②✱ ✹✻✳✽ ❈❍❋ ✇❡r❡ ✇♦rt❤ ✩ ✸✶✳✹✸ ✐♥ t❤❡ ❯✳❙✳ ❝♦♥s✐❞✲ ❡r✐♥❣ t❤❡ ♣✉r❝❤❛s✐♥❣ ♣♦✇❡r ♣❛r✐t② ❧❡✈❡❧ ♣r♦✈✐❞❡❞ ❜② ❖❊❈❉ ✭❤tt♣✿✴✴st❛ts✳♦❡❝❞✳♦r❣✮✳ ✽❲❡ ✐♠♣♦s❡ t❤❡ ❧♦✇❡r ❜♦✉♥❞ t✐♠❡ ❝♦♥str❛✐♥❡❞ ❜❛s❡❞ ♦♥ ❡①♣❡r✐❡♥❝❡ ❢r♦♠ ❛ ♣✐❧♦t st✉❞② t♦ ❡①❝❧✉❞❡ ▼❚✉r❦ ♣❛rt✐❝✐♣❛♥ts t❤❛t ❝❧✐❝❦✲t❤r♦✉❣❤ t❤❡ s✉r✈❡② t♦ ♠❛①✐♠✐③❡ t❤❡✐r ❤♦✉r❧② ✇❛❣❡ ✇✐t❤♦✉t ❝❛r❡❢✉❧❧② r❡❛❞✐♥❣ t❤❡ ✻Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ♣❛rt✐❝✐♣❛♥ts t♦ ✷✼✻ ✭▼❚✉r❦ s❛♠♣❧❡✿ ✶✸✾❀ ❙t✉❞❡♥t s❛♠♣❧❡✿ ✶✸✼✮✳ ❚❤❡ ❢r❛❝t✐♦♥ ♦❢ ✇♦♠❡♥ ✐s ✹✵✳✸✪ ❛♥❞ ✸✵✳✼✪ ✐♥ ▼❚✉r❦ ❛♥❞ ❙t✉❞❡♥t s❛♠♣❧❡✱ r❡s♣❡❝t✐✈❡❧②✳ ❚❤❡ t✇♦ s❛♠♣❧❡s r❡s❡♠✲ ❜❧❡ t②♣✐❝❛❧ ❞✐✛❡r❡♥❝❡s ❜❡t✇❡❡♥ st✉❞❡♥ts ❛♥❞ ▼❚✉r❦ ♣❛rt✐❝✐♣❛♥ts ✭●♦♦❞♠❛♥ ❡t ❛❧✳✱ ✷✵✶✸✮✿ P❛rt✐❝✐♣❛♥ts ✐♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ❤❛✈❡ ✈❛r✐♦✉s ❡❞✉❝❛t✐♦♥❛❧ ❜❛❝❦❣r♦✉♥❞s ❛♥❞ t❤❡ ❛❣❡ ❞✐s✲ tr✐❜✉t✐♦♥ ✐s ♠♦r❡ ❤❡t❡r♦❣❡♥❡♦✉s ✭❜❡t✇❡❡♥ ✶✽ ②❡❛rs ❛♥❞ ✼✹ ②❡❛rs✮ ❝♦♠♣❛r❡❞ t♦ t❤❡ ②♦✉♥❣❡r st✉❞❡♥ts ✭❜❡t✇❡❡♥ ✶✽ ②❡❛rs ❛♥❞ ✸✵ ②❡❛rs✮✳ ▼♦r❡♦✈❡r✱ t❤❡ ♠❡❞✐❛♥ t✐♠❡ t❤❛t st✉❞❡♥ts s♣❡♥t ♦♥ t❤❡ s✉r✈❡② ✭✷✸✿✷✷♠✐♥✮ ✐s ♠♦r❡ t❤❛♥ t✇✐❝❡ ❛s ❧♦♥❣ ❛s ✐♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ✭✾✿✺✹♠✐♥✮✳ ❙❛♠♣❧❡ ❘❡q✉❡st❡❞ ❈♦♥❢✳ ▼✐♥✳ ◗✶ ▼❡❞✐❛♥ ▼❡❛♥ ◗✸ ▼❛①✳ ▼❚✉r❦ ✸✵✪ ✵ ✷✵ ✸✵ ✸✶✳✸ ✹✵ ✽✵ ▼❚✉r❦ ✻✵✪ ✵ ✷✵ ✸✵ ✸✵✳✼ ✹✵ ✽✵ ▼❚✉r❦ ✾✵✪ ✵ ✷✵ ✹✵ ✹✵✳✵ ✺✵ ✶✵✵ ❇✐❛✐s ❡t ❛❧✳ ✭✷✵✵✺✮ ✾✵✪ ✵ ✷✵ ✸✵ ✸✻✳✵ ✺✵ ✶✵✵ ❚❛❜❧❡ ✶✿ ❚❤✐s t❛❜❧❡ r❡♣♦rts s✉♠♠❛r② st❛t✐st✐❝s ♦❢ t❤❡ ❤✐t r❛t❡ ❞✐str✐❜✉t✐♦♥s ♦❢ ❛❧❧ tr❡❛t♠❡♥ts ✐♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡✳ ❚❤❡ s❡❝♦♥❞ ❝♦❧✉♠♥ ❞✐s♣❧❛②s t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ❚❤❡ ❝♦❧✉♠♥s ◗✶ ❛♥❞ ◗✸ r❡❢❡r t♦ t❤❡ ✜rst ❛♥❞ t❤✐r❞ q✉❛rt✐❧❡s✱ r❡s♣❡❝t✐✈❡❧②✳ ❆❧❧ ✜❣✉r❡s ❛r❡ ✐♥ ♣❡r❝❡♥t❛❣❡s✳ ❚❤✐s t❛❜❧❡ ❛❧s♦ r❡♣♦rts t❤❡ ❤✐t r❛t❡ ❞✐str✐❜✉t✐♦♥ ❢♦✉♥❞ ✐♥ ❇✐❛✐s ❡t ❛❧✳ ✭✷✵✵✺✮✳✸ ❘❡s✉❧ts
❚❤❡ t✇♦ ♣❛rt✐❝✐♣❛♥t s❛♠♣❧❡s ②✐❡❧❞ ✈❡r② ❝♦❤❡r❡♥t r❡s✉❧ts✳ ■♥ t❤✐s s❡❝t✐♦♥✱ ✇❡ t❤❡r❡❢♦r❡ ♣r❡s❡♥t ✜♥❞✐♥❣s ♦❢ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ❛♥❞ ❞✐s❝✉ss ♦♥❧② t❤♦s❡ r❡s✉❧ts ♦❢ t❤❡ st✉❞❡♥t s❛♠♣❧❡ t❤❛t ❛r❡ q✉❛❧✐t❛t✐✈❡❧② ❞✐✛❡r❡♥t✳ ❆ ❞❡t❛✐❧❡❞ ❛♥❛❧②s✐s ♦❢ ♦✉r ✜♥❞✐♥❣s ✐♥ t❤❡ st✉❞❡♥t s❛♠♣❧❡ ✐s ♣r♦✈✐❞❡❞ ✐♥ ❆♣♣❡♥❞✐① ❆✳✸✳✶ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡
●❡♥❡r❛❧❧②✱ ✇❡ ✜♥❞ t❤❛t ❤✐t r❛t❡s ❛r❡ ✈❡r② ❧♦✇ ✇✐t❤ ❛ ❣❧♦❜❛❧ ❛✈❡r❛❣❡ ♦❢ ✸✹✳✷✪✳ ❚❤✐s ✐s ✐♥ ❧✐♥❡ ✇✐t❤ ♣r❡✈✐♦✉s ✜♥❞✐♥❣s ✭❡✳❣✳ ❇✐❛✐s ❡t ❛❧✳✱ ✷✵✵✺❀ ●❧❛s❡r ❛♥❞ ❲❡❜❡r✱ ✷✵✵✼❀ ❍✐❧t♦♥ ❡t ❛❧✳✱ ✷✵✶✶✮✳ ❚❤❡ ❈r♦♥❜❛❝❤ ❛❧♣❤❛ ✐s ✵✳✻✹ ✇❤✐❝❤ ✐♥❞✐❝❛t❡s r❡❛s♦♥❛❜❧❡ ✐♥t❡r♥❛❧ ❝♦♥s✐st❡♥❝② ♦❢ t❤❡ ■P✳ ❚❛❜❧❡ ✶ s❤♦✇s s✉♠♠❛r② st❛t✐st✐❝s ♦❢ t❤❡ ❤✐t r❛t❡ ❞✐str✐❜✉t✐♦♥s ❢♦r ❛❧❧ t❤r❡❡ tr❡❛t♠❡♥ts✱ ✐✳❡✳ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s ♦❢ ✸✵✪✱ ✻✵✪ ❛♥❞ ✾✵✪✳ ❚♦ r✉❧❡ ♦✉t t❤❛t ♦✉r r❡s✉❧ts ❛r❡ ❞r✐✈❡♥ ❜② t❤❡ ❧✐♠✐t❡❞ ❝♦♥tr♦❧ ✐♥ ♦♥❧✐♥❡ ❡①♣❡r✐♠❡♥ts ♦r t❤❡ ♣♦t❡♥t✐❛❧❧② ❧♦✇ q✉❛❧✐✜❝❛t✐♦♥ ♦r ♠♦t✐✈❛t✐♦♥ ♦❢ ▼❚✉r❦ ♣❛rt✐❝✐♣❛♥ts✱ ✇❡ ❝♦♠♣❛r❡ t❤❡ ❤✐t r❛t❡ ❞✐str✐❜✉t✐♦♥s ♦❢ t❤❡ ✾✵✪ tr❡❛t♠❡♥t ✇✐t❤ ✐♥str✉❝t✐♦♥s✳ ❚❤❡ ✉♣♣❡r ❜♦✉♥❞ ❝♦♥str❛✐♥t s❤♦✉❧❞ ♣r❡✈❡♥t ✐♥t❡rr✉♣t✐♦♥s✳ ❊♠♣❧♦②✐♥❣ ♦t❤❡r t✐♠❡ ❝♦♥str❛✐♥ts ✭❡✳❣✳ ❜❡t✇❡❡♥ ✼ ❛♥❞ ✸✵ ♠✐♥✉t❡s✮ ❞♦❡s ♥♦t q✉❛❧✐t❛t✐✈❡❧② ❝❤❛♥❣❡ ♦✉r r❡s✉❧ts✳ ❲❡ ❢✉rt❤❡r ❡①❝❧✉❞❡ t✇♦ ♣❛rt✐❝✐♣❛♥ts ✇❤♦ ❛❞♠✐tt❡❞ ❝❤❡❛t✐♥❣ ✐♥ t❤❡✐r ❢✉❧❧ t❡①t ❡①♣❧❛♥❛t✐♦♥s✳ ✼Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 0.00 0.25 0.50 0.75 1.00 30% 60% 90%Requested Confidence Level (Treatment)
A ve ra g e H it R a te MTurk Students Benchmark ✭❛✮ ❆✈❡r❛❣❡ ❤✐t r❛t❡s ✇✐t❤ st❛♥❞❛r❞ ❡r✲ r♦r ❜❛rs ❛♥❞ ♣❡r❢❡❝t ❝❛❧✐❜r❛t✐♦♥ ❜❡♥❝❤♠❛r❦ ❣r♦✉♣❡❞ ❜② ♣❛rt✐❝✐♣❛♥t s❛♠♣❧❡s ❛♥❞ tr❡❛t✲ ♠❡♥ts✳ 0.0 0.2 0.4 0.6 30% 60% 90%
Requested Confidence Level (Treatment)
A ve ra g e O ve rco n fi d e n ce MTurkStudents ✭❜✮ ❆✈❡r❛❣❡ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❜❛s❡❞ ♦♥ ❤✐t r❛t❡s ✇✐t❤ st❛♥❞❛r❞ ❡rr♦r ❜❛rs ❣r♦✉♣❡❞ ❜② ♣❛rt✐❝✐♣❛♥t s❛♠♣❧❡s ❛♥❞ tr❡❛t♠❡♥ts✳ ❋✐❣✉r❡ ✶✿ ❍✐t r❛t❡s ❛♥❞ t❤❡ ■P ♠❡❛s✉r❡ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ t❤❡ ✜♥❞✐♥❣s ♦❢ ❇✐❛✐s ❡t ❛❧✳ ✭✷✵✵✺✮ ✇❤♦ s♦❧✐❝✐t ♠❛st❡r ❧❡✈❡❧ st✉❞❡♥ts t♦ ❞♦ t❤❡ s❛♠❡ ■P ✇✐t❤ ❛ ✾✵✪ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ✭❛♥❞ t❤❡ s❛♠❡ ■P q✉❡st✐♦♥s✮ ✐♥ ❛ ❧❛❜♦r❛t♦r② s❡tt✐♥❣✳ ■t t✉r♥s ♦✉t t❤❛t t❤❡ t✇♦ ❞✐str✐❜✉t✐♦♥s ❛r❡ str✐❦✐♥❣❧② s✐♠✐❧❛r✱ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ❡✈❡♥ ❛❝❤✐❡✈❡s s❧✐❣❤t❧② ❧❛r❣❡r ❤✐t r❛t❡s✳ ❍❡♥❝❡✱ ♦✉r ■P r❡s✉❧ts ❞♦ ♥♦t s❡❡♠ s♣❡❝✐✜❝ t♦ ❛♥ ♦♥❧✐♥❡ ❡①♣❡r✐♠❡♥t ✇✐t❤ ▼❚✉r❦ ♣❛rt✐❝✐♣❛♥ts✳ ❉❡s♣✐t❡ t❤❡ ❡①tr❡♠❡ ✈❛r✐❛t✐♦♥ ✐♥ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❢r♦♠ ✸✵✪ t♦ ✾✵✪✱ ❛✈✲ ❡r❛❣❡ ❤✐t r❛t❡s ❛♥❞ ❤✐t r❛t❡ ❞✐str✐❜✉t✐♦♥s ♦❢ t❤❡ t❤r❡❡ tr❡❛t♠❡♥ts ❛♣♣❡❛r ❛❧♠♦st ✐❞❡♥t✐❝❛❧✳ ❋✐❣✉r❡ ✶❛ ✐❧❧✉str❛t❡s t❤❡ ❛✈❡r❛❣❡ ❤✐t r❛t❡ ♦❢ ❡❛❝❤ tr❡❛t♠❡♥t ✐♥ ❝♦♠♣❛r✐s♦♥ t♦ ✐ts r❡s♣❡❝t✐✈❡ ♣❡r❢❡❝t ❝❛❧✐❜r❛t✐♦♥ ❜❡♥❝❤♠❛r❦✳ ■♥ ❛❧❧ tr❡❛t♠❡♥ts t❤❡ ❤✐t r❛t❡s ❛r❡ ✈❡r② s✐♠✐❧❛r ❛♥❞ s❡❡♠ ✈✐rt✉❛❧❧② ✉♥❛✛❡❝t❡❞ ❜② t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ❚❤✐s ♦❜s❡r✈❛t✐♦♥ ✐s s✉♣♣♦rt❡❞ ✐♥ ❛ ❑r✉s❦❛❧✲❲❛❧❧✐s r❛♥❦ s✉♠ t❡st ✇❤✐❝❤ ❞♦❡s ♥♦t ②✐❡❧❞ st❛t✐st✐❝❛❧❧② s✐❣♥✐✜❝❛♥t ❞✐✛❡r❡♥❝❡s ✐♥ ❤✐t r❛t❡s ❜❡t✇❡❡♥ t❤❡ t❤r❡❡ tr❡❛t♠❡♥ts✱ ❛❧t❤♦✉❣❤ ❥✉st ♠❛r❣✐♥❛❧❧② ♥♦t ✭p = 0.10✮✳ ❍❡♥❝❡✱ t❤❡ ♣❛rt✐❝✐♣❛♥ts s❤♦✇ ❤❛r❞❧② ❛♥② r❡❛❝t✐♦♥ t♦ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✱ ✇❤✐❝❤ ❝♦♥✜r♠s t❤❡ ✜♥❞✐♥❣s ♦❢ ❚❡✐❣❡♥ ❛♥❞ ❏ør❣❡♥s❡♥ ✭✷✵✵✺✮✳ ❋✐❣✉r❡ ✶❜ ✐❧❧✉str❛t❡s ✇❤❛t t❤✐s r❡s✉❧t ✐♠♣❧✐❡s ❢♦r t❤❡ ❧❡✈❡❧ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ ✐♥ t❤❡ t❤r❡❡ tr❡❛t♠❡♥t ❣r♦✉♣s✳ P❛rt✐❝✐♣❛♥ts ✐♥ t❤❡ ❝♦♠♠♦♥❧② ✉s❡❞ ✾✵✪ ❛♥❞ t❤❡ ✻✵✪ tr❡❛t♠❡♥t ❛r❡ ♦✈❡r✲ ❝♦♥✜❞❡♥t ♦♥ ❛✈❡r❛❣❡✱ ✇❤❡r❡❛s ♣❛rt✐❝✐♣❛♥ts ✐♥ t❤❡ ✸✵✪ tr❡❛t♠❡♥t ❛♣♣❡❛r ❛❧♠♦st ♣❡r❢❡❝t❧② ❝❛❧✐❜r❛t❡❞✳ ❙✐♥❝❡ ♣❛rt✐❝✐♣❛♥ts ❢❛✐❧ t♦ ❛❞❥✉st t❤❡✐r ✐♥t❡r✈❛❧ ❡st✐♠❛t❡s✱ t❤❡ ■P ②✐❡❧❞s ❞❡❣r❡❡s ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ t❤❛t ❧❛r❣❡❧② ❞❡♣❡♥❞ ♦♥ t❤❡ ❛r❜✐tr❛r② ❝❤♦✐❝❡ ♦❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❜② t❤❡ ❡①♣❡r✐♠❡♥t❡r✳ ❆s ❛ ❝♦♥s❡q✉❡♥❝❡✱ ✐t ❛♣♣❡❛rs ❞✐✣❝✉❧t t♦ ❞r❛✇ ❝♦♥❝❧✉s✐♦♥s ❢r♦♠ t❤❡ ■P ❛❜♦✉t t❤❡ tr✉❡ ♠❛❣♥✐t✉❞❡ ♦❢ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛t t❤❡ ❣r♦✉♣ ❧❡✈❡❧✳ ❲❡ ❛♥❛❧②③❡ t❤❡ ❡①t❡♥t t♦ ✇❤✐❝❤ ❜❡❤❛✈✐♦r❛❧ ❝❤❛r❛❝t❡r✐st✐❝s ❛r❡ ❛❜❧❡ t♦ ❡①♣❧❛✐♥ t❤❡ ✽
Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ❍✐t r❛t❡ r❡❣r❡ss✐♦♥s ♦♥ ❜❡❤❛✈✐♦r❛❧ ❢❛❝t♦rs ❈♦♥✜❞❡♥❝❡ ❈❘❚ ❇◆❚ ❘❆ ▲❆ ❲■❆ ❉✐✣❝✉❧t② ●❡♥❞❡r ■♥t❡r❝❡♣t ❆❞❥✳ R2 ★ ♦❜s✳ P❛♥❡❧ ❆✿ ▼❚✉r❦ s❛♠♣❧❡ 0.114 0.131∗ −0.033∗ −0.007 −0.046 −0.060∗ 0.001 0.566∗∗ ✵✳✶✵✶ ✶✸✾ (0.073) (0.050) (0.015) (0.010) (0.064) (0.028) (0.037) (0.151) 0.120 0.133∗ −0.035∗ −0.005 −0.049 −0.071∗ 0.011 0.651∗∗ ✵✳✵✽✽ ✶✸✾ (0.074) (0.061) (0.015) (0.010) (0.065) (0.028) (0.037) (0.149) P❛♥❡❧ ❇✿ ❙t✉❞❡♥t s❛♠♣❧❡ 0.220∗∗ ✵✳✵✻✼ ✲✵✳✵✵✺ ✵✳✵✶✷ ✲✵✳✵✾✾ ✵✳✵✵✹ ✵✳✵✷✾ ✵✳✶✺✽ ✵✳✵✺✷ ✶✸✼ (0.079) (0.068) (0.024) (0.013) (0.071) (0.029) (0.047) (0.141) 0.205∗ ✵✳✵✹✶ ✲✵✳✵✵✺ ✵✳✵✶✷ ✲✵✳✵✾✶ ✵✳✵✶✵ ✵✳✵✹✶ ✵✳✶✻✸ ✵✳✵✹✾ ✶✸✼ (0.079) (0.056) ✭✵✳✵✷✹✮ ✭✵✳✵✶✸✮ ✭✵✳✵✼✵✮ ✭✵✳✵✷✽✮ ✭✵✳✵✹✹✮ ✭✵✳✶✹✷✮ ❙✐❣♥✐✜❝❛♥❝❡ ❧❡✈❡❧s✿∗∗:p <0.01❀∗:p <0.05✳ ❚❛❜❧❡ ✷✿ ❚❤✐s t❛❜❧❡ r❡♣♦rts ❖▲❙ r❡❣r❡ss✐♦♥ ❝♦❡✣❝✐❡♥ts ❛♥❞ t❤❡ r❡s♣❡❝t✐✈❡ st❛♥❞❛r❞ ❡rr♦rs ✐♥ ♣❛r❡♥t❤❡s✐s ♦❢ t❤❡ r❡❣r❡ss✐♦♥ ♠♦❞❡❧HRi=α+β1Conf idencei+β2CAi+β3RAi+β4LAi+β5W IAi+β6Dif f icultyi+β7Genderi+ǫi✇❤❡r❡HRi✐s t❤❡ ❤✐t r❛t❡✱ ❛♥❞CAi ✐s ♦♥❡ ♦❢ t❤❡ t✇♦ ♠❡❛s✉r❡s ♦❢ ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s✿ CRTi ♦r BN Ti✳ P❛♥❡❧ ❆ ❛♥❞ ❇ s❤♦✇ r❡s✉❧ts ♦❢ t❤❡ ▼❚✉r❦ ❛♥❞ st✉❞❡♥t s❛♠♣❧❡✱ r❡s♣❡❝t✐✈❡❧②✳ ❚❤❡ ✐♥❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡s ❛r❡ ❛s ❢♦❧❧♦✇s✿ ❈♦♥✜❞❡♥❝❡ ✐♥❞✐❝❛t❡s t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧❀ ❈❘❚ ✐s t❤❡ r❛t✐♦ ♦❢ ❝♦rr❡❝t ❛♥s✇❡rs ✐♥ t❤❡ ❈♦❣♥✐t✐✈❡ ❘❡✢❡❝t✐♦♥ ❚❡st❀ ❇◆❚ ✐s t❤❡ r❛t✐♦ ♦❢ ❝♦rr❡❝t ❛♥s✇❡rs ✐♥ t❤❡ ❇❡r❧✐♥ ◆✉♠❡r❛❝② ❚❡st❀ ❘❆ ✐s t❤❡ ❡st✐♠❛t❡❞ ♣❛r❛♠❡t❡r ♦❢ r✐s❦ ❛✈❡rs✐♦♥❀ ▲❆ ✐s t❤❡ ❡st✐♠❛t❡❞ ♣❛r❛♠❡t❡r ♦❢ ❧♦ss ❛✈❡rs✐♦♥❀ ❲■❆ ✐s r❛t✐♦ ♦❢ ♥❛rr♦✇ ✐♥t❡r✈❛❧s ❝❤♦s❡♥ ✐♥ t❤❡ ✇✐❞❡ ✐♥t❡r✈❛❧ ❛✈❡rs✐♦♥ t❡st❀ ❉✐✣❝✉❧t② ✐s t❤❡ ♣❡r❝❡✐✈❡❞ ❧❡✈❡❧ ♦❢ ❞✐✣❝✉❧t② ♦❢ t❤❡ q✉❡st✐♦♥ s❡t❀ ●❡♥❞❡r ✐s ❛ ❞✉♠♠② ✈❛r✐❛❜❧❡ ❡q✉❛❧ t♦ ♦♥❡ ✐❢ t❤❡ ♦❜s❡r✈❛t✐♦♥ st❡♠s ❢r♦♠ ❛ ♠❛❧❡ ♣❛rt✐❝✐♣❛♥t✳ ❆ ❞❡t❛✐❧❡❞ ❡①♣❧❛♥❛t✐♦♥ ♦❢ t❤❡ ❝♦♥str✉❝t✐♦♥ ♦❢ t❤❡ ✐♥❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡s ❝❛♥ ❜❡ ❢♦✉♥❞ ✐♥ ❆♣♣❡♥❞✐① ❇✳ ❜❡t✇❡❡♥✲s✉❜❥❡❝t ✈❛r✐❛t✐♦♥ ✐♥ t❤❡ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡ ✉s✐♥❣ ❖▲❙ r❡❣r❡ss✐♦♥s ✇✐t❤ ❤✐t r❛t❡ ❛s ❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡✳ ❚♦ t❤❛t ❡♥❞✱ ✇❡ ♣♦♦❧ t❤❡ ❞❛t❛ ♦❢ t❤❡ t❤r❡❡ tr❡❛t♠❡♥ts ❛♥❞ ✉s❡ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❛s ❛♥ ✐♥❞❡♣❡♥❞❡♥t ✈❛r✐❛❜❧❡✳ ■♥ ❛❧❧ r❡❣r❡ss✐♦♥s ✇❡ ❝♦♥tr♦❧ ❢♦r ❣❡♥❞❡r ❛♥❞ t❤❡ ♣❡r❝❡✐✈❡❞ ❞✐✣❝✉❧t② ♦❢ t❤❡ q✉❡st✐♦♥ s❡t✳ ❚❤❡ r❡s✉❧ts ❛r❡ s❤♦✇♥ ✐♥ ❚❛❜❧❡ ✷✳ ❲❡ ✜♥❞ t❤❛t ♣❛rt✐❝✐♣❛♥ts ✐♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ✇✐t❤ ❜❡tt❡r ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s ❛♥❞ ❜❡tt❡r ♥✉♠❡r❛❝② s❦✐❧❧s ❛❝❤✐❡✈❡ ❧❛r❣❡r ❤✐t r❛t❡s ❛♥❞ t❤❡r❡❢♦r❡ ❛♣♣❡❛r s✐❣♥✐✜❝❛♥t❧② ❧❡ss ♦✈❡r❝♦♥✜✲ ❞❡♥t ✭p < 0.05✮✳ ❚❤✐s ✐s ❛♥ ✐♥t❡r❡st✐♥❣ ✜♥❞✐♥❣ ❛s ✐t ♠✐❣❤t ❡①♣❧❛✐♥ s♦♠❡ ♣✉③③❧✐♥❣ r❡s✉❧ts ✐♥ ✇❤✐❝❤ ❡①♣❡r✐♠❡♥t❡rs tr② t♦ ❧✐♥❦ ♦✈❡r❝♦♥✜❞❡♥❝❡ t♦ ✐♥❞✐✈✐❞✉❛❧ ❡❝♦♥♦♠✐❝ ❜❡❤❛✈✐♦r✱ ❡✳❣✳ t❤❡ ✜♥❞✐♥❣ t❤❛t ♦✈❡r❝♦♥✜❞❡♥t ✐♥✈❡st♦rs ❛r❡ ❧❡ss ♣r♦✜t❛❜❧❡ ❜✉t ❞♦ ♥♦t tr❛❞❡ ♠♦r❡✳ ■♥ ❧✐♥❡ ✇✐t❤ t❤❡ r❡s✉❧ts ❛❜♦✈❡ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❞♦❡s ♥♦t ❛✛❡❝t ❤✐t r❛t❡s✳ ■♥t❡r❡st✐♥❣❧②✱ ✇❡ ✜♥❞ ♥♦ ❣❡♥❞❡r ❡✛❡❝ts ❜✉t ❛ s✐❣♥✐✜❝❛♥t ❤❛r❞✲❡❛s② ❡✛❡❝t ✐♥❞✐❝❛t✐♥❣ ❛ ♥❡❣❛t✐✈❡ r❡❧❛t✐♦♥ ❜❡t✇❡❡♥ ❤✐t r❛t❡s ❛♥❞ t❤❡ ♣❡r❝❡✐✈❡❞ ❞✐✣❝✉❧t② ♦❢ t❤❡ q✉❡st✐♦♥ s❡t ✭p < 0.05✮✳ ❋✐♥❛❧❧②✱ t❤❡ r❡❣r❡ss✐♦♥ r❡s✉❧ts s✉❣❣❡st t❤❛t ❛ ❧❛r❣❡r ❞❡❣r❡❡ ♦❢ r✐s❦ ❛✈❡rs✐♦♥ ✐s ❛ss♦❝✐❛t❡❞ ✇✐t❤ ❧♦✇❡r ❤✐t r❛t❡s ✭p <0.05✮✳ ❚❤❡ r❡❣r❡ss✐♦♥ r❡s✉❧ts ✐♥ t❤❡ st✉❞❡♥t s❛♠♣❧❡ ❞✐✛❡r ✐♥ t✇♦ ♠❛✐♥ ❛s♣❡❝ts✳ ❋✐rst✱ ✇❡ ✜♥❞ t❤❛t ♥♦♥❡ ♦❢ t❤❡ ❝♦❡✣❝✐❡♥ts ♦❢ t❤❡ ❜❡❤❛✈✐♦r❛❧ ❝❤❛r❛❝t❡r✐st✐❝s ✐s s✐❣♥✐✜❝❛♥t✳ ❆ ♣♦t❡♥t✐❛❧ ✾
Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 r❡❛s♦♥ ❢♦r t❤❡ ✐♥s✐❣♥✐✜❝❛♥t ❝♦❡✣❝✐❡♥ts ♦❢ t❤❡ ❈❘❚ s❝♦r❡ ❛♥❞ t❤❡ ❝♦❡✣❝✐❡♥t ♦❢ r✐s❦ ❛✈❡rs✐♦♥ ✐s t❤❡ ❧❛❝❦ ♦❢ ✈❛r✐❛t✐♦♥ ✐♥ t❤❡s❡ ♠❡❛s✉r❡s✳ ■♥ t❤❡ st✉❞❡♥t s❛♠♣❧❡ ♠♦r❡ t❤❛♥ ✼✾✪ ✭✹✾✪✮ ♦❢ t❤❡ st✉❞❡♥ts ❛r❡ ❛❜❧❡ t♦ ❝♦rr❡❝t❧② ❛♥s✇❡r ❛t ❧❡❛st t✇♦ ✭❛❧❧✮ ♦❢ t❤❡ q✉❡st✐♦♥s ✐♥ t❤❡ ❈❘❚ ❝♦♠♣❛r❡❞ t♦ ✻✷✪ ✭✸✶✪✮ ✐♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡✳ ▲✐❦❡✇✐s❡✱ t❤❡ ✈❛r✐❛t✐♦♥ ✐♥ r✐s❦ ❛✈❡rs✐♦♥ ✐s s♠❛❧❧❡r ✐♥ t❤❡ st✉❞❡♥t s❛♠♣❧❡ ✭sd= 0.85✮ t❤❛♥ ✐♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ✭sd= 1.19✮✳ P♦ss✐❜❧② ♠♦r❡ s♦♣❤✐st✐❝❛t❡❞ t❡sts ✇♦✉❧❞ ❤❛✈❡ ❧❡❞ t♦ ♠♦r❡ ♣r♦♥♦✉♥❝❡❞ r❡s✉❧ts ❛♥❞ ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s ♠✐❣❤t st✐❧❧ ❜❡ ❧✐♥❦❡❞ ✇✐t❤ ✐♥t❡r✈❛❧ ✇✐❞t❤s✳ ❚❤❡ ❧❛❝❦ ♦❢ ✈❛r✐❛t✐♦♥ ❝❛♥♥♦t ❡①♣❧❛✐♥ ❞✐✛❡r❡♥t r❡s✉❧ts ✐♥ t❤❡ ❇◆❚ s❝♦r❡ ❝♦❡✣❝✐❡♥t✳ ❇✉t s✐♥❝❡ st✉❞❡♥ts t❛❦❡ s✐❣♥✐✜❝❛♥t❧② ♠♦r❡ t✐♠❡ t♦ ✜♥✐s❤ t❤❡ s✉r✈❡② ♦♥❡ ♠✐❣❤t ❛r❣✉❡ t❤❛t t❤❡ ❛❜✐❧✐t② ♦❢ t❤❡ ❈❘❚ ❛♥❞ ❇◆❚ t♦ ❞✐s❝r✐♠✐♥❛t❡ t❤❡ st✉❞❡♥ts ❜② t❤❡✐r ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s ✐s ❧✐♠✐t❡❞✳ ❚❤❡ s❡❝♦♥❞ ❞✐✛❡r❡♥❝❡ ✐s t❤❛t st✉❞❡♥ts ❛r❡ s❧✐❣❤t❧② ♠♦r❡ s❡♥s✐t✐✈❡ t♦ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ t❤❛♥ ▼❚✉r❦ ♣❛rt✐❝✐♣❛♥ts✳ ❚❤❡ ❝♦❡✣❝✐❡♥t ♦❢ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ✐s s✐❣♥✐✜❝❛♥t ✭✐♥ ♦♥❡ s♣❡❝✐✜❝❛t✐♦♥ ❛t t❤❡ ✶✪ ❧❡✈❡❧✮✳ ◆❡✈❡rt❤❡❧❡ss✱ t❤❡ s✐③❡ ♦❢ t❤❡ ❡✛❡❝t ✐s ✐♥ ❧✐♥❡ ✇✐t❤ t❤❡ ❧✐♠✐t❡❞ s❡♥s✐t✐✈✐t② ❢♦✉♥❞ ✐♥ t❤❡ ❤✐t r❛t❡ r❡s✉❧ts ♦❢ t❤❡ st✉❞❡♥t s❛♠♣❧❡ ✭s❡❡ ❆♣♣❡♥❞✐① ❆✮✳ ❆❧t♦❣❡t❤❡r✱ t❤❡ r❡❣r❡ss✐♦♥ r❡s✉❧ts ❛r❡ ♠✐①❡❞✱ ❜✉t s✉❣❣❡st t❤❛t t❤❡ ♣r❡❝✐s✐♦♥ ♦❢ t❤❡ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡ ♠✐❣❤t ❜❡ ✐♠♣❛✐r❡❞ ❜② ❛❞❞✐t✐♦♥❛❧ ✐♥❞✐✈✐❞✉❛❧ ❝❤❛r❛❝t❡r✐st✐❝s s✉❝❤ ❛s ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s ❛♥❞ r✐s❦ ♣r❡❢❡r❡♥❝❡s✳ 0.0 2.5 5.0 7.5 10.0 30% 60% 90%Requested Confidence Level (Treatment)
Frequency J udgments MTurk Students Benchmark ❋✐❣✉r❡ ✷✿ ❆✈❡r❛❣❡ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ✇✐t❤ st❛♥❞❛r❞ ❡rr♦r ❜❛rs ❣r♦✉♣❡❞ ❜② tr❡❛t♠❡♥t✳ ❚❤❡ ❞❛r❦ ❣r❡② ❜❛rs ✐♥❞✐❝❛t❡ t❤❡ r❡s♣❡❝t✐✈❡ ❜❡♥❝❤♠❛r❦ ❧❡✈❡❧s ♦❢ ❝♦♥✜❞❡♥❝❡ t❤❛t ✐s ❛ss✉♠❡❞ ❜② t❤❡ ■P✳
✸✳✷ ❋r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡
❲❡ ✈❡r② ❝❧❡❛r❧② ✐♥str✉❝t❡❞ ♣❛rt✐❝✐♣❛♥ts t❤❛t t❤❡② s❤♦✉❧❞ ❡①♣❡❝t t♦ ❛❝❤✐❡✈❡ ❛ ❤✐t r❛t❡ t❤❛t ❡q✉❛❧s t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ◆❡✈❡rt❤❡❧❡ss✱ ♦♥❧② ✶✷✳✸✪ ♦❢ t❤❡ ♣❛rt✐❝✐♣❛♥ts✬ ✶✵Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ❉✐str✐❜✉t✐♦♥ ♦❢ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❇✐♥s✿ ✵ ✶ ✷ ✸ ✹ ✺ ✻ ✼ ✽ ✾ ✶✵ ❚♦t❛❧ P❛♥❡❧ ❆✿ ▼❚✉r❦ s❛♠♣❧❡ ❚r❡❛t♠❡♥ts✿ ✸✵✪✻✵✪ ✸✷ ✵✶ ✼✺ ✻✺ ✺✺ ✶✸✻ ✸✹ ✹✺ ✹✸ ✶✷ ✶✸ ✹✹✹✹ ✾✵✪ ✶ ✸ ✹ ✻ ✹ ✽ ✺ ✷ ✽ ✽ ✶ ✺✵ P❛♥❡❧ ❇✿ ❙t✉❞❡♥t s❛♠♣❧❡ ❚r❡❛t♠❡♥ts✿ ✸✵✪✻✵✪ ✵✵ ✵✵ ✶✸ ✶✷✷ ✹✷ ✹✽ ✶✽✹ ✶✵✹ ✼✻ ✵✵ ✶✵ ✹✺✹✶ ✾✵✪ ✷ ✵ ✶ ✷ ✷ ✶✵ ✻ ✶✷ ✼ ✽ ✶ ✺✶ ❚❛❜❧❡ ✸✿ ❚❤✐s t❛❜❧❡ ❧✐sts ❢♦r ❛❧❧ ♣♦ss✐❜❧❡ ✈❛❧✉❡s ♦❢ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts t❤❡ ♥✉♠❜❡r ♦❢ ♣❛rt✐❝✐♣❛♥ts t❤❛t ❝❤♦s❡ t❤❡ r❡s♣❡❝t✐✈❡ ✈❛❧✉❡✳ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ✇❡r❡ ❡①❛❝t❧② ✐♥ ❧✐♥❡ ✇✐t❤ t❤❡ ✐♥str✉❝t✐♦♥s✳ ■♥ ❛❧❧ tr❡❛t♠❡♥ts✱ t❤❡ ❛✈❡r❛❣❡ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❛r❡ ✈❡r② ❝❧♦s❡ t♦ ✜✈❡ ❛♥❞ r❛♥❣❡ ♦♥❧② ❢r♦♠ ✹✳✼✸ ✐♥ t❤❡ ✻✵✪ tr❡❛t♠❡♥t t♦ ✺✳✺✵ ✐♥ t❤❡ ✾✵✪ tr❡❛t♠❡♥t ✭s❡❡ ❋✐❣✉r❡ ✷✮✳ ❯s✐♥❣ ❛ ❑r✉s❦❛❧✲❲❛❧❧✐s t❡st ✇❡ ❝❛♥✲ ♥♦t r❡❥❡❝t t❤❡ ♥✉❧❧ ❤②♣♦t❤❡s✐s t❤❛t t❤❡ ❧♦❝❛t✐♦♥ ♣❛r❛♠❡t❡rs ♦❢ t❤❡ ❞✐str✐❜✉t✐♦♥s ♦❢ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ✐♥ t❤❡ t❤r❡❡ tr❡❛t♠❡♥ts ❛r❡ t❤❡ s❛♠❡✳ ▼♦r❡♦✈❡r✱ ✇❡ ✜♥❞ ❧❛r❣❡ ❤❡t❡r♦❣❡♥❡✐t② ✐♥ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❜❡t✇❡❡♥ s✉❜❥❡❝ts ✐♥ ❛❧❧ tr❡❛t♠❡♥ts ❝♦✈❡r✐♥❣ t❤❡ ✇❤♦❧❡ r❛♥❣❡ ❢r♦♠ ③❡r♦ t♦ t❡♥ ✭s❡❡ ❚❛❜❧❡ ✸✮✳ ❚❤❡ st❛♥❞❛r❞ ❞❡✈✐❛t✐♦♥ ✐s ❜❡t✇❡❡♥ ✷✳✸✶ ❛♥❞ ✷✳✼✹✳ ■♥ ❧✐♥❡ ✇✐t❤ t❤✐s✱ ❥✉st ✸✳✻✪ ♦❢ t❤❡ ♣❛rt✐❝✐♣❛♥ts ♠❡♥t✐♦♥ t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ✐♥ t❤❡ ❢r❡❡ t❡①t ❡①♣❧❛♥❛t✐♦♥s ♦❢ ❤♦✇ t❤❡② ❝♦♥str✉❝t❡❞ t❤❡ r❛♥❣❡ ❡st✐♠❛t❡s✳ ❙t✉❞❡♥ts ❛♣♣❡❛r s❧✐❣❤t❧② ♠♦r❡ s❡♥s✐t✐✈❡ t♦ t❤❡ ✐♥str✉❝t✐♦♥s t❤❛♥ ▼❚✉r❦ ✇♦r❦❡rs ✭s❡❡ ❋✐❣✉r❡ ✷✮✳ ❍♦✇❡✈❡r✱ t❤❡ ♠❛✐♥ r❡s✉❧ts ❛r❡ q✉❛❧✐t✐✈❡❧② ✐♥ ❧✐♥❡ ✇✐t❤ t❤❡ ▼❚✉r❦ s❛♠♣❧❡✿ ✇❡ ✜♥❞ ❧❛r❣❡ ✈❛r✐❛t✐♦♥s ✐♥ ❢r❡q✉❡♥❝② ❥✉❞❣✲ ♠❡♥ts ✐♥ ❛❧❧ tr❡❛t♠❡♥ts ❛♥❞ ♦♥❧② t❤❡ ❞✐✛❡r❡♥❝❡ ✐♥ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❜❡t✇❡❡♥ t❤❡ ✸✵✪ ❛♥❞ ✾✵✪ tr❡❛t♠❡♥t ✐s st❛t✐st✐❝❛❧❧② s✐❣♥✐✜❝❛♥t ✭s❡❡ ❆♣♣❡♥❞✐① ❆✳✷✮✳ ❚❤❡ ❧❛r❣❡ ❛♠♦✉♥t ♦❢ ❤❡t❡r♦❣❡♥❡✐t② ✐♥ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ✐♥❞✐❝❛t❡s t❤❛t t❤❡r❡ ❡①✐st s✉❜st❛♥t✐❛❧ ❞✐✛❡r❡♥❝❡s ✐♥ ❝♦♥✜❞❡♥❝❡ ❡✈❡♥ ❛♠♦♥❣ ♣❛rt✐❝✐♣❛♥ts ♦❢ t❤❡ s❛♠❡ tr❡❛t♠❡♥t✳ ❋✉rt❤❡r✱ t❤❡ ♠✐ss✐♥❣ tr❡❛t♠❡♥t ❡✛❡❝t s❤♦✇s t❤❛t t❤❡ r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧ ❤❛r❞❧② ❛✛❡❝ts ♣❡r❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧s✱ ✐❢ ❛t ❛❧❧✱ ❡✈❡♥ t❤♦✉❣❤ t❤❡ r❡❧❛t✐♦♥ ✐s ❡①♣❧✐❝✐t❧② str❡ss❡❞ ✐♥ t❤❡ ✐♥str✉❝t✐♦♥s✳ ❚❤✐s ✐s ✐♥ st❛r❦ ❝♦♥tr❛st t♦ t❤❡ ✐♠♣❧✐❝✐t ❛ss✉♠♣t✐♦♥ ♦❢ t❤❡ ■P✳ ■♥ t❤❡ ❢♦❧❧♦✇✐♥❣✱ ✇❡ ❝❛❧❝✉❧❛t❡ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❜❛s❡❞ ♦♥ ♣❡r❝❡✐✈❡❞ ❝♦♥✜❞❡♥❝❡ ✭❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡✮ ❛s t❤❡ ❞✐✛❡r❡♥❝❡ ❜❡t✇❡❡♥ r❡❧❛t✐✈❡ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❛♥❞ ❛❝❤✐❡✈❡❞ ❤✐t r❛t❡s ❢♦r ❡❛❝❤ ♣❛rt✐❝✐♣❛♥t✳ ❚❤❡ r❡s✉❧t✐♥❣ ❞✐str✐❜✉t✐♦♥s ♦❢ ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛r❡ ✐❧❧✉s✲ tr❛t❡❞ ❛s ❜♦①♣❧♦ts ✐♥ ❋✐❣✉r❡ ✸✳ ■♥ ❧✐♥❡ ✇✐t❤ ♣r❡✈✐♦✉s ✜♥❞✐♥❣s ✭❈❡s❛r✐♥✐ ❡t ❛❧✳✱ ✷✵✵✻❀ ❲✐♥♠❛♥ ❡t ❛❧✳✱ ✷✵✵✹✮✱ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ②✐❡❧❞ ♠♦❞❡r❛t❡ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ ❚❤❡ ❛✈❡r❛❣❡ ❧❡✈❡❧ ♦❢ ❢r❡✲ q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ✐s ♥❡❛r❧② ✐❞❡♥t✐❝❛❧ ✐♥ ❛❧❧ tr❡❛t♠❡♥ts ❛t r♦✉❣❤❧② ✶✻✪✳ ■♥ ❝♦♥tr❛st t♦ ✶✶Accepted Manuscript
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 t❤❡ ■P ♠❡❛s✉r❡✱ t❤❡ ❞✐✛❡r❡♥❝❡s ✐♥ ♦✈❡r❝♦♥✜❞❡♥❝❡ ❜❛s❡❞ ♦♥ ❢r❡q✉❡♥❝② ❥✉❞❣♠❡♥ts ❛r❡ ♥♦t st❛✲ t✐st✐❝❛❧❧② s✐❣♥✐✜❝❛♥t ❜❡t✇❡❡♥ t❤❡ t❤r❡❡ tr❡❛t♠❡♥ts ✐♥ ❛ ❑r✉s❦❛❧✲❲❛❧❧✐s t❡st✳ ❈♦♥s❡q✉❡♥t❧②✱ ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ t✉r♥s ♦✉t t♦ ❜❡ ❝♦❤❡r❡♥t t❤r♦✉❣❤♦✉t ❛❧❧ tr❡❛t♠❡♥ts✳ ❆♥❛❧♦❣♦✉s −1.0 −0.5 0.0 0.5 1.0 30% 60% 90%Requested Confidence Level (Treatment)
Frequency Ov erconfidence MTurk Students ❋✐❣✉r❡ ✸✿ ❇♦①♣❧♦t ♦❢ ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ❣r♦✉♣❡❞ ❜② tr❡❛t♠❡♥ts✳ ❚❤❡ ❡❞❣❡s ♦❢ t❤❡ ❜♦①❡s ✐♥❞✐❝❛t❡ t❤❡ ✜rst ❛♥❞ t❤✐r❞ q✉❛rt✐❧❡✳ ❚❤❡ ✉♣♣❡r ❛♥❞ ❧♦✇❡r ✇❤✐s❦❡rs ❡①t❡♥❞ ❢r♦♠ t❤❡ ❤✐♥❣❡ t♦ t❤❡ ❤✐❣❤❡st ❛♥❞ ❧♦✇❡st ✈❛❧✉❡ t❤❛t ✐s ✇✐t❤✐♥ ✶✳✺ t✐♠❡s ♦❢ t❤❡ ✐♥t❡r q✉❛rt✐❧❡ r❛♥❣❡ ♦❢ t❤❡ ❤✐♥❣❡✱ r❡s♣❡❝t✐✈❡❧②✳ t♦ t❤❡ ❛♥❛❧②s✐s ♦❢ t❤❡ ■P ♠❡❛s✉r❡ ✇❡ r✉♥ r❡❣r❡ss✐♦♥s t♦ t❡st ✐❢ ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ✐s ❛ss♦❝✐❛t❡❞ ✇✐t❤ ✐♥❞✐✈✐❞✉❛❧ ❝❤❛r❛❝t❡r✐st✐❝s✳ ❚❤❡ r❡❣r❡ss✐♦♥ r❡s✉❧ts ✭s❡❡ ❚❛❜❧❡ ✹✮ ❝♦♥✜r♠ t❤❛t ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ✐s ♥♦t ❛✛❡❝t❡❞ ❜② r❡q✉❡st❡❞ ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ■♥ ❝♦♥tr❛st t♦ t❤❡ ■P ♠❡❛s✉r❡✱ ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ❞♦❡s ♥♦t ❝♦rr❡❧❛t❡ ✇✐t❤ ❝♦❣♥✐t✐✈❡ ❛❜✐❧✐t✐❡s✳ ■♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡ ❧❛r❣❡r ❞❡❣r❡❡s ♦❢ ❢r❡q✉❡♥❝② ♦✈❡r❝♦♥✜❞❡♥❝❡ ❛r❡ ♣♦s✐t✐✈❡❧② ❛ss♦❝✐❛t❡❞ ✇✐t❤ ✇✐❞❡✲✐♥t❡r✈❛❧ ❛✈❡rs✐♦♥✳ ▼♦r❡♦✈❡r✱ ✇❡ ✜♥❞ s♦♠❡ ❡✈✐❞❡♥❝❡ t❤❛t ♠❛❧❡ ♣❛rt✐❝✐♣❛♥ts ❛r❡ ♠♦r❡ ♦✈❡r❝♦♥✜❞❡♥t✳ ❙✐♠✐❧❛r t♦ t❤❡ ❤✐t r❛t❡ r❡❣r❡ss✐♦♥s ✇❡ ✜♥❞ ♥♦ s✐❣♥✐✜❝❛♥t ❝♦❡✣❝✐❡♥ts ✐♥ t❤❡ st✉❞❡♥t s❛♠♣❧❡ r❡❣r❡ss✐♦♥s ✭s❡❡ P❛♥❡❧ ❇ ♦❢ ❚❛❜❧❡ ✹✮✳ ■t r❡♠❛✐♥s t❤❡ q✉❡st✐♦♥ t♦ ✇❤❛t ❡①t❡♥t t❤❡ t✇♦ ♠❡❛s✉r❡s ❛r❡ ❝♦♥s✐st❡♥t ✐♥ t❤❡✐r r❛♥❦✲ ✐♥❣ ♦❢ ✐♥❞✐✈✐❞✉❛❧ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ ❆ ♣r✐♦r✐ ♦♥❡ s❤♦✉❧❞ ❡①♣❡❝t t❤❛t t❤❡ t✇♦ ♠❡❛s✉r❡s ❛r❡ ♣❡r❢❡❝t❧② ❝♦♥s✐st❡♥t s✐♥❝❡ ❜♦t❤ ❛r❡ ❞❡s✐❣♥❡❞ t♦ ♠❡❛s✉r❡ ✐♥❞✐✈✐❞✉❛❧ ♦✈❡r❝♦♥✜❞❡♥❝❡✳ ❚❤❡ ♦❜✲ s❡r✈❡❞ ❤❡t❡r♦❣❡♥❡✐t② ✐♥ ✐♥❞✐✈✐❞✉❛❧ ❝♦♥✜❞❡♥❝❡✱ ❤♦✇❡✈❡r✱ s✉❣❣❡sts t❤❛t t❤❡ ■P ♠❡❛s✉r❡ r❛♥❦s ♣❡♦♣❧❡ ❞✐✛❡r❡♥t❧② ❜❡❝❛✉s❡ ✐t ✐s ❜❛s❡❞ ♦♥ ❛ ❤♦♠♦❣❡♥❡♦✉s ❝♦♥✜❞❡♥❝❡ ❧❡✈❡❧✳ ❆❞❞✐t✐♦♥❛❧❧②✱ t❤❡ ❞✐✛❡r❡♥❝❡s ✐♥ t❤❡ r❡❣r❡ss✐♦♥ r❡s✉❧ts ✐♥❞✐❝❛t❡ ❧✐♠✐ts ♦❢ t❤❡ ❝♦♥s✐st❡♥❝② ❜❡t✇❡❡♥ t❤❡ t✇♦ ♠❡❛s✉r❡s✳ ❚♦ q✉❛♥t✐❢② ❛♥❞ ❛♥❛❧②③❡ t❤❡ ❝♦♥s✐st❡♥❝② ♦❢ t❤❡ t✇♦ ♠❡❛s✉r❡s ✇❡ ❝♦♠♣✉t❡ t❤❡ r❛♥❦ ❝♦rr❡❧❛t✐♦♥ ❝♦❡✣❝✐❡♥t ❑❡♥❞❛❧❧✬s τb✳ ❲❡ ❞♦ t❤❡ ❝❛❧❝✉❧❛t✐♦♥ ❢♦r ❡❛❝❤ tr❡❛t♠❡♥t s❡♣❛✲ r❛t❡❧② ❞✉❡ t♦ t❤❡ tr❡❛t♠❡♥t ❡✛❡❝t ♦♥ t❤❡ ■P ♦✈❡r❝♦♥✜❞❡♥❝❡ ♠❡❛s✉r❡✳ ■♥ t❤❡ ▼❚✉r❦ s❛♠♣❧❡✱ ✇❡ ✜♥❞ t❤❛t t❤❡ ❝♦rr❡❧❛t✐♦♥ r❛♥❣❡s ❜❡t✇❡❡♥ ✵✳✸✵ ✭✸✵✪ ❛♥❞ ✾✵✪ tr❡❛t♠❡♥t✮ t♦ ✵✳✹✷ ✭✻✵✪ ✶✷
