Chapter 3: Question format as an implicit source of goal emphasis across cognitive domains
3.3.5. Proportion confidence
Decision A confidence measures: HighAprop, MedAprop, LowAprop; Decision B
confidence measures: HighBprop, MedBprop, LowBprop). To reiterate, confidence was assumed to serve as an index of evidence strength, such that decisions rendered with low confidence were assumed to reflect heightened uncertainty. Following on from the counter-emphasis bias described in the previous chapter (i.e. the reduced likelihood of endorsing decisions emphasised by the question; see Section 2.3.), it was expected that question emphasis would serve to reduce the proportion of those emphasised decisions specifically given low confidence ratings (and associated with greater uncertainty). Hence, a series of 2(decision type: Decision A and Decision B) x 2(question emphasis: “Decision A?” or “Decision B?”) repeated measures ANOVAs were conducted on the proportion of high, medium and low confidence decisions, separately for each of the four tasks. Means and standard deviations for these proportion confidence analyses are provided in Table 3.3.
For the analyses of the proportion of high confidence decisions (HighAprop and HighBprop), the main effect of decision type was significant in the temporal and recognition tasks, F(1,20) = 4.88, p = .039, ηp2 = .20, and F(1,27) = 5.22, p = .030,
ηp2 = .16 respectively. The proportion of high confidence ratings was higher for
“second half” decisions (M = .46, SD = .22) compared to “first half” decisions (M = .40, SD = .22), and for “old” decisions (M = .38, SD = .18) compared to “new”
decisions (M = .31, SD = .18). The main effects of decision type did not generalize across the two tasks, with non-significant effects observed in the semantic and colour tasks, both Fs(1,31) < 1. The main effects of question emphasis were non- significant in the semantic, colour, temporal and recognition tasks, F(1,31) < 1,
138
F(1,31) < 1, F(1,20) < 1, and F(1,27) = 1.21, p = .281, ηp2 = .04 respectively.
Crucially, the decision type x question emphasis interactions were also non- significant across the four tasks, F(1,31) = 2.06, p = .162, ηp2 = .06, F(1,31) < 1,
F(1,20) < 1, and F(1,27) = 2.91, p = .100, ηp2 = .10 respectively. As before,
interactions of decision type and question emphasis served as the primary index of counter-emphasis effects in the proportion confidence analyses. Hence, the
complete absence of such interactions for the analyses of the high confidence decision proportions across tasks is consistent with the inferred high evidence
strength associated with these decisions. High evidence strength ensures sufficiently accurate responding that precludes the need to engage controlled strategies, such as heightening strategic caution towards the rendering of emphasised decisions. A similar pattern of results was observed for the analyses of the proportion of medium confidence decisions (MedAprop and MedBprop). The 2(decision type) x
2(question emphasis) ANOVAs revealed non-significant main effects of decision type across the semantic, colour, temporal and recognition tasks, F(1,30) = 2.74, p = .108, ηp2 = .08, F(1,31) < 1, F(1,20) < 1, and F(1,27) < 1 respectively. The main
effects of question emphasis across these four tasks were also non-significant,
F(1,30) < 1, F(1,31) < 1, F(1,20) < 1, and F(1,27) = 3.53, p = .071, ηp2 = .12. Finally,
the interaction of decision type and question emphasis was non-significant in the semantic, colour and recognition tasks, F(1,30) < 1, F(1,31) < 1 and F(1,27) < 1 respectively. A significant interaction was observed in the temporal task, although in a converse direction to that expected of the counter-emphasis bias i.e. question emphasis increased the proportion of emphasised decision endorsements at
medium confidence (see Table 3.3. for means), F(1,20) = 23.20, p < .001, ηp2 = .54.
139 significant for both “first half” and “second half” decisions, t(20) = 4.43, p < .001, d = 0.98, and t(20) = 4.46, p < .001, d = 1.01 respectively. Overall, the proportion of medium confidence decisions did not display consistent counter-emphasis shifts across tasks. Rather, the converse “pro-emphasis” shift evidenced in the temporal task highlights the virtue of segregating performance by confidence level and inferred evidence strength, as converse behavioural trends might be present at different confidence levels.
Unlike the analyses of high and medium proportion confidence measures (and those involving the earlier “primary” response measures), the analyses of the proportion of low confidence decisions yielded a reliable cross-domain effect in the anticipated goal emphasis direction. The 2(decision type) x 2(question emphasis) ANOVAs revealed main effects of decision type in the semantic, temporal and recognition tasks, F(1,30) = 9.42, p = .005, ηp2 = .24, F(1,20) = 5.34, p = .032, ηp2 = .21, and
F(1,27) = 6.98, p = .014, ηp2 = .21 respectively. Across these tasks, a greater
proportion of low confidence ratings were given for “disagreeable” decisions (M = .13, SD = .11) compared to “pleasant” decisions (M = .09, SD = .09), as well as for “first half” (M = .23, SD = .15) compared to “second half” decisions (M = .18, SD = .13), and for “new” (M = .31, SD = .22) compared to “old” decisions (M = .24, SD = .17). The main effect of decision type was non-significant in the colour task, F(1,28) < 1. These decision type main effects are putatively driven by domain-specific characteristics of the evidence strength under evaluation in each task. Further, the main effects of question emphasis were non-significant in the semantic, colour, temporal and recognition tasks, F(1,30) < 1, F(1,28) < 1, F(1,20) = 1.06, p = .317, ηp2
140 Crucially, the decision type by question emphasis interactions were significant in the semantic, temporal and recognition tasks, F(1,30) = 13.76, p = .001, ηp2 = .31,
F(1,20) = 17.50, p < .001, ηp2 = .47, and F(1,27) = 14.12, p = .001, ηp2 = .34
respectively. The interactions were in the same direction across these tasks,
reflecting “counter-emphasis” shifts in response proportion i.e. a reduced proportion of low confidence decisions made under question emphasis (see Figure 3.5.). Although the interaction was clearly non-significant in the colour task, the direction was nevertheless in the same counter-emphasis direction as the other three tasks,
F(1,28) = 1.48, p = ..233, ηp2 = .05. Pairwise t-tests demonstrated significant counter-
emphasis low confidence shifts for “disagreeable” but not “pleasant” decisions in the semantic task, t(30) = 2.60, p = .014, d = 0.48, and t(30) = 1.99, p = .055, d = 0.36 respectively; for both “first half” and “second half” decisions in the temporal task,
t(20) = 4.68, p < .001, d = 1.23, and t(20) = 3.00, p = .007, d = 0.70 respectively; for “new” but not “old” decisions in the recognition task, t(27) = 3.17, p = .004, d = 0.60, and t(20) = 1.85, p = .075, d = 0.35 respectively; and neither for “orange” nor “yellow”
decisions in the colour task, t(28) = 0.75, p = .459, d = 0.15, and t(28) = 1.34, p = .190, d = 0.26 respectively. The low confidence proportion results therefore bear out the prediction that strategic effects of question emphasis would manifest across cognitive domains exclusively in the presence of low evidence strength and heightened levels of uncertainty.
141
Table 3.3. Design key and descriptive statistics for cross-task analyses of proportion of endorsements of available decision categories (Decision A and Decision B) made at particular confidence levels, comprising the proportion of decisions made with high (HighAprop and
HighBprop), medium (MedAprop and MedBprop) and low confidence levels (LowAprop and LowBprop).
Note: M, mean; SD, standard deviation.
Task Semantic Colour Temporal Recognition
Question pleasant? disagreeable? orange? yellow? first half? second half? old? new? Emphasis Decision A Decision B Decision A Decision B Decision A Decision B Decision A Decision B
HighAprop M .63 .60 .77 .75 .40 .40 .40 .35 SD .18 .19 .18 .17 .22 .25 .20 .18 HighBprop M .60 .63 .75 .76 .47 .45 .30 .32 SD .20 .19 .16 .19 .25 .21 .19 .19 MedAprop M .31 .31 .17 .18 .43 .31 .37 .39 SD .12 .13 .13 .12 .15 .12 .15 .14 MedBprop M .27 .28 .17 .18 .30 .42 .35 .41 SD .13 .15 .11 .16 .13 .16 .11 .14 LowAprop M .08 .10 .07 .07 .16 .29 .22 .26 SD .11 .10 .08 .10 .12 .20 .18 .17 LowBprop M .15 .11 .09 .07 .23 .13 .35 .27 SD .13 .11 .09 .07 .18 .12 .23 .22
142
Figure 3.5. Question emphasis effects on the proportion of low confidence decisions made for decision categories across tasks: a. semantic task, b. colour task, c. temporal task and d.
143