I suggested that bias in judgement and decision making should be defined more precisely as a property of a given strategy or heuristic, which results in systematic deviation from
a normative model, holding across a range of contexts. In chapter two, I also discussed different normative models that have been used to make attributions of bias in the confirmation bias literature. I found that confusion around normative models in general,
and disagreement about the appropriate normative model, made it very unclear whether so-called demonstrations of confirmation bias showed a genuine bias (as opposed to a
bias-as-inclination or intuitive bias, as I distinguished at the end of the last section.) The question of what this ‘normative model’ is, and what justifies its status, is therefore
very important. So where do normative models in psychology come from?
4.3.1 The use of normative models in psychology
Baron (2012) overviews several different kinds of normative model used in different areas
of psychology, for different kinds of judgement and decision making tasks. In the simplest cases, the normative standard is simply the objectively correct answer - if someone is
using a heuristic to estimate a quantity (the population of a country, say), then we can judge whether their strategy is biased by looking at whether their estimates deviate from
the known correct answer. However, for most aspects of reasoning and decision making psychology is interested in, it’s much less straightforward than this - there’s no single,
known, ‘correct’ answer. If I’m trying to figure out what information is most likely to help reduce my uncertainty in making a decision, then the answer depends on what I
already know, the costs of obtaining different kinds of information, and how much time I have, among other things. The question of what inferences I should draw from new
information and how to update my beliefs relies similarly on what I already know, what different explanations there might be of the information, how reliable the source is, and
so on.
Rather than being able to judge what the ‘correct answer’ is in some objective sense,
what we want to do here is instead ask: what problem are people trying to solve here, and what’s the optimal solution to this problem? Sometimes, we can do this formally:
- providing us with formal normative models against which to judge human performance. For example, the problem of drawing inferences from new data is essentially a problem
in probability theory - given that I have observed data D, what probability should I put in my hypothesis H? The formal solution to this problem is given by Bayes’ theorem,
which provides a formal answer for how to calculate Pr(H |D) given some other relevant probabilities (more on this later.) We can then judge people’s inferences by how closely
they follow Bayes’ theorem.
A different but closely related approach to developing normative models in psychology
is to ask what minimal standards we think reasoning should meet, and then find ways to formalise those requirements. For example, certain logical principles formalise the
idea that beliefs should beconsistent in certain ways - I cannot believe both P and ¬P, and beliefs should be closed under implicature (if I believe A, and that A implies B, I
should also believe B.) More recently, normative standards for beliefs have moved away from logical principles and towards probability theory (Oaksford and Chater, 2007), on
the recognition that beliefs tend to be graded, not binary, and that we have to deal with uncertainty in most of the problems we are trying to solve. The basic rules of probability
theory formalise the requirement that people’s degrees of belief need to be consistent in certain ways so that they are not open to exploitation. The Dutch book theorem (De Finetti, 1964, Ramsey, 1926) says, for each of the laws of probability theory, that violating them would leave an agent open to making bets they cannot win - the converse
Dutch book theorem then says that human reasoning should follow these laws because doing so prevents people from taking self-defeating actions.
The risk, of course, with grounding normative models in minimal standards we expect reasoning to meet, is that this can start to slip into territory where people disagree about
what these minimal standards are, and start invoking intuitions about more ‘general reasoning principles’ as discussed above. The further we move from situations where
people are simply estimating known quantities, or there is a ‘correct answer’, the more difficult it becomes to agree how peopleshould reason, what the appropriate normative model is. We may also need to start factoring in other considerations such as what the individual’s goals are, what cognitive constraints they are operating under, and how
features of the environment affect what the optimal strategy is - complex issues which we will come to when discussing different notions of ‘rationality’ in the next section.
4.3.2 Normative models in the study of confirmation bias
We have said a bit more about how normative models in psychology might be justified, and where they come from - suggesting that most such models are based in attempts
to formalize mathematically the problem a person is trying to solve. More general principles such as Bayesian probability theory are justified in that they formalize the
most minimal requirements we think reasoning should fulfil, in particular, some notion of consistency and not being open to exploitation.
Disagreements about whether a strategy is biased might therefore arise from disagreements about what the correct normative model for a task is. A key example of this is the debate
around the correct normative interpretation of Wason’s hypothesis-testing tasks - with Wason originally using logical principles of inference as normative standards, and oth-
ers later arguing that the normative solution should instead be understood in terms of optimal data selection, grounded in probability theory (Austerweil and Griffiths, 2008,
Oaksford and Chater, 1994), as well as taking into account constraints based on the kinds of ‘real-life’ hypothesis testing tasks people face.
Focusing for now just on the ‘inference’ aspect of confirmation bias, Bayes’ rule then tells us how we should, ideally, update our beliefs in light of new evidence (see 4.1, 4.2, and
4.3 below for three equivalent formulations of Bayes’ theorem).2 The claim that people exhibit a confirmation bias (in inference) can therefore be understood more formally as
the claim that people update their beliefs more towards their current hypothesis (or less away from it) than Bayes’ theorem prescribes.
2
It can be shown that obeying the laws of Bayesian probability theory has lawful connections with accuracy - Leitgeb and Pettigrew (2010a,b), for instance, argue that for a suitable measure of accuracy, Bayesianism follows from the simple premise that an agent ought to approximate the truth.
P r(H|D) = P r(D|H)P r(H) P r(D) (4.1) P r(H |D) = P r(D|H)P r(H) P r(D|H)P r(H) +P r(D| ¬H)P r(¬H) (4.2) P r(H |D) P r(¬H|D) = P r(D|H) P r(D| ¬H) × P r(H) P r(¬H) (4.3)
However (as discussed earlier), many of the classic papers on confirmation bias do not refer to formal normative models at all - perhaps one of the main challenges for the
confirmation bias literature has been that it tends to focus on the kinds of beliefs which do not have ‘correct answers’ (as opposed to, say, the overconfidence literature), making
normative standards much more complex. Studies of confirmation bias have typically fallen into one of two categories to deal with this issue, as Eil and Rao (2011) point out.
Those in the first category use intentionally simple, abstract tasks (such as the 2-4-6 task or having people estimate the proportion of balls of a given colour in a bag), with
clear priors and objective signals that make the normative response easy to compute and compare participants’ responses to (Edwards, 1982, Wason, 1960). The downside
here is that these findings have unclear relevance to more ‘realistic’ situations. Studies in the second category, by contrast, focus on more ‘realistic’ beliefs, such as opinions on important issues (Lord et al., 1979, for example) - but it is very difficult to develop a
normative standard for comparison here, since we do not have an objective measure of participants’ prior beliefs, and the signals they receive from new information are often
ambiguous (i.e. they could be interpreted differently given different background assump- tions.) A few recent studies have attempted to bridge this gap, looking at how people
update their beliefs about their own intelligence or attractiveness (more relevant/im- portant beliefs than the number of balls in an urn!), given objective signals (ranking
relative to others on an IQ test, for example) (Eil and Rao, 2011, M¨obius et al., 2014).
Fischoff and Beyth-Marom (1983) document a number of different ways in which reasoning
might deviate from Bayes’ theorem - using the third form of Bayes’ rule, known as the odds ratio form (4.3). (From left to right, the components of this theorem are: the
true relative to alternative hypotheses, and the prior odds that H is true before observing the data.) We summarise the potential sources of bias that Fischoff and Beyth-Marom
(1983) list in table 4.1 below.3
Task Potential bias
Hypothesis formation Hypothesis is untestable, e.g. because it is ambiguous; Alternative hypotheses arepoorly defined
Assessing component probabilities
Misrepresentation: people may give the response that is expected of them rather than what they actually believe;
Incoherence: sometimes Pr(H) and Pr(¬H) may not equal one if not evaluated simultaneously, or if the beliefs themselves are not well thought-through;
Miscalibration: failure of ones confidence to correspond to reality - overconfidence, for example;
Nonconformity with expert judgements: due to reliance on availability or representativeness;
Objectivism
Assessing prior odds Poor survey of background: not treating the probabilities for different hypotheses equally;
Failure to assess: i.e. base rate neglect Assessing likelihood
ratio
Failure to assess;
Distortion by prior beliefs;
Neglect of alternative hypotheses: taking the current hypothesis as a given, treating it as definitely true
Aggregation Wrong rule: for example, averaging rather than multiplying the likelihood ratio and prior odds;
Misapplying right rule: e.g. making a computational error Information search Failure to search: perhaps due to premature conviction;
Nondiagnostic questions;
Inefficient search: particularly failure to ask potentially falsifying questions;
Unrepresentative sampling
Action Incomplete analysis: neglecting certain consequences, for example;
Forgetting critical value: confusing acting as if H were true (as a best guess) and actually believing H is true
Table 4.1: Ways reasoning can deviate from Bayes’ theorem - adapted from Fischoff
and Beyth-Marom (1983)
It could be helpful to ask, therefore, how these potential sources of bias correspond to the ways in which confirmation bias has been said to occur - and which of these potential
errors might result in a bias towards confirming the present hypothesis. For example, based on the potential errors in the table above, a confirmation bias could result in some
of the following ways:
• People may initially overestimate Pr(H) relative to Pr(¬H). This could happen for a number of reasons: the fact that people simply have difficulty translat-
ing their subjective beliefs into probability judgements; failing to consider enough 3
This is based on the table in the original paper - we have simply added a bit more detail to explain some of the potential sources of bias.
alternative hypotheses and so under-weighting their joint probability; or informa- tion supporting H may be more immediately accessible than the opposite.
• People may seek out data D such that Pr(D| H) is higher than Pr(D| ¬H): that is, seek out data that is more likely to support the hypothesis, and not account for this bias in search when calculating these probabilities.
• People may miscalculate Pr(Pr(DD|¬|HH)): either by neglecting to calculate the denom- inator entirely, or perhaps underestimating the relative value of the denominator because one struggles to think of relevant alternative hypotheses.
• In general, people may neglect ¬H, and so overestimate both items on the right hand side of the equation: either not realising that it is important to consider the alternative hypothesis at all, assuming that their current hypothesis is essentially true, or failing to bring to mind alternatives to the current hypothesis.
Research has done relatively little to relate confirmation bias to these kinds of more specific errors in applying Bayes’ rule - or to discuss what kinds of heuristics people
might be using to approximate Bayes’ rule that might lead to bias. We might get a clearer picture of confirmation bias by attempting to specify more clearly what these
heuristics might be, and to what extent they result in incorrectly estimating various important aspects of the equation, or whether they are estimating different quantities.
We can then ask whether these errors occur on average across all scenarios, and at what cost.