4.2.1 ‘Bias’ in different areas of research
The term ‘bias’ has been used to mean different things in different contexts (Hahn and
Harris, 2014, Klayman, 1995). In its everyday use, to be ‘biased’ generally means alack of impartiality - showing an undue preference for a particular alternative or perspective (as in racial bias or gender bias, for example). TheCambridge English Dictionary defines bias as, “the action of supporting or opposing a particular person or thing in an unfair
way, because of allowing personal opinions to influence your judgement.” Of course, what exactly makes a preference ‘unfair’ - and therefore, what makes it constitute a bias - is unclear. What would it mean, exactly, for one’s personal opinions not to influence one’s judgement, at all? What distinguishes a fair reason for supporting a particular person or alternative, from an unfair one? These issues suggest that even our intuitive
notion of bias is not entirely straightforward.
In some areas of social psychology, researchers have tended to refer to ‘biases’ in a similar way: as a tendency to express an unfair preference for a particular group or
idea, such as the ‘ingroup bias’ - tending to evaluate one’s ingroup more positively relative to an outgroup (Mullen et al., 1992). In other parts of psychology, ‘bias’ means
failing to conform togeneral principles of rationality - things like the ‘neutral evidence principle’, which says that evidence which is neutral (equally supportive of a hypothesis
and its negation) should not change one’s beliefs in one direction or another. This understanding of bias underlies the claim that people evaluate and assimilate evidence
in a ‘biased’ manner (Lord et al., 1979, for example) - because they strengthen their beliefs on the basis of apparently neutral evidence.
In cognitive psychology (particularly work on ‘heuristics and biases’), ‘bias’ is defined more precisely, as a systematic deviation from some normative model, such as proba-
bility or decision theory. For example, ‘base rate neglect’ is said to be a bias because people give less weight to base rates in estimating probabilities than Bayes’ theorem
would prescribe (Tversky and Kahneman, 1974). The difference between this and the previous notion is that ‘bias’ is understood as a deviation from some formal theory - rather than ‘general principles of rationality’ which are based largely on intuition. One challenge for the more intuitively-based notion of bias is that different people sometimes
have differing intuitions about what is rational, and these intuitions also sometimes conflict with normative models. This suggests that we should not blindly trust what
‘sounds reasonable’ intuitively; the standards against which bias is measured need more thorough, perhaps formal, justification.
It is worth clarifying several points about this more precise notion of bias, and how it differs from the intuitive or social psychological notion of bias. First,systematic deviation means that to constitute a bias, the same patterns of error must occur repeatedly, across individuals and some range of different scenarios. This means that bias is a
property we attribute to a heuristic, or some kind of decision-making strategy - not to an individual judgement or decision. Second, bias should be distinguished here from
noise - if judgements are noisy, they might deviate on average from some normative model, but in a random rather than systematic way. Finally, bias is defined relative
to some normative model - so what that normative model is, and what justifies its normative status, is crucial (something I will discuss in more detail later).
This is much closer to how ‘bias’ is understood in statistics, as a property of anestimator: some formula or strategy for estimating an unknown quantity. An estimator is said to
be biased if, on average, it shows a systematic pattern of errors from the ‘correct answer’ - the value it is trying to estimate. Again, this means bias is not something that can be
applied to a single estimate - bias is a property of a procedure for estimating something, so can only be identified when the output of that procedure is observed repeatedly.
If we think of heuristics used in reasoning as ‘estimators’ (shortcuts for making judge- ments and decisions given cognitive constraints), then to claim that a bias exists is to
say that a given heuristic deviates, on average and systematically, from some normative model. So if we want to claim that a bias arises in human judgement or decision making,
we need to be able to specify (a) what the heuristic is that produces the bias, and (b) what the heuristic is trying ‘estimate’ - what the optimal solution to the problem would
be. While these features are present in some discussion of bias in the psychology litera- ture - most notably the heuristics and biases program (Tversky and Kahneman, 1974) -
this is far from the standard way of talking about bias. In the literature on confirmation bias in particular, there is often little discussion of what the normative standards are
for different tasks, and what heuristics people might be using that result in the claimed biases. 1
There are two more nuances in this discussion of what it means to be biased worth men- tioning. First, bias does not always necessarily come at a cost to accuracy. Sometimes a more biased strategy will be more accurate than a less biased one, if the second strategy
is very high variance. This is because of something known as the bias-variance tradeoff in statistics: for a given estimator, there’s generally a tradeoff between how biased it
is (how much it errs in one specific direction), and the variance of its estimate (how much it deviates from the actual value overall, regardless of direction.) If an estimator
sometimes errs in one direction and sometimes equally far in the opposite direction, its overall ‘bias’ might be close to zero - but it still makes large errors. By contrast, an
estimator might be more biased if it tends to err in the same direction systematically, but at the same time more accurate, if the errors aren’t too far off the actual value.
A nice analogy for understanding this is to imagine two types of darts player: a high variance player might not display any bias, sending darts all over the board, whereas a
1
Why use this definition of bias rather than a more intuitive notion? Being more precise about what it means to be biased allows us to draw clearer conclusions about what these biases mean - if we want to say things about what it means to reason well or poorly, we need some kind of clear standard against which to measure reasoning. Without any kind of precise definition of bias, discussion can become confusing - as different people have different understandings of exactly what the term means, and what its implications are.
highly biased player may well be more accurate if all their darts fall in the same place not too far from the bullseye (4.1). To see how this translates to bias in judgement:
consider two people attempting to estimate the probabilities of different events. One person consistently underestimates the likelihood of these events, by a similar amount
each time, and so we would say they were biased. A second person displays no bias on average but their estimates are very high variance - sometimes wildly underestimating,
other times wildly overestimating. We would very likely say the first person’s judgement is more accurate and prefer to trust their forecasts, despite the fact they are more biased.
Figure 4.1: An illustration of the bias-variance tradeoff
A second nuance in our understanding of bias arises when we ask what kind of ‘average’
we are talking about in defining bias as ‘deviation on average.’ If we take this to be the mean, then it’s possible for a strategy or estimator to follow a highly skewed pattern, but for the ‘average’ deviation to still be zero. For example, suppose the true quantity I am trying to estimate is zero, and the strategy I am using ‘undershoots’ 90% of the
time, but only slightly: falling around -0.1. The other 10% of the time, I overshoot: my estimates falling around 0.9. This strategy is technically ‘unbiased’ - but still, my
estimate undershoots far more often than it overshoots, and so we might be tempted to say that it is ‘biased’ towards under-estimating. Le Mens and Denrell (2011) show
that it’s possible for even Bayesian rational agents to end up systematically favouring one of two hypotheses if there is an asymmetry in the information they receive about
them (even if they are aware of this asymmetry in information.) This is because the distribution of judgements about the two hypotheses is highly skewed. When averaged
across all individuals, the distribution of estimates has mean zero (i.e. there is technically no bias when averaging across judgements), but still a large proportion of individual
judgements come out in favour of one hypothesis rather than the other. If what we care about is accuracy, calling this strategy unbiased seems a little strange, as does calling a
high-variance strategy unbiased.
Klayman also distinguishes between bias as a “systematically flawed judgement process
that is ultimately deleterious to the interests of the actor or society”, and a “moral middle ground... people may deviate systematically from theoretical standards, but may still
be behaving optimally when broader concerns are taken into account.” (Klayman, 1995, p.386) This raises the question of not just whether judgements are biased in the sense
of deviating from a normative model, but what the consequences of those biases are. I will return to this issue in our later discussion of rationality more broadly. For now,
following Hahn and Harris (2014) I assume that the main consequence we are interested in is whether a bias comes at a cost to accuracy - a heuristic or strategy is biased if it deviates systematically from some normative standard, and does so at a cost to accuracy.
4.2.2 ‘Bias’ in the confirmation bias literature
In the last section I distinguished three main meanings of the term ‘biased’:
1. A tendency to favourone response/choice over another (without normative or evaluative implications)
2. Failure to conform tointuitively-based principlesof rationality
These different meanings of ‘bias’ have been used in different places in the confirmation bias literature, underpinning some of the disagreement about whether confirmation
bias ‘really exists.’ For example, Snyder and Swann (1978), when discussing confir- matory strategies in social hypothesis testing, equate bias with the tendency to ask
more confirmatory than disconfirmatory questions - without explicitly justifying why this is irrational at all. The selective exposure literature (Hart et al., 2009) similarly
seems to eschew any explicit normative standards, instead assuming that reading ‘un- balanced’ articles is non-normative. Here, ‘bias’ is being interpreted in the first, most
simple, sense - as any tendency to favour one side over another.
Lord et al.’s (1979) study on biased assimilation and attitude polarization is a classic
example of attributing the second kind of bias - they do not discuss formal normative models, but claim that subjects’ behaviour deviates from the (intuitively compelling)
rule that two people with prior beliefs should not strengthen those beliefs in different di- rections after reading the same evidence. We see more formal, explicit definitions of what
it means to be biased and clearer normative models in the literature on pseudodiagnos- ticity (Crupi et al., 2009, Doherty et al., 1979), and in the literature on overconfidence,
where it is more common to compare judgements to actual ‘correct answers’ (Moore et al., 2015).
We can also understand many of the disagreements about confirmation bias in light of these different understandings of bias. In the hypothesis-testing literature in particu-
lar, disagreement about whether a positive-test strategy should be interpreted as a bias seems to be rooted in disagreement about what the correct normative model for the
situation is - with Wason’s original standard being that of falsification, and others ar- guing for more complex normative models based in probability theory (Austerweil and
Griffiths, 2008, Klayman and Ha, 1987, Oaksford and Chater, 1994). Jern et al. (2014) challenge the classical belief polarization findings, arguing that the ‘neutral evidence
principle’ that they are based on is not always normative, and providing a more formal analysis to show how responses might sometimes be considered in line with normative
prescriptions. There is some disagreement about what the ‘correct’ normative model is in the pseudodiagnosticity tasks (Crupi et al., 2009, Tweney et al., 2010). Le Mens and
Denrell (2011) argue that it’s possible for individuals’ beliefs to systematically favour one of two hypotheses even under purely ‘rational’ assumptions (i.e. their judgements are
predicted by Bayes’ theorem), because the distribution of judgements across all people can end up skewed but technically ‘unbiased’.
It’s also worth noting here that there’s a distinction between saying that a given judgement strategy is biased and saying it is biased in the specific way we are interested in - i.e. towards the focal or current hypothesis. Some of the discussion about whether a positive test strategy is really a confirmation bias (e.g. Klayman and Ha, 1987) is not necessarily
claiming that a positive test strategy does not lead to systematic errors - but simply that those errors do not necessarily always favour the focal hypothesis.
Disagreements about the term ‘bias’, if not made explicit, can therefore fuel a great deal of confusion. In particular, the main sources of disagreement seem to be the following:
• Whether ‘bias’ carries normative implications - or whether it simply describes a
tendency or inclination that may be overall harmless (related to Klayman’s 1995
distinction between bias as inclination and faulty judgement, which I discussed in chapter two);
• If ‘bias’ does carry normative implications, whether it should be judged relative
intuitive principles, or whether standards should be grounded in more formal nor- mative models;
• If a formal normative model, what the appropriate formal model actually is;
• What it means for a strategy to deviate ‘on average’ from that normative model.
To reduce some of this confusion, we could introduce some new terms: using ‘bias-as- inclination’ to describe ‘biases’ that may or may not be non-normative, and ‘intuitive
bias’ to refer to biases that deviate from intuitive principles of rationality but may not deviate from some more formal normative standards. In what follows, therefore, I will
use the term ‘bias’ to refer to a systematic deviation from a normative model. But this still leaves issues unresolved and room for possible disagreement - in particular, what justifies using a given normative model as the standard against which bias is judged,
and what do we do when different competing normative models are proposed? The next section will consider these questions in more detail.