Why does this matter? Rationality and improving human reasoning

One might naturally ask how much these different meanings of terms like ‘rational’ really

matter - isn’t this just pedantic quibbling over definitions? I think it does matter, firstly because it can help us to better understand many of the disagreements that arise in

the psychology literature about whether a given behaviour is ‘actually rational’ or not. Clarifying different uses of these terms can help us to clarify where disagreement is just

terminological confusion, and where the more substantive disagreement arises, as we have discussed. But I think it also matters for something even more important than

this: because clarifying whether a given behaviour is ‘biased’ or ‘irrational’, and what this really means, has implications for the possibility of improving human reasoning.

Given the complexities of the normative issues we’ve discussed so far, it’s not surprising that this question - ‘can, and should, we try to improve human reasoning?’ - does

not have a straightforward answer, and has generated a fair amount of debate. There are various different viewpoints on this issue, which arise from differing views on the

following questions:

• What is the appropriate normative standard against which to judge human

reasoning?

• Where and to what extent do people’s reasoning strategies fall short of this

standard?

• Is it possible, within the constraints of cognitive and biological capacity, for people

to use better strategies as judged by this standard?

These questions are closely related to the disagreements raised in the previous section

on rationality - whether or not one believes that it’s possible to improve reasoning will depend on whether one thinks there are standards people are failing to meet which are

realistically attainable. Another way of framing this, linking back to the earlier section, is to ask whether there are problems that people could solve better or more effectively

if they were to reason in different ways.

If we think of rationality too broadly, in terms of just the most minimal goals, and

then we are likely to arrive at the conclusion that people frequently fall short, but not feel particularly optimistic about people’s ability to do any better. For example,

Tversky and Kahneman (1974) take the view that reasoning falls short of these very broad normative standards, and Kahneman in particular seems sceptical that there is

any way to really ‘debias’ judgement against these errors, saying, at the end ofThinking Fast and Slow that, “little can be achieved without a considerable investment of effort... Except for some effects that I attribute mostly to age, my intuitive thinking is just as prone to overconfidence, extreme predictions, and the planning fallacy as it was before

I made a study of these issues.” (Kahneman, 2011, p.417)

By contrast, if we think of rationality too narrowly, then there is always some way that

we can frame people’s behaviour as rational, by specifying more and more constraints and assumptions people are operating under - and so the idea of improving human

reasoning becomes simply unnecessary or undesirable. For example, some evolutionary psychologists believe the appropriate standard against which to judge human rationality

is adaptiveness on the level of the genes (Tooby and Cosmides, 1992), and others argue that human reasoning can be viewed as basically optimal when we look at how well-

adapted it is in specific domains (Gigerenzer and Goldstein, 1996, Todd et al., 2000).

I think both of these perspectives go too far. If we construe rationality too broadly, we are

expecting too much of human reasoning - we cannot be expected to have general-purpose reasoning strategies that work well without error across a broad range of situations, given

the constraints we’re operating under and different scenarios we’re operating in. Though it’s interesting and useful to ask how we fare relative to this ‘broad’ notion of rationality,

I’m not sure this is the appropriate standard against which to assess the feasibility of improving human reasoning. But if we construe rationality too narrowly, then we may be

asking toolittleof reasoning: I think it’s highly unlikely that human reasoning is as well- adapted as some proponents of bounded or ecological rationality suggest. I think we do

want to judge human reasoning against standards other than purely evolutionary ones, for example. There are various different goals we might care about as individuals and as

part of larger groups, some which will be pretty much universally accepted, some which might conflict with one another (my short-term and long-term goals might conflict, or

my personal goals might conflict with those of a group I’m part of, for example). Given that each person has various different, often competing, goals in different scenarios, and that different people’s goals often come into conflict with one another, it seems

very unlikely that our natural judgement and decision-making processes are the best they could possibly be: that each of us couldn’t realistically achieve our personal goals

better, that different strategies couldn’t help us to better navigate tradeoffs and conflicts within groups of people. Especially given how many people in this world seem to be

unsatisfied, misinformed, or both, it seems to me hard to deny that there is room for reasoning to be improved in various ways, even given biological and cognitive constraints.

However, the task of improving human reasoning is much more complex than simply identifying biases relative to formal normative models and then somehow trying to ‘fix’

these biases. Rather than simply showing how reasoning deviates from broad normative models, we need to understand the constraints that people face in reasoning, what

different goals they have, the tradeoffs that arise given those constraints and goals, and how people navigate those tradeoffs. Once we understand this we can then ask

whether people could navigate those tradeoffs better in certain ways. We need to take into account all of these things - but be careful not to fall into the trap of being able

to describe any behaviour as rational, construed narrowly enough. In many ways this is much more complex and difficult a project than the standard normative rationality

project. Understanding how different reasoning processes actually result in errors we care about, in the real world, is far from straightforward - but ultimately much more

important and potentially useful.

It might be more useful, therefore, to focus on much more specific questions than “Can we

make people more rational?”. Questions like “Where do the strategies people use seem particularly ill-suited to their goals?”, “Where does this actually cause problems in the

world?”, “What alternative strategies might people realistically use in these contexts”, or “How could judgements and decisions be made easier for people?”

When it comes to confirmation bias, the suggestion here is that we might be better off asking first what kinds of strategies people use to seek out and draw inferences from

information, and then ask in what contexts and relative to what goals those strategies might perform particularly well or poorly.

4.5

Summary

I have discussed a number of different interpretations of what it means to say that someone is biased or irrational, and some of the disagreements and confusions that arise

from different uses of these terms. I will now briefly summarise this discussion and what seem to be the implications for confirmation bias.

4.5.1 What does it mean to be biased?

Colloquially, the term ‘bias’ is generally taken to mean a lack of impartiality, and often used interchangeably with the term ‘irrational.’ Hahn and Harris (2014) make the case for using a more precise definition of bias: bias as a property of an estimator (or

heuristic), which occurs when that estimator deviates systematically from a normative standard. To say that reasoning is biased in a certain way, therefore, it is not enough

to observe a single judgement: bias occurs when the same reasoning process leads to a systematic pattern of error on average, relative to some clearly defined standard.

I also discussed how bias isn’t necessarily always the same as inaccuracy: given the

constraints we are operating under, it’s possible that bias might be the optimal solution to some tradeoff (as we see with the bias-variance tradeoff in statistics.) Even if we tend to use reasoning processes that are biased, there may not be alternative processes that

we could feasibly use which would be more accurate without incurring other costs. This means we need to ask not just whether a reasoning process or heuristic is biased, but

what costs that bias incurs - and what costs an alternative strategy might incur.

In the confirmation bias literature, we can understand some confusion and disagreement as stemming from different uses of the term ‘bias.’ In the social hypothesis testing and selective exposure literature, for example, the term ‘bias’ refers to a general tendency or

preference - rather than bias in the strict, systematic deviation sense. Lord et al.’s (1979) study on biased assimilation demonstrates that people fail to conform to intuitively-

based rational principles - but also does not talk about bias in a strict sense. In relatively few places in the confirmation bias literature is ‘bias’ talked about in the more precise