Lack of normative standards

The phenomenon of belief polarization - when two groups with opposing initial views both strengthen their beliefs based on reading the same evidence - has often been cited

as evidence of confirmation bias. It seems that the most plausible explanation for this polarization is that people apply different standards of evaluation to evidence that sup-

ports what they believe than that which conflicts with it, resulting in each side weighing supportive evidence more heavily.7

Jern et al. (2014) show that belief polarization can be consistent with a normative account of belief revision - that in some cases, rational agents with opposing beliefs

should both strengthen their positions as a result of reading the same information. Typically, studies of belief polarization have not explicitly included normative models of

how people should interpret information and update their beliefs, simply relying on the common-sense assumption that belief polarization is irrational. Jern et al. show that this

assumption is not as reasonable as it might seem by presenting a normative probabilistic analysis within which belief polarization can arise. They then apply this model to

previous studies of belief polarization to show how their results may be consistent with a normative theory of belief updating.

Consider the situation in which two people observe data D which bears on some hy- pothesis H. Contrary updating occurs whenever one person’s belief in H increases after

observing D, and the other person’s belief in H decreases after observing D. This can be contrasted with parallel updating, where both people update their beliefs in the same

direction. The conventional wisdom is that parallel updating is always the normative outcome (Lord et al., 1979).

However, in Bayesian terms, whether or not two people increase or decrease their belief in H after observing D depends on their likelihood ratios - which in turn may depend

on the assumptions they each make about factors influencing the relationship between the hypothesis H and the data D. Jern et al. (2014) consider a number of different

7

Belief polarization may better be thought of as an illustration of a broader tendency for people to interpret information in asymmetric ways, depending on whether or not it fits with their preconceptions.

relationships between H and D that might give rise to contrary updating, represented using Bayesian networks. As a simple example, suppose two doctors are given a patient’s

test result (D), which we assume has only two possible outcomes (positive/negative), and there are two hypotheses for what disease the patient has. If the patient has disease

1, the test is likely to produce a positive result, and it the patient has disease 2, the test is likely to produce a negative result. However, if factor V represents whether the patient

has low or high blood sugar, and this factor affects the meaning of test result D, and two doctors disagree about the value of V, then two doctors could agree on everything

else, behave as normative Bayesian agents, but end up updating in different directions based on data D.

More generally, in the real-world, hypotheses and data are rarely considered in isolation, and inferences about one hypothesis typically depend on other hypotheses and beliefs.

Jern et al. (2014) take this approach to explain how the results of Lord et al.’s classic (1979) study may arise under normative probabilistic inference. To recap, in this study

supporters and opponents of the death penalty were asked to read about two fictional studies, one supporting and another opposing the idea that the death penalty is an

effective crime deterrent. Jern et al. suggest that if participants make two simple as- sumptions: (1) that studies are influenced by research bias, and (2) that one’s own beliefs

about the effectiveness of the death penalty differ from the consensus opinion among researchers, then belief divergence can arise through normative probabilistic inference.

Given these assumptions, Alice’s prior belief that the death penalty is an effective de- terrent gives her reason to be sceptical of the study showing the opposite conclusion -

she expects the researchers believed the opposite conclusion, and so researcher bias may have influenced the results. If Bob had the opposite prior belief and the same assump-

tions, he would be sceptical of the other study, and so each would put less weight on the study opposing their initial viewpoint, therefore leading them to update in opposite directions.

Of course, no claim is being made here about whether it is reasonable for participants

to make such assumptions, or even that it is likely they were making such assumptions. This simply illustrates how,given certain assumptions, putting more weight on evidence that supports your prior beliefs may not be entirely irrational, and result in two people with different prior beliefs drawing opposite conclusions from the same information.

Jern et al. (2014) also discuss some other conditions under which findings of belief polarization may be normative. First, they suggest that polarization may emerge as

a consequence of mapping an ordinal variable (the strength of the effect the death penalty has on crime deterrence) onto a binary variable (whether or not the death

penalty is an effective deterrent.) This seems similar to the suggestion we made that overconfidence may arise from how people map their beliefs onto probabilities rather

than biases in reasoning. Second, they consider the case where participants with strong and weak Christian beliefs read a story describing how church leaders had conspired

to cover up new evidence undermining the idea that Jesus is the son of God (Batson, 1975). They suppose that participants have other beliefs which influenced both their

initial judgements about whether Jesus is the son of God or not, and which influence their expectations about what the information would mean if he were - characterised

as a certain worldview. For instance, someone with a Christian worldview believes that Jesus is probably the son of God, and that followers of Jesus are likely to have their

faith challenged by others. Someone with a secular worldview believes that Jesus is probably not the son of God, but that if he were, his followers would be unlikely to

encounter challenges. These worldviews affect their interpretation of the data that seems to challenge faith in Jesus - and so two people with differing prior views will disagree

about whether this provides support for or against the hypothesis that Jesus is the son of God, and diverge as a result.

Again, the authors are not claiming that these interpretations necessarily explain what is going on in the experiments discussed. However, they are suggesting that these in-

terpretations or similar ones arepossible, and that therefore it is not straightforward to simply claim that belief polarization is irrational. More broadly, Jern et al. (2014) illus-

trate how, given certain assumptions, it is rational to give more weight to confirmatory evidence, and therefore interpret apparently ‘balanced’ or neutral data as supportive of one’s current hypothesis. As Klayman puts it, “from a Bayesian point of view, the fact

that a study gives a surprising result does constitute valid probabilistic evidence that the study was done incorrectly... how much distrust of disconfirming results is appro-

priate and how much is too much? The normative issues here are complex and remain unresolved.” (Klayman, 1995, p.395) We will look at some of these complex normative

issues, and their impact on understanding confirmation bias, in more detail in chapter 4.