Risk-taking and Social Learning
Study 1: Manipulating Confidence
3.1 Methods 1 Participants
3.1.6 Statistical Analysis
The data were analysed using Bayesian generalized linear mixed models (GLMMs),
modelling the performance of the participants on the task (ie whether they got each trial
correct or incorrect), and their confidence per trial, in R and JAGS using the R2Jags
package.
Mediation Analysis
Throughout this chapter I will be using mediation analysis. Mediation analysis is a type of
path analysis, akin to structural equation modelling (SEM), in which a third variable is added
to the interpretation of a relationship between an independent and a dependent variable.
Structural equation modelling (SEM) is a powerful means of comparing different theoretical
relationships between variables by comparing how well the different models fit the data. In
contrast, mediation analysis is concerned with one particular aspect of a relationship
between two variables. Specifically, a mediating variable is intermediate in the causal path
between the independent and dependent variable, i.e. there is a causal relationship between
the independent variable, the mediating variable, and the dependent variable (MacKinnon
2008). Thus, in contrast to a covariate, a confound, or a moderator, a mediating variable
changes the relationship between the independent variable and the dependent variable such
that it adds an extra causal step between the independent and dependent variable, via the
mediating variable (MacKinnon 2008; MacKinnon, Fairchild & Fritz 2007). Mediation analysis
is very common in social psychological research as it is intrinsic to the theoretical
frameworks that researchers use, i.e. in understanding how psychological or social factors
play a mediating role in affecting behavioural outcomes (MacKinnon, Fairchild & Fritz 2007).
Indeed, 59% of articles published in the Journal of Personality and Social Psychology from
2005-2009 contained at least one mediation analysis, as did 65% of articles in Personality
Traditionally, the majority of mediation analyses involve following a “causal steps” approach
outlined originally by Baron and Kenny (1986), in which the relationship between two
variables is tested prior to introducing a potential mediating variable into the analysis.
Testing the relationship before and after adding a mediating variable ensures that there is
indeed a relationship to mediate (a “direct effect”), and that the mediating variable accounts
for part (or all) of the effect (an “indirect effect”). However, many researchers have called for
a move away from the emphasis on obtaining significant direct effects, arguing that focusing
on the presence of direct effects can lead to misleading or false conclusions (Rucker et al.
2011), as well as preventing researchers from exploring important and interesting theoretical
questions (Zhao, Lynch & Chen 2010). Thus, many researchers argue that the only
requirement for mediation is that an indirect effect be present (Zhao, Lunch & Chen 2010).
Furthermore many researchers have heavily criticized the traditional Baron & Kenny
approach (Hayes 2009; Zhao, Lunch & Chen 2010; MacKinnon, Krull & Lockwood 2000;
Shrout & Bolger 2002; Rucker et al. 2011) and have demonstrated that it is not only
underpowered (Fritz & MacKinnon 2007) but also the least effective means of testing
intervening variables (MacKinnon et al. 2002; Hayes 2009).
Examples of how an indirect effect may be present whilst the direct effect is absent include
suppression effects, asymmetries in statistical power between the effects, and differing
strengths of effects (Rucker et al. 2011). Suppression effects occur when an indirect effect
has a sign which is opposite to the total (direct) effect (MacKinnon 2000). For example, the
presence of lifeguards at a beach may have a direct negative effect on number of fatalities at
the beach (i.e. reducing fatalities). However, there could be a mediating positive effect, in
that the presence of lifeguards reduces the perceived risk of swimming in the sea. As the
perceived risk of swimming in the sea decreases, the number of people choosing to swim in
is large enough, and the strength of the direct effect is small enough, this could lead to a
non-significant direct effect whilst a significant indirect effect is present. Furthermore, it could
be the case that both effects are present, perhaps the perceived risk only has a small effect,
and the overall “net gain” of lifeguard presence is positive, in which case both the indirect
and direct effect are “significant,” with a small “partial” indirect effect. Nevertheless, this
‘partial’ indirect effect is still important and worthy of study, as it could have important
implications for understanding that number of lifeguards may increase number of swimmers.
Accordingly, if there is theoretical evidence or support for the presence of an indirect effect,
this effect should be investigated, regardless of the size or presence of a direct effect.
In this chapter, I am basing my research on previous findings that suggest confidence may
be a mediating factor, not only in the effect of gender on conformity, but also the effect of
stereotype threat on performance. Furthermore, previous work by Cross and colleagues
(2016), to which I used similar methodology, found evidence of an indirect effect of sex on
conformity, mediated by confidence, but no evidence of a direct effect. Thus, I investigated
the mediating variable of confidence on conformity, regardless of the presence of a direct
effect of sex on conformity or not. Specifically, I conducted a Bayesian mediation analysis
(Yuann & Mackinnon 2009) to assess the indirect effect of stereotype threat on performance
via confidence, as well as the indirect effect of sex on conformity via confidence.
Binomial logistic regression vs. Ordered logit
Throughout this chapter, confidence was measured via a 7-point Likert scale (0-6). We
chose to model this ordinal data using a binomial logistic regression, with 6 trials. This was
in contrast to using an ordered logit model, although both methods were used and compared
on pilot data to ensure that they would not lead to meaningful differences in inference. The
binomial model with 6 trials can be interpreted as a kind of multinomial model, although each
trials can be understood by imagining that each confidence level corresponds to flipping a
coin 6 times; heads = “confident” and tails = “not confident.” Thus, a confidence rating of “3”
represents landing on “heads” three out of six times. Although seemingly counterintuitive,
representing the confidence scale in this way effectively makes the middle of the scale the
most likely (i.e. 3/6), and the extremes of the scale least likely (i.e. 0/6, 6/6), which we feel is
representative of the scale, particularly as the default position for the slider-scale is 3, and
participants have to actively move the slider-scale up and down to select their level of
confidence (discussed further in the methods section).
In contrast, modelling the data as their ordered categories requires using a log-cumulative-
odds scale with a cumulative link function (McElreath 2016). The cumulative probability of a
value is the probability of that value, or any smaller value. In our confidence scale, the
probability of a confidence ratings of 3 is the sum of the probability of confidence ratings 1, 2
and 3. Due to this summing of probabilities, our scale cannot include a zero, and thus the
scale was shifted from 0-6 to 1-7 to allow for ordinal logit modelling. Using cumulative
probability enables the use of a linear model with predictor values whilst ensuring the correct
ordering of the outcomes. This requires defining intercepts that represent the cumulative
probability of each confidence level. Thus the model output provides 6 estimates of the
relative frequencies of each confidence level (when the predictors are set to zero), as well as
the estimates for the predictors in the model. Therefore, interpreting the predictor estimates
relies on interpreting the relative change in cumulative log-odds for every value of the
response variable (McElreath 2016). For this reason, alongside the significant increase in
running-time for ordered logit models, we decided to use the binomial model for all
subsequent analyses. There was no meaningful difference in inference between the models,
and both types of models gave very similar model fits. Examples of both types of model
The model of participants’ confidence ratings included a baseline value, an effect for sex,
condition one, condition two, an effect for performance, an interaction between sex and each
condition, and two random effects to allow for variation between individuals and between
question numbers. Again, the control condition was represented in the model as the
baseline, so that any effects from condition one and two are in comparison to the control
condition.
The probability that a participant’s answer was correct was modelled as a Bernoulli variable
(appropriate for binary data, correct = 1, and incorrect = 0). A binomial logistic regression
model was written in JAGS, and ran in R using R2Jags. The model included a baseline
value, an effect for sex, condition one and two, interaction between sex and condition, and
two random effects to allow for variation between individuals and between trials. The control
condition was represented in the model as the baseline, so that condition one and condition
two are compared to the control condition separately, and not to each other.
Participants’ responses to the manipulation check questions were modelled in the same
manner. Whether the participant remembered and believed the task manipulation was
modelled as a Bernoulli variable. The 0-6 likert response for how participant’s expected to
perform compared to the opposite sex was modelled as a binomial with 6 trials in the same
manner as the confidence analysis.
All models were chosen a priori based on hypotheses formed before the data were
collected. All models ran 3 chains with 61,000 iterations, a burnin period of 1000 and went
through thinning of 20. Full model code can be found at www.github.com/lottybrand22 and
examples of model output such as posterior distributions of parameter estimates and trace
plots of Markov chains can be found in the appendix to this chapter.
All plots are a display of the raw data only, with error bars showing 95% confidence intervals.
and 95% posterior distribution parameter estimates. 95% credible intervals that include zero
suggest there is little evidence that the parameter had an effect on the outcome variable.
95% credible intervals that do not include zero suggest that the parameter had a negative or
positive effect on the outcome variable in the model, depending on the direction of the