Study 2

A.2.1. Study 2 signal detection model

Sensitivity analyses

Study 2 hierarchical signal detection model. A series of sensitivity analyses were conducted to assess the sensitivity of the estimates to the choice of priors. In the first, a robust version of the model was implemented replacing the normal priors on the α coefficients with t distributions. For all α (except α0, for which the location parameter of the prior was μ as before)

𝛼 ~ 𝑡(0, 𝜎, 𝜈 = 5) (84)

The second sensitivity analysis placed a hyperprior on the scale of the prior for the precision of the coefficients of the fixed factors, to allow all factor coefficients to exert shrinkage upon each other rather than shrinkage solely occurring between levels of the same factor. For all σF

𝜎𝑓𝐹 ~ 𝑡(0, 𝜎𝑠ℎ𝑟𝑖𝑛𝑘𝑎𝑔𝑒, 𝜈), 𝜎𝑓𝐹≥ 0

𝜎𝑠ℎ𝑟𝑖𝑛𝑘𝑎𝑔𝑒 _{~ 𝑡(0,1, 𝜈) , 𝜎}𝑠ℎ𝑟𝑖𝑛𝑘𝑎𝑔𝑒_{≥ 0}

(85) (86)

134

The final sensitivity analysis replaced all folded t prior distributions for σ with Gamma (0.001,0.001) priors for τ where

𝜎 = √1 𝜏

(87)

None of the alternative prior distributions substantively affected the model outcomes. Posterior predictive check

Model fit was assessed by comparing d’ and c estimates derived from the model to point estimates derived directly from the Study 2 data. 1000 random draws were made from the posterior distributions of d’ and c for each participant in each condition to create 1000 simulated datasets which were then compared with point estimates of d’ and c, calculated directly from the observed data as follows: 𝑑̂′ = 𝛷 −1_(𝜌̂ 1) + 𝛷−1(𝜌̂2) √2 𝑐̂ = −0.5(𝛷−1(𝜌̂₁) − 𝛷−1_(𝜌̂ 2)) (88) (89) where p̂1 is the proportion of correct responses following a left-sided stimulus, and p̂2 is the proportion of correct responses following a right-sided stimulus. p estimates of 0.99 or 0.01 were substituted for estimates of 1 or 0 respectively, in order to avoid infinite estimates of d’ or c (NB this adjustment employed a constant rather than a proportion of trials due to the highly variable trial Ns resulting from the adaptive paradigm).

135

Figure A.2: Study 2 signal detection model: posterior predictive check. Model predictions for d’ are consistent with (top left panel) and no more extreme than (bottom left panel) point estimates derived from the observed data. Predictions for c are somewhat more conservative than observed point estimates with respect to large positive or negative values of c, but in general are consistent with (top right panel) and no more extreme than (bottom right panel) the study observations.

Figure A.2 shows the point estimates by participant and condition plotted against the mean model- derived predicted data by participant and condition (upper panels) and against p-values respectively derived from the formulae:

𝑃(𝑑′𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑒𝑑 ≥ 𝑑′̂̂𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑) 𝑃(𝑐𝑠𝑖𝑚𝑢𝑙𝑎𝑡𝑒𝑑 ≥ 𝑐̂𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑)

(90) (91) Examination of the p-values indicates that for the vast majority of d’ and c estimates the predicted data are no more extreme than the observed data. In the case of d’, the model predictions tend to be larger than occasional point estimates less than 0, suggesting that the model has succeeded in avoiding large theoretically impossible negative predictions of d’. In the case of c, the model predicts somewhat larger values when point estimates are very small, and somewhat smaller values when point estimates are very large, suggesting that its estimates have been affected by shrinkage and are therefore more conservative than those calculated directly from the data. The positive relationships apparent between mean model predictions and observed point estimates plotted by participant and condition suggest that the model is succeeding in predicting participant and condition effects.

136

The same simulated data are presented in Figure A.3, averaged over group and condition.

Figure A.3: Study 2 signal detection model: group-level posterior predictive check. Histograms show the distributions of 1000 model predictions for values of d’ and c averaged by group and condition. Vertical lines show the point estimates for d’ and c averages derived from the observed data. In general the model predictions are reasonably consistent with the point estimates.

A.2.2. Study 2 RT distributions mixture model

Sensitivity analysis

The mixture model with a linear model placed upon two Weibull parameters was derived from an unrestricted random effects model with gamma priors described below in Appendix F.2. The outcomes of the linear model and the unrestricted model were very consistent, in terms both of the distribution of lapses and fast guesses, and in terms of the effects on the Weibull parameters.

Posterior predictive check

Model fit was assessed by comparing observed RTs to simulated RTs derived from underlying 3- parameter Weibull distributions predicted by the model. A random draw was made from the posterior distribution of each Weibull parameter for each participant in each condition, and reaction times in the same proportion to those occurring in the observed data were simulated by making draws from the resulting Weibull distributions. This process was repeated 1000 times to create 1000 simulated

137

datasets. Figure A.4 shows the observed RT means plotted against the predicted RT means for each participant in each condition and against p-values derived from the formula

𝑃(𝑅𝑇̅̅̅̅_𝑗𝑘𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑≥ 𝑅𝑇̅̅̅̅𝑗𝑘𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑) (92)

Examination of the p values indicates that the predicted data are no more extreme than the observed data. The positive relationship apparent between model predictions and observed data suggest that the model is succeeding in predicting participant and condition effects. The bottom-left panel Figure A.4 shows a quantile-quantile plot in which the quantiles of the observed data (excluding probable lapses and fast guesses) are plotted against the quantiles of 50 simulated datasets. The simulated data are comprised of 50 randomly chosen Weibull-derived datasets drawn from the set of 1000 described above. As can be seen from the QQ plot, this predicted distribution is similar to the distribution of the observed data. This provides some reassurance that the Weibull is an appropriate distribution for modelling these data.

Figure A.4: Study 2 Weibull fit to RT data: posterior predictive check. Posterior predictive check on the Weibull fit to the RT data with linear models on the ψ and λ parameters. Predicted mean RTs are in general consistent with (top left panel) and no more extreme than (top right panel) observed mean RTs by participant and condition. The QQ plot (bottom left panel) indicates that the observed data distribution (excluding probable lapses and fast guesses) is similar to a predictive distribution derived from the Weibull component of the mixture model (50 simulations shown).

138

The fit of the linear models on ψ and λ was also checked against the individual-level parameter estimates. A random draw was made from the posterior distribution of each component of the linear model on ψ, and these were combined with each other and with a draw from the relevant noise distribution to produce a posterior predictive estimate of ψ for each participant in each condition. The process was then repeated 1000 times. A similar process produced posterior predictive estimates of λ. The estimates (averaged by group and over laterality for brevity) are shown in the form of histograms in Figure A.5. Vertical lines represent mean individual estimates of the parameters, averaged by group and over laterality. In summary, the individual Weibull fits are consistent with the observed data, and the predictions of the linear models are consistent with the individual Weibull fits.

Figure A.5: Study 2 fit of the linear models on Weibull parameters ψ and λ to the individual estimates. Histograms show 1000 draws from the posterior distributions of the linear model components, combined into 1000 posterior predictive estimates of each parameter by group and condition. Lines represent the mean individual parameter estimates, averaged by group and condition.