Model Fitting

As m entioned in the exam ple, a model frequently chosen to describe observer’s perform ance and the one used throughout this thesis is the cum ulative Gaussian. The form ula for this curve is given below.

1 m - { x - f i Ÿ

I 1 OO 9

/ ( x ; / t , c r ) =

* I g 2**

dx

4 ïn * a

It is evident that there are two free param eters, p and a , that can be varied to dictate the exact shape o f the w aveform , although the general sigm oidal form is constant. W hen fitting a m odel o f this type to data these param eters are varied to m inim ise som e error m etric that m easures the overall difference betw een the m odel and the data. In the chapters that follow the error m etric used is the sum o f the squared differences betw een the em pirical data and the m odel. The process o f approxim ating this m inim um is com m only referred to as least squares m inim isation. The m inim um sum o f squares value can be approxim ated by an iterative process that adjusts the param eters p and a in reducing am ounts. The ‘solver’ routine M icrosoft ® Excel is well suited to this task, and has been used throughout these experim ents. One further refinem ent to the fitting process is req u ired to re fle c t the use ad ap tiv e m ethods o f se le c tin g lev els o f the independent variable. Unlike the m ethod o f constant stim uli, adaptive m ethods such as the Q uest algorithm (W atson & Pelli, 1983) used here, result in a different num ber o f trials being collected for each level o f the independent variable. This is because they progressively ‘hom e-in’ on certain stim ulus values yielding high levels o f p erform ance that are e fficie n t determ inants o f the location o f the psychom etric function. C onsequently these stim ulus values often obtain many m ore sam ples than values presented early in a block o f trials. W hen fitting a curve to data collected in this way it is desirable to w eight the fitting process m ore heavily by data points resulting from m any sam ples, as these are

likely to be m ore accurate m easures o f perform ance. To achieve this w eighting the fitting process m odulates the difference betw een each data point and the m odel by the variability associated with the data point. For points w ith m any sam ples the variability is low , for those w ith few it is high. T he standard deviation for each data po in t is calculated in dividually, assum ing th at the subject’s responses are draw n from the binom ial distribution, and then divided into the difference betw een data point and model.

O ther researchers com m only use P robit analysis to fit a m odel to a data set. Essentially this procedure avoids fitting a sigm oiod to a sigm oidal data set by replotting the data on log axes. This has the effect o f straightening the data set, to w hich a straight line can then be fitted w ith ease using a least squares m inim isation process o f the type described above. The availability o f pow erful com puter hardw are allow s autom ation o f the fitting procedure to take place without going through the transform ation process, hence its use here.

C onventionally fitting a m odel like the cum ulative G aussian to a data set requires only two param eters (p. and a , as described above) to be perturbed to obtain the low est error m etric. However, W ichm ann & Hill (2001) advocate that one fu rth er p a ra m ete r be included. T he cu m u lativ e G aussian fu nction is constrained to asym ptote at 100%, a level o f perform ance that an ideal observer would achieve. How ever, even at very high levels o f stim ulus strength observers are prone to lapses o f concentration and ‘finger erro rs’ (i.e. m aking an incorrect response d espite know ing the co rrect answ er). In such circu m stan ces an im proved fit to the em pirical data can be obtained if the asym ptotic value o f the model is allow ed to take on values less than 100%. In their analysis o f this topic the authors recom m end that the fu nction m axim um be perm itted to range betw een 96% and 100%, and this strategy has been adopted throughout the experim ents reported here. O ne possible criticism that could be levelled at this procedure is that any m odel can be m ade to fit a data set m ore closely by increasing the num ber o f param eters that are free to vary. Key to the decision to add a further param eter is w hether it is done m erely to im prove the fit, or w hether it is intended to m ore accurately m odel the o b se rv e r’s behaviour.

F a ilu re to ad o p t the a dditional p a r a m e te r w ith a d a ta set that in c o rp o ra te d lapses at h ig h levels o f p e r f o r m a n c e w o u ld resu lt in an in c o rre c t m o d e l b e in g a p p lied , w h ic h w o u ld in turn lead to e rro rs in the v a lu e s e s tim a te d fro m it. F ig u r e 2.4 i llu s tra te s the p r o b l e m w ith a n e w e x a m p l e d a ta set in w h ic h th e m a x i m u m em p iric a l p e rf o r m a n c e re a c h e d w a s 9 6 % correct. T h e blue c u rv e is a c u m u la tiv e G a u s s ia n c o n s tra in e d to reach 100%, as e v id e n c e d by it o v e r s h o o tin g the d a ta at h ig h stim u lu s values. T h e red c u r v e is also a c u m u la tiv e G a u s s ia n , but has b een a l l o w e d to r e a c h a s y m p t o t e at v a l u e s b e t w e e n 9 6 % a n d 1 0 0 % . It c a n be a p p re c ia te d that the d isto rtio n in tr o d u c e d in fitting the b lu e m o d e l resu lts in an a p p re c ia b ly d iffe re n t th re s h o ld e s tim a te fro m that o b ta in e d u s in g the red c u rv e (d a sh e d lines e s tim a tin g the s tim u lu s v alu e re su ltin g in 7 5 % correct).

S u b -a sym p to tic m odel fittin g

1 . 1 ♦ em pirical d a ta 100% as y m p to te m odel — — 96% asy m p to te m odel 1 0 .9 0. 8 0 .7 0. 6 Q. 0 .5 0 .4 0 .01 0 .0 2 0 .0 3 0 .0 8 0 0 .0 4 0 .0 5 0 .0 6 0 .0 7 0 .0 9 0.1 In d e p e n d e n t v a ria b le (e g c o n tra s t)

Figure 2.4 S u b -a sy m p to tic m odel fitting. Blue d ia m o n d s rep re se n t a new d ata set, similar to that o f figure 2.3, except that the o bserver does not reach 100% p erform ance at the highest stim ulu s values. T w o m ode ls have been fitted to this new data set. The blue curve is constrained to reach asym pto te at a probability correct o f 1. The effect that this has on the 75% threshold estimate (broken lines) is evident by co m p ariso n with the red curve, whose asym ptotic level was allow ed to take on values betw een 0.96 and 1.

The blue model has been distorted, leading to a higher threshold than predicted by the red model.