Results: 1D Psychometric Function

For 1D PFs, both GPC and PR produced outlier trials for small sample numbers that disproportionately affected computation for the means and standard deviations. Following data collection from the simulations, we detected these outliers by thresholding at the 98th percentile (i.e., removing the 2% of scores farthest from the mean) across all trials and conditions for GPC and PR independently. A total of 981 and 716 outliers were detected out of 28000 total simulations each for the PR and GPC runs, respectively, primarily from trials with fewer than 20 observed samples. We excluded these outliers from the computations of means and standard deviations. We also omitted any trials using fewer than 10 observed samples from these computations because of generally poor performance at low sample numbers for both methods.

82

Figure 4.2 shows four representative examples of unidimensional PFs estimated by the GPC and

PR techniques with four different numbers of observed samples (i.e., simulated subject responses) at fixed absolute intensity values. Identical observations were used for both methods in each panel. Qualitatively and quantitatively, GPC and PR perform very similarly, with systematically increasing estimation accuracy as the number of observed samples increases.

Figure 4.2: Examples of 1D PF estimation using PR and GPC. Each subplot shows model performance after (A) 20, (B) 100, (C) 150, and (D) 200 samples. True values of α and β were 66 dB and 10 dB/%, respectively, and sampling was performed at 20 distinct intensities within the interval. Units for α and β error are dB and dB/%, respectively.

Figure 4.3 shows the mean and standard deviation of absolute errors for α and β as a function of

number of observed samples, averaged across all 140 conditions and trials, for both PR and GPC. As expected, the accuracy and reliability for both techniques increased with the number of samples. Estimates for α are consistent between PR and GPC. For β estimates at lower sample numbers, GPC appears to be slightly less accurate or equivalently accurate yet more reliable than

83

PR, although this difference disappears at higher sample numbers. Nevertheless, only 2 of 382 total comparisons showed statistically significant differences between PR and GPC (p<0.05, Kolmogorov-Smirnov test, Bonferroni corrected for multiple comparisons), consistent with the assessment that these two methods generally exhibit statistically indistinguishable performance.

Figure 4.3: Error in 1D α and β estimates for PR and GPC across all conditions. Absolute error in estimates of (A) α and (B) β as a function of number of observed samples in the unidimensional case. Blue solid and red dashed lines denote mean absolute errors of the PR and GPC estimates, respectively, and the matching shaded regions designate

1 standard deviation above and below the mean.

Because of increased uncertainty in the transition zone for higher values of β, we evaluated estimator performance for larger versus smaller spreads as a function of β value. Figure 4.4 shows the mean and standard deviation absolute errors in α and β for each β value tested. In all cases, both accuracy and reliability generally increase as a function of sample number. However, large values of β decrease the accuracy of both GPC and PR, particularly for lower numbers of observed samples. Overall, the trends for GPC and PR are generally quite similar, revealing no consistent difference in estimator quality between the two methods.

84

Figure 4.4: Error in 1D PR and GPC α and β estimates for different β values. Absolute error in estimates of (A-G) α and (H-N) β as a function of number of observed samples for unidimensional PFs. Blue solid and red dashed lines denote mean absolute errors of the PR and GPC estimates, respectively, and the matching shaded region designates

1 standard deviation above and below the mean. Each subplot corresponds to a distinct β value.

Because a fixed number of samples can be distributed across intensity in different ways, we investigated the effect that the number of distinct intensities and the number of repetitions per intensity had on the performance of both estimators. Figure 4.5 shows the mean and standard deviation of absolute errors in α and β for each unique trial count per intensity. Both accuracy

85

and reliability generally increase as a function of sample number. Again, overall estimator performance is quite similar between the two methods, with 0 of 1910 comparisons resulting in statistically significant differences for either α or β (p<0.05, K-S test, Bonferroni corrected for multiple comparisons).

Figure 4.5: Error in 1D PR and GPC α and β estimates for different sampling distributions. Absolute error in estimates of (A-E) α and (F-J) β as a function of number of observed samples for unidimensional PFs. Blue solid

and red dashed lines denote mean absolute errors of the PR and GPC estimates, respectively, and the matching shaded region designates 1 standard deviation above and below the mean. Each subplot corresponds to a distinct

condition for number of intensities and number of repetitions per intensity.

We repeated all of the analysis described above for the numerical accuracy and reliability values of 50% point and 25-75% interquartile range. We observed identical trends for these measures, again indicating that GPC results in functionally indistinguishable estimator performance

86

compared to parametric maximum-likelihood PR for 1D PF estimation. These results are shown in Appendix 1, Figures A1.1–A1.3.

To further investigate the agreement between GPC and PR estimates on 1D PFs, we directly compared the GPC and PR performance for estimates derived from the same set of data. Figure

4.6 shows plots of GPC estimates versus PR estimates, with α and β behavior shown in Figure

4.6A and Figure4.6B, respectively. Note that 1 outlier was removed from the comparison for α. Coefficient of determination values for both α and β linear fits were very high, indicating that the linear functions were good fits for the data. For both α and β, linear slope terms were very close to 1 and linear intercept terms were near 0, indicating high PR and GPC estimate agreement. Correlation coefficients between PR and GPC estimates were 0.9992 and 0.9941 for α and β, respectively, again indicating a high degree of agreement between estimates. GPC does appear to overestimate small β values compared to PR, consistent with the data in Figure 4.4H-J.

Figure 4.6: Direct comparison of 1D PR and GPC α and β estimates. Black points represent individual PR/GPC pairs for estimates derived from the same set of data, and the red line is a linear fit to the data. Equations describing the

87

Across all 28000 2D trials, the median χ statistic was 3.45 for GPC trials and 5.28 for PR 2 trials. Using a significance level of p < 0.05, Bonferroni corrected for multiple comparisons, 165 of 28000 GPC simulations (0.59%) demonstrated statistically poor fits, while 201 of 28000 PR simulations (0.72%) demonstrated statistically poor fits.