Judgment performance and the impact of task-specific experience

This section describes the tests of hypothesis 1: “Audit managers’ Judgment Performance regarding the tasks ‘identification of client’s business risks’ and

‘identification of client’s entity-level controls’ correlates positively with task-specific Experience”. Task-specific experience has been measured by asking participants to provide their best estimate of the number of times they performed last year the tasks

‘identification of client’s business risks’ and ‘identification of entity-level controls’

(resulting in the variables IDBR1 and IDBM1, respectively) as well es their best estimate of the summed hours spent on these tasks (resulting in the variables IDBR2 and IDBM2 respectively).

Table 4.20 presents the 2-tailed correlations between task-specific experience and the audit managers’ judgment performance. In the table, two outliers (respondents reporting having performed the task ‘identification of client’s business risk 50 times a year, or reporting 500 hours spent on the identification of client’s entity-level controls) have been excluded.

Table 4.20 Pearson correlations between judgment performance and task-specific experience of audit managers risks) a negative (not significant) correlation with task-specific experience. Further, Table 4.20 shows a significant negative correlation between accuracy1 (accuracy3) and task-specific experience as measured by IDBM1 (IDBM2). As a result hypothesis 3 is rejected. This finding is inconsistent with Bonner (1990). Bonner reported that task-specific knowledge related to cue selection (i.e., selection is a sort of proxy for the identification tasks) results in increased judgment performance in an analytical risk assessment task, but does not result in increased judgment performance in a control risk assessment task. Future research should be directed towards exploration of the reasons behind the lacking positive association between judgment performance and task-specific experience. Specifically related to the identification of client’s business risks it is imaginable that each company faces its own engagement-specific business risks.

Auditors performing more often the identification task or spending more hours on the task can develop a biased mind-set. This potential bias could probably result in relatively lower judgment performance and auditors with more client-specific experience become to a lesser extent sensitive for alternative cues. It is recommended that this avenue for future research also takes into account whether, and if so, to what extent decision-aids

(e.g., industry-specific templates with respect to business models including business risks) result in increased judgment performance. Recently, Kotchetova (2002) investigated whether the analysis of client strategy content or strategy process was associated with more accurate (inherent and control) risk assessments. The empirical results supported this hypothesis, although the variability of the risk assessments did not decrease, probably due to a relatively large number of cues. One might indeed assume that a large number of cues require a clear focus on the most important cues. The more experienced auditors experience more difficulty in the selection process. Additionally, the questionnaire used in the experiment requested participants not to make use of any reference material or decision aid. The empirical results could have shown to be different from the results presented here, if the participants had made use of e.g.

industry-specific templates³⁴.

Table 4.21 presents the 1-tailed correlations between task-specific experience and the auditing students’ judgment performance. In the Table, one outlier (respondent reporting having spent 600 hours on the task ‘identification of client’s entity-level controls’) has been excluded.

Table 4.21 Pearson correlations between judgment performance and task-specific experience of auditing students

Table 4.21 shows predominantly positive correlations between judgment performance and task-specific experience for the group of auditing students. This contrasts with the

34 Of the 85 audit managers, 34 (i.e., 40%) indicated that they would have used a decision aid if one should have been available. Some respondents mentioned ‘sparring with colleagues’, other respondents

predominantly negative correlations reported for audit managers (Table 4.20). The significant correlation (correlation of .351; significance 7.7%) between judgment performance related to accuracy 1 (the identification of client’s business risks) and IDBM2 (this task-specific experience variable relates to the identification of client’s entity-level controls) is remarkable. This might be explained by the fact that auditing students already gained some experience in the auditing practice. This type of experience potentially implies that auditing students learn from their experience with other audit clients, i.e. auditing students’ knowledge base is relatively easily transferable to other audit clients. For audit managers, the learning curve regarding the identification of entity-level controls is assumed to be less steep. Task-specific experience as measured by the number of hours spent on the task ‘identification of client’s entity-level controls’ significantly and positively correlates with task-specific experience variables IDBR2 (number of hours spent on identification of client’s business risks; 1-tailed Pearson correlation .523; significance 1.3%) and IDBM1 (number of times the task

‘identification of entity-level controls’ has been performed; 1-tailed Pearson correlation .727; significance 0%). The positive correlation between judgment performance as measured by accuracy1 and task-specific experience as measured by IDBM2 suggests that auditing students spending more time on identification of entity-level controls identify more accurately client’s business risks. The two tasks, although distinguished from each other, are closely connected. Indeed, in the identification of entity-level controls the auditor will be looking for those controls which are most effective in mitigating client’s business risks. This suggests that if auditors spend more hours on the identification of entity-level controls, this will also benefit the accuracy of identification of client’s business risk (transferable knowledge across tasks).

Regarding judgment performance as measured by accuracy3 a significant and positive correlation has been reported in Table 4.21 with task-specific experience as measured by IDBM1 (the number of times the auditor performs the task ‘identification of entity-level controls’). For this correlation, hypothesis 3 (judgment performance is positively correlated with task-specific experience) has been supported. As audit managers performed the task ‘identification of entity-level controls’ an equal number of times (i.e., 9) compared to auditing students, the positive correlation between judgment performance and task-specific experience for auditing students suggests that auditing students experience relatively higher learning effects compared to audit managers (transferable knowledge across clients).

4.4.6 Judgment performance and the impact of the ‘perceived level of feedback