Performance Execution: Data collection and model building

To prepare data collection all measures were documented using a modified “performance measure record sheet” proposed by Neely et al. (Neely et al., 1997). The purpose of this documentation was to ensure that all implications of the proposed measures were considered. Besides that, the record sheet could also serve for communicating the measures to the employees. Two exemplary record sheets for the measure “inhabitants” and “total active volume” are displayed below.

Table 16: performance measure record sheet (measure "inhabitants")

TS5: performance targets

Title Abbreviation What is being measured

Inputs

Customers per

employee CUST Total number of customers served per employee Inhabitants per

employee INHAB Number of inhabitants living within the branch´s catchment area

Salary SAL Salary

Active volume of

customer´s deposits VOLACT Sum of the total active volume of deposits of the employee´s customers

Outputs Contribution margin II CMII

Contribution margin II resulting from interest earnings

Commission earnings CommE Sum of commissions earned

Title Inhabitants per employee

Abbreviation INHAB What is being

measured?

Number of inhabitants living within the branch´s catchment area

Formula (if applicable)

n.a.

Frequency annual

Source Federal Statistical Office

178

Table 17: performance measure record sheet (measure "Total active volume")

Except from the number of inhabitants within the catchment area, the data were provided by the bank´s controlling department. Since it was agreed that the present case study should be a test run, archival data from 2012 were used. Therefore, no conclusions about current performances of employees could be drawn. A total of 44 employees were working as account managers in 2012. Since four of them left the bank or changed their jobs within the bank during 2012, they were excluded from the sample. The 40 remaining employees all worked full-time. Thus, a full-time equivalent did not have to be calculated. The data were provided as linked-anonymous data using numbers from 1 to 40 instead of names. Although the data included personal data (e.g. salary) no sensitive data as defined in the Data Protection Act of 1998 were used. The data were transferred to an Excel spreadsheet.

In designing the DEA application, adequate DEA models had to be selected for each transformation stage. In a first step, this involved considerations about the rationale behind the performance evaluation. It was agreed that, in general, the purpose of the evaluation was not to reduce resource usage (inputs), but merely to stimulate improvement and growth of outputs. Additionally, it was assumed, that outputs could be changed or controlled by the employees more easily than inputs. Therefore, it seemed reasonable to employ an output-oriented model. Thus, the difference between a score of 1.00 and the score calculated represents the proportions by which the employees could improve their outputs by maintaining the same input. A second consideration was to decide whether to assume constant or variable returns to scale (see 2.3.2). The production of a banking service is, due to customer participation and a high degree of intangibility, very heterogeneous. Thus, the outputs of this process rely to a high degree on individual effort (e.g. to selling financial products to a customer) requiring the same effort each time the service is produced. In this respect, scale effects cannot be

Title Total active volume of customer´s deposits

Abbreviation VOLACT What is being

measured?

Sum of the total active volume of deposits of the employee´s customers Formula (if

applicable)

Sum of active volume of customer´s deposits per employee Frequency annual

Source Controlling department

179

anticipated. Thus, assuming constant returns to scale, the CCR model with an output

orientation was employed for all stages8_.

High correlations among input and output variables may cause difficulties in DEA analysis. Thus, one purpose of the study was to find a “parsimonious” model. Adding highly correlated variables may result in the increase of performance scores, indicating that many DMUs are efficient, when they actually differ by only a small and random fluctuation. A parsimonious model typically shows low correlations among the variables and uses as many variables as needed, but as few as possible (Wagner et al., 2003). To ensure a parsimonious model, a correlation analysis for the set of proposed input and output variables was run for each stage. The results of the linear Pearson correlation (correlation coefficient r) are provided in the tables below.

Table 18: correlation matrix TS1 “acquisition” Table 19: TS2: correlation matrix TS2 “consultancy and sales” Table 20: correlation matrix TS3 “profitability” 8 _{In TS4 „bonuses“ an input-oriented CCR model was applied additionally as an alternative way to} calculate the amount of bonus payments

CUST INHAB VOLACT SELL APP

CUST 1.000 -0.280 -0.496 -0.097 0.379 INHAB 1.000 0.272 -0.030 -0.057 VOLACT 1.000 0.055 -0.061 SELLQ 1.000 0.305 APPQ 1.000

SELL APP VOLACT VOLL BSH RuV Union

SELL 1.000 0.305 0.055 0.364 0.207 0.487 0.352 APP 1.000 -0.061 0.175 0.567 0.203 0.488 VOLACT 1.000 0.150 -0.267 0.195 -0.184 VOLL 1.000 0.499 0.763 0.604 BSH 1.000 0.410 0.533 RuV 1.000 0.598 Union 1.000

SAL Bonus TRAI CMII CommE

SAL 1.000 0.738 0.743 0.670 0.721 Bonus 1.000 0.686 0.397 0.542 TRAI 1.000 0.397 0.553 CMII 1.000 0.742 CommE 1.000

180 Table 21: correlation matrix TS4 “bonuses” Table 22:correlation matrix TS5 “performance targets”

The higher the correlation coefficient r, the stronger is the relationship between the two variables. In literature, there are several categorizations concerning the strength of the relationship. A very common classification is the classification by Dancy and Reidy (2007):

Table 23: classification of Pearson´s correlation coefficient r (Dancey and Reidy, 2007)

Other authors assess ranges from 0.8 to 0.9 as “very strong” or from 0.75 to 0.95 as “high” (Myers et al., 2010). Applying a cautious approach, all correlations that exhibited an r > 0.75 were marked as highly correlated. Examining the correlations with regard to this condition lead to several insights. Two variable combinations are highly correlated. The strongest relationship is observed between the variables “RuV” and “VOLL”. Also, the variables “VOLACT” and “CMII” are highly correlated. All other variable combinations correlate on rather weak or moderate levels.

The correlation analysis was discussed with management. The correlation between “VOLACT” and “CMII” could be neglected since the variables belong to different categories (“VOLACT” serves as an input variable and “CMII” as an output variable). The

SAL CMII CommE Bonus

SAL 1.000 0.670 0.721 0.738

CMII 1.000 0.742 0.409

CommE 1.000 0.542

Bonus 1.000

CUST INHAB VOLACT SAL CMII CommE

CUST 1.000 -0.280 -0.496 -0.290 -0.387 -0.474 INHAB 1.000 0.272 0.196 0.277 0.327 VOLACT 1.000 0.554 0.778 0.628 SAL 1.000 0.670 0.721 CMII 1.000 0.742 CommE 1.000

Value of the Correlation Coefficient Strength of Correlation

1 Perfect 0.7 - 0.9 Strong 0.4 - 0.6 Moderate 0.1 - 0.3 Weak 0 Zero

181

correlation of “VOLL” and “RuV” is between variables of the same category (outputs). However, management found both variables important for mapping the performance process precisely and for reflecting strategic directions of thrust. Thus, it was agreed not to remove them in a first run, but to investigate the issue in the context of a subsequent sensitivity analysis (which is discussed in section 6.3.2).

In a final stage of data collection, a data summary sheet was created for each stage. The sheet for the stage “acquisition” is illustrated below (the data summary sheets for TS2 to TS5 are provided in annex 1).

Figure 33: data summary sheet TS1 “acquisition”

182

7.3 Performance Assessment

The following section presents the results for transformation stages 1 - 5 that were calculated applying the DEA CCR-O model. Additionally, a sensitivity analysis including a superefficiency analysis and weight restrictions was carried out to ensure the validity of results.

7.3.1 Results for transformation stages 1 - 5

Since multiple performance stages were assessed, performance scores were calculated separately for each transformation stage. To calculate the scores, a CCR-O model was used. No weight restrictions were incorporated. The calculation was computed with the Banxia Frontier Analyst Software (Version 4.2.0).

7.3.1.1 Results for TS1 “acquisition”

The table below holds the results for the transformation stage “acquisition”.

Table 24: results for TS1 “acquisition”

In total, eleven employees achieved a score of 1.00. Those employees cannot improve their output, without increasing the input. In other words, if they had to acquire more customers they need to be assigned to another territory with more inhabitants or with customers that have higher average value in deposits. Those employees are also

Unit name Score Ref. count number of peers peers Unit name Score Ref. count number of peers peers 1 0.68 0 3 12 38 40 21 0.86 0 2 20 25 2 1.00 1 0 22 0.52 0 4 12 34 37 38 3 0.66 0 4 36 37 38 39 23 0.48 0 3 12 38 40 4 0.67 0 4 12 34 38 40 24 0.82 0 3 20 25 34 5 0.67 0 2 20 34 25 1.00 11 0 6 0.92 0 1 20 26 0.65 0 1 34 7 0.37 0 2 34 36 27 0.41 0 4 12 34 37 38 8 0.53 0 1 40 28 0.93 0 1 34 9 1.00 1 0 29 0.40 0 3 12 37 38 10 0.91 0 2 20 25 30 0.90 0 3 25 38 39 11 0.64 0 1 20 31 0.69 0 4 36 37 38 39 12 1.00 14 0 32 0.91 0 2 34 40 13 0.99 0 2 12 20 33 0.31 0 3 12 25 37 14 0.80 0 3 25 38 39 34 1.00 14 0 15 0.46 0 4 25 36 37 39 35 0.38 0 3 12 34 40 16 0.60 0 4 12 20 25 34 36 1.00 5 0 17 0.78 0 3 12 20 25 37 1.00 8 0 18 0.35 0 4 12 34 38 40 38 1.00 12 0 19 0.78 0 4 12 20 25 34 39 1.00 6 0 20 1.00 11 0 40 1.00 8 0

183

referenced as peers to employees who obtained a score less than 1.00. One of the most valuable procedures in DEA is the comparison of an inefficient DMU with its counterpart – the peer or the reference set. The total reference count indicates how often an “efficient” employee serves as peer to others. The reference count does not equal the number of “inefficient” employees, but is slightly higher. This is due to the fact that an inefficient unit’s reference set may contain several efficient units with a similar input/output orientation. Together they form a reference set for the inefficient DMU and provide examples of good operating practice for the inefficient DMU to emulate. This is why usually more than one peer is indicated per employee.

Considering the distribution of reference counts for TS1, it is striking that they are quite evenly distributed. Thus, nine out of eleven efficient DMUs repeatedly are assigned as references. This may be an indication, that the DMUs are distributed quite even on the efficient frontier and that there are not many outliers with very exceptional input-output structures. To verify this hypothesis, a subsequent sensitivity analysis should be instructive.

Table 25: results summary for TS1 “acquisition”

The results summary for TS1 reveals that the range between the maximum and the minimum score is 0.69, which is quite large. The mean score however is within the upper third and the standard deviation of 0.23. is rather moderate. Thus, there may be some DMUs who are outliers downwards. Here, a more detailed look at the results on individual level should explain the result and give hints for improvement (see section 6.3.2). MIN 0.31 MAX 1.00 MEAN 0.75 SD 0.23 no eff. Units 11.00

184

7.3.1.2 Results for TS2 “consultancy and sales”

The results for the second stage “consultancy and sales” are presented in the tables below:

Table 26: results for TS2 “consultancy and sales”

Table 27: results summary for TS2 “consultancy and sales”

In contrast to “acquisition”, the number of employees achieving the maximum score of 1.00 in “consultancy and sales” has decreased to six. Interestingly, none of the employees who managed to be efficient in stage “acquisition” is also efficient in consultancy and sales. Considering the six efficient employees in this stage, all of them serve as peers to others. However, employees no. 7, 26 and 29 are assigned as peers four times more often than the other efficient DMUs. This may be an indication that the set is dominated by those DMUs. The subsequent sensitivity analysis will shed more light on this. The range from maximum to minimum scores is 0.83 and with this even larger than in stage “acquisition”.

Unit

name Score reference count

number of peers

peers Unit

name Score reference count

number of peers peers 1 0.35 0 5 7 19 26 29 21 1.00 5 0 21 2 0.64 0 4 7 19 29 35 22 0.30 0 3 7 26 29 3 0.53 0 4 7 19 26 29 23 0.25 0 4 7 26 29 35 4 0.32 0 3 7 26 29 24 0.94 0 2 26 29 5 0.99 0 2 26 29 25 0.69 0 3 19 26 29 6 0.62 0 3 21 26 29 26 1.00 30 0 26 7 1.00 27 0 7 27 0.42 0 3 7 26 29 8 0.58 0 3 7 26 29 28 0.63 0 2 19 26 9 0.17 0 3 7 26 29 29 1.00 33 0 29 10 0.78 0 3 21 26 29 30 0.59 0 3 7 19 29 11 0.81 0 4 19 21 26 29 31 0.48 0 3 7 26 29 12 0.31 0 4 7 19 29 35 32 0.21 0 4 7 19 26 29 13 0.32 0 4 7 19 26 29 33 0.54 0 4 7 19 26 29 14 0.67 0 3 7 26 29 34 0.24 0 3 7 19 26 15 0.63 0 5 7 19 26 29 35 1.00 10 0 35 16 0.88 0 5 7 19 26 29 36 0.33 0 4 7 19 26 29 17 0.68 0 4 7 19 26 29 37 0.77 0 3 7 19 29 18 0.72 0 4 7 19 29 35 38 0.48 0 4 7 19 26 29 19 1.00 20 0 19 39 0.44 0 3 7 26 29 20 0.30 0 3 21 26 29 40 0.22 0 3 7 26 29 MIN 0.17 MAX 1.00 MEAN 0.59 SD 0.27 no eff. Units 6.00

185

7.3.1.3 Results for TS3 “profitability”

The tables below hold the results for the stage “profitability”: