Experiment 1: Project selection, effort and overconfidence

This experiment tries to provide a context in which investment project selection and effort level decisions of subjects can be observed. In addition, by applying the concept of skill-rank dependent success probabilities (Camerer et al., 1999), individual confidence levels are simultaneously observed. In this set-up, subjects' expected pay-offs from their investment choices are dependent on self-assessed (relative) ability and knowledge. Another source of inspiration in the design of the experiment were Heath

& Petersen (1996) who induce uncertainty using a random event for which they determine only ex-post which of two probability distributions will apply to the evaluation of the different strategies. The experimental subjects thus know the space of permissible probability distributions, but do not know which one will apply. In my experiment, it is each individual participant's rank based on performance in a simple general knowledge (pub) quiz that determines which of the two probability distributions will be used. Uncertainty about the probability of success is thus linked to uncertainty about own relative knowledge and performance, and ultimately therefore to individual confidence.

Hence, in this experiment, and consistent with the literature in psychology, it is uncertainty that causes the interference of psychological biases. The analysis of the model has shown that overconfidence may cause excessive effort, since overconfident individuals overestimate the benefits from their effort contribution relative to the cost

Chapter 4: Data collection methods

of effort. In addition, the model predicts that overconfidence should be associated with overinvestment in the sense that overconfident individuals invest in projects that have a negative NPV. The aim of this experiment is to investigate these two predictions empirically by exploring whether the hypothesised behaviour can be observed in the laboratory, and if it can be shown to be associated with measures of individual confidence. The independent variable in this experiment is thus individual confidence, and the dependent variables are the number of projects accepted by a participant (project selection behaviour) and the level of effort. Individual confidence is measured using ten calibration questions, as well as using the accuracy of self-estimated performance in relative and absolute terms. Effort is for the benefit of simplicity limited to a binary variable, hence participants exert effort, or they do not. The procedure of the experiment is described in sequence below.

At the beginning of the experimental session, subjects are assured that the experiment does not affect the marks or grades for their degree course and that individual responses would be treated confidentially by the experimenter. In addition, subjects are told that the ten best performing participants will receive prizes. The experiment is then introduced as consisting of the following two parts. In a first step, subjects complete the 'test questionnaire'. They begin by responding to twenty questions in a general knowledge quiz. Subjects are told that their performance in the test will affect the probability with which their investment decisions later on would be successful.

This feature represents the idea of rank-dependent success probabilities (Camerer et al., 1999).

Out of the twenty questions, however, only ten are actual quiz questions. They comprise multiple-choice general knowledge questions and simple calculus exercises.

The remaining ten questions of the quiz are confidence interval questions; they were included to assess individual overconfidence scores but this is not disclosed to participants. Following the quiz and the confidence interval questions, subjects are asked to evaluate their own performance in the twenty knowledge questions in absolute and in relative terms. In addition, two simple gambles (Figure 4.2) are proposed as a way of accounting for individual risk preferences.

Capital investment decisions with managerial overconfidence and regret aversion

Figure 4.2: Risk preference questions

Q4.4 Please indicate which option you would prefer in each case

(i) 500 pounds for sure OR a 45% chance of winning 1,000 and nothing otherwise (ii) 500 pounds for sure OR a 55% chance of winning 1,000 and nothing otherwise

The idea behind these questions is that respondents who prefer the safety equivalent in both cases can be classified as risk-averse, and respondents who choose to gamble in both cases as risk loving. Someone who prefers the certain cash payment in the first choice pair but chooses to enter the gamble in the second proposal is assumed here to be weakly risk-averse or risk-neutral. The final questions in this first section of the experiment record a respondent’s gender and email address (for contacting the prize winners). Fifteen minutes are allocated for this first part of the experiment.

The second part of the experiment contains the actual investment decision task for which subjects have to make an investment decision in combination with an effort decision for a given budget. Another fifteen minutes is allocated for this second part of the experiment. Responses are recorded on the experimental handout, which is collected at the end of the experiment. Subjects are reminded that in order to qualify for a prize, this second section has to be properly completed. The basic structure of the investment task is as follows:

• Subjects initially dispose of a virtual budget of 10 monetary units (MU). In the first period, subjects make two interdependent decisions. For one, subjects must decide how much of their budget they wish to invest in the uncertain projects. Each project requires an investment of 2 MU, so that subjects can invest in at most five of the investment projects⁵⁸ . However, subjects are also given the option to expend 2 MU as

‘effort’ to improve the probability with which the projects will be successful. Given the budget constraint, making this payment reduces the amount disposable for investment so that only four projects can be invested in if effort is chosen.

• If a subject does not make any investments in the first period, and does not pay for

‘effort’, his final wealth position at the end of the game will simply be his original

58 Projects are identical.

Chapter 4: Data collection methods

budget of 10 MU. In contrast, if a subject invests in one or more projects, his final wealth depends on the cash flows from the investment(s). These, in turn, depend on which state of nature will occur in the second period.

• The realisation of the state of nature depends on performance in the ten knowledge questions, and whether a participant purchased the ‘effort’ option in period one. In terms of test performance, I distinguish between the top quartile and the rest, whereby the success probability for the top quartile is greater. As for the effort choice, the probability of success is greater when effort was bought. The resulting four possible probability distributions, illustrated in Figure 4.3 below, are available to participants in the experiment already in period one. Since subjects can only estimate their individual ranking in the test, they are uncertain which probability distribution applies for them⁵⁹.

• In the good state, each project delivers a cash flow of 4 MU. Under the bad state of nature, each project fails and results in a negative cash flow of (-1) MU. To facilitate subjects’ decision-making, payout information is stated in a table in terms of net present values per number of projects chosen. For illustration, investing in two projects would have a bad-state NPV of (-6) MU and a good-state NPV of 4 MU.

Assuming that subjects will base their investment decision on the expected net present values for the different choice options, they will need to estimate which probability distribution is most likely to apply for them. Hence, in this experiment individual investment decisions depend to a large extent on how well subjects believe they fared in the quiz compared to all the other experimental candidates. Consequently, the observable decision behaviour (effort choice, number of projects invested in) should depend on an individual’s level of confidence. Hereby, individual confidence is measured using two variables. The use of confidence intervals in measuring overconfidence was already introduced in the presentation of the survey questions.

59 Uncertainty in an investment game is also induced in this way by Heath et al. (1996).

Figure 4.3: Probability of success for different groups of subjects

No effort Effort

Top-quartile 0.50 0.83

Rest 0.33 0.50

Capital investment decisions with managerial overconfidence and regret aversion

Now, in this experiment, subjects respond to ten instead of five questions, and are set an 80% confidence level. As a result, the first measure of individual confidence in this experiment is

In addition, having completed the quiz section, subjects are asked to assess their performance in relative and in absolute terms. These assessments can then be compared against actual performance to determine the accuracy of self-perceived performance. Someone who thinks he has done well while in fact he has not will be deemed to be overconfident, and underconfident in the opposite case. The second measure of confidence is hence calculated as the mean level of perceptive accuracy for relative and absolute performance.

y _, : Relative performance scaled from well below average (-2) to well above average (+2), estimated and actual

For the neutral point (no over- or under-confidence), each of the measures takes a value of zero. Positive values indicate overconfidence, with the maximum level at (+1); negative values indicate under-confidence, with the minimum level of confidence at (-1). While the overall experimental design is aimed at testing the effect of overconfidence on project selection and effort in general, determining individual confidence should – assuming there are individual differences in behaviour – enable a finer analysis, particularly for analysing different degrees of potential over-investment.