Hypotheses

To put our theoretical conjectures and developed proof-of-concept interactive system to the test, we experimentally test the following five hypotheses (Table 10).

151 # Hypothesis

H1 Positive, rather than negative emotion associates with augmented

creativity.

H2 Positive, rather than neutral or negative manipulation of feedback

presented by the interactive system augments creativity.

H3 Positive, rather than neutral or negative manipulation of feedback

presented by the interactive system causes positive emotion.

H4 Negative, rather than neutral or positive manipulation of feedback

presented by the interactive system causes negative emotion.

H5 Positive, rather than neutral or negative manipulation of feedback

presented by the interactive system augments creativity via its influence on the emotion-creativity link.

Table 10 Hypotheses for study 3.

6.5

Method

To test the hypotheses and thereby experimentally evaluate the interactive system, we used an experimental within-subject design, with each of the participants doing three creative tasks while being exposed the negative,

neutral, and positive feedback manipulations described above. The feedback

manipulations and the objects used in the creative tasks were randomized to prevent research bias.

6.5.1

Participants

In total, 49 people (25 women, 24 men, Mage=30, SDage=8.38) participated in

our study. Two participants guessed the purpose of the study and five people reported to have tried to game the interactive system by typing in bizarre ideas to gain high originality scores during one or more of the tasks. We removed these cases from further analysis to ensure that the possible

152

extraneous sources of variation they introduce did not influence testing the hypotheses, and thereby threaten the internal validity of the study. This resulted in 134 usable cases. All participants were students or employees of City University London.

6.5.2

Materials and measurements

6.5.2.1Creative tasks

To gather data based on which we could assess the pa ti ipa t s creative capabilities during idea generation, we again used the AUT (section 3.3.1). Participants we e i st u ted to … o e up ith as a , di erse, and

original uses for the common object as you can , within 4 minutes. See

section 4.5.2.1 for the rationale underlying these particular instructions. Participants used the interactive system to do the AUT three times, with the brick, paperclip, and knife as a subject. That is, the AUT subjects about hi h the i te a ti e s ste is a le to esti ate the o igi alit of people s ideas (Table 8). Presentation order was randomized. Note that the AUTs results are susceptible to learning effects, which can introduce an extraneous source of variation in the data (section 3.3.1). This is introduced by the use of a within-subject design. Although randomization might mitigate learning effects to some degree, it is unclear to what extent this occurs. Therefore, we need to accept this threat to validity.

6.5.2.2Assessment of creativity

We used the s ste s originality estimates to assess creativity (cf. section 3.3.2). Any idea scoring above the 75th _{rank, according to the} unmanipulated estimate calculated by the system, was counted as an original idea (26% of the ideas in this study). For each participant, we again

153

used the percentage scoring method. That is, we divided the number of original ideas by the total number of ideas generated during a task. See section 5.5.2.2 for the rationale underlying this correction. Note that use of the s ste s o igi alit esti ate might introduce measurement error. Despite the results of our pre-study, which show that the system estimates originality with a similar consistency as humans do, we do not know whether the systems estimates agree or disagree in the same way people agree or disagree, which could threaten construct validity. However, since the feed a k that is used to i flue e the pa ti ipa t s e otio s is ased o this auto ated o igi alit s o e, ot usi g it ould th eate the stud s internal validity. We chose to support the latter.

6.5.2.3Assessment of emotion

The participants used Likert scales with emotion words on opposite ends to self-report feelings of satisfaction (1=not satisfied, 9=very satisfied) and frustration (1=not frustrated, 9=very frustrated) they had experienced during the task (section 3.3.3). We assumed that these emotion words would reflect the type of negative and positive emotions typically associated with goal-conduciveness and goal-obstruction while pursuing a goal under time pressure in this way (cf. Roseman, 2011; Scherer, 2009). We assumed that this would make it easier for participants to recall their feelings after the tasks, and therefore help reduce measurement error. We also assessed positive and negative feelings separately, instead of as opposites on one scale, which we did in study 1 (section 4.5.2.3) and study 2 (section 5.5.2.3). This was changed because this allowed us to test the effects of positive and negative emotion on creativity separately, which better reflects empirical findings that show that positive emotion influences creativity (section 2.2.1.1), but negative emotion does not necessarily have any influence on

154

creativity, or has a diminishing influence on creativity, during idea generation (section 2.2.1.2). Further potential sources of measurement error were addressed in the way described previously in section 4.5.2.3.

6.5.2.4Manipulation checks

Several checks were carried out to support the internal validity of the study design (section 3.3.4). It is conceivable that the feedback manipulations ould ha e ade the s ste s esti ates less elie a le, athe than having the intended effects. To check whether the feedback manipulations in fact led to the intended influences on appraised originality of ideas, the participants used a Likert scale to rate their own creative performance after each task (1=worse, 9=better than expected), as well as how reliable the participants thought that the feedback was (1=very unreliable, 9=very reliable).

6.5.3

Procedure

Upon arrival the participants were seated at the computer and introduced to the study and its procedure. We used a cover story that informed the participants that we were testing ... the effi a of usi g o puter

supported idea e aluatio , but withheld information about the actual

experimental conditions until the end of the experiment. Informed consent was signed, and the participants filled in a brief questionnaire to collect personal data (age, gender). We then provided instructions for the tasks. That is, that the participants would do three AUTs during which our interactive system would provide feedback about the originality of each idea they came up with, provided further instructions needed to do the AUT, and e phasised that thei goal as to … o e up ith as a , di erse, a d

155

s ste s feedback we emphasized that participants should … use the

feed a k as a guide that helps ou duri g our idea ge eratio pro ess. A

picture of the common object used during each AUT was shown just before each task to i fo the pa ti ipa t s a out the t pe of alternative uses they were supposed to generate. After this participants did the AUT. That is, participants would type in their ideas, and during which they received manipulated feedback about the originality of their ideas each time they typed in an idea and pressed ENTER. Thus, attempting to manipulate their appraisal processes and subsequent emotional responses. Each task took exactly 4 minutes. These 4 minutes were timed internally by the interactive system, after which the system prevented the participant to typing in further ideas. The common objects used, and the feedback manipulations were randomised automatically by the interactive system. Right after each task ended the interactive system prompted a request to the participants to fill in a questionnaire. This questionnaire contained the measurement instruments used to assess emotion and carry out the manipulation checks. Throughout the tasks, the interactive system automatically logged the actual unmanipulated originality scores it computed for each idea, which were used to assess creativity. After the three tasks and questionnaires were finished the participants were debriefed. During this debrief the true purpose of the study was explained, and we asked the participants whether they had guessed this purpose, had tried to game the feedback by typing in bizarre ideas, or had problems using the system otherwise. To compensate the participants for their effort, we handed them a £5 voucher for a large online retailer, and a chocolate bar. A graphic representation of the procedure is presented in Figure 18.

156 In fo rm a ti o n F e e d b a ck m a n ip u la ti o n T a sk R a ti n g s Procedure Cover story Informed consent Personal details Instructions tasks Instructions feedback use T im e i n d ic a to r Feedback manipulation Task 1: AUT (4 min) Logged creativity Ass. Emotion Manip. Checks Feedback manipulation Task 2: AUT (4 min) Logged creativity Ass. Emotion Manip. Checks Feedback manipulation Task 3: AUT (4 min) Logged creativity Ass. Emotion Manip. Checks Debrief

Figure 18 Graphic representation of the order and timing of information, when the feed a k a ipulatio s ere used to i flue e the appraisals of the user’s o ideas, the

tasks performed, and the ratings used in the procedure.