Experiment 2

3.4.1 Experimental Design

In addition to its difference as being a lab study, some other notable differences in experiment 2 from experiment 1 were:

1. Mood inducing videos from arousal quadrant were selected to induce moods as within-subject factor.

2. Some extra measures based on participants subjective observations were introduced in order to further understand the impact of mood on programmers. 3. In experiment 1 a neutral movie clip was used as a covariant factor. This

experiment was designed considering arousal as a within-subjects factor. The setup of the experiment was to measure only main effect and no interaction effect. The experiment was setup to measure effect of arousal on performance, but it was also possible to measure main effect for valence without the possibility of measuring the interaction effect between arousal and valence as not all conditions for a full 2x2 factorial design were included in the experiment. Therefore it was not a 2x2 factorial design. It consisted of arousal rating after watching high arousal and low arousal movie clips as an independent variable. The dependent variables used were: the numbers of correct answers, the average time in which debug questions were answered and the four perception questions.

3.4.2 Material and Procedure

For the experiment a windows application quite similar to web version was developed. The start included introduction to the experiment as well as consent form for participant. After obtaining the consent, a sample video was played followed by a couple of sample questions with multiple answers options. This was followed by a mood rating form including SAM (Self Assessment Manikin) for rating valence and arousal level. In addition, the following subjective questions were also included on the mood rating form.

1. I found this quiz easy

2. I think I made no errors in this quiz. 3. I think I performed well in this quiz.

4. I found that there was enough time to answer the quiz questions.

Participants were able to rate their opinion on a 10-point Likert scale ranging from ‗strongly agree‘ starting from 1 to ‗strongly disagree‘ representing by 10. Because of the training phase, participants understood what they could expect and thus eliminating the element of surprise. The sequence of experiment 2 was similar to experiment 1 as explained in Figure 3.2. As indicated earlier movies varying on arousal dimension and was therefore introduced as a within-subject factor. The movie

clips were displayed in a pre set format as is explained in the Table 3.3 below. To reduce the complexity, the experiment was primarily setup to study a main effect for arousal and secondarily a main effect for valence. Examining a potential 2-way interaction between these factors was not pursued.

Table 3.3: Preset format of Movie Sequence, each row indicating one complete test with a participant.

Quiz Movie Clip 1 Quiz Movie Clip 2 N

A HVHA B HVLA 2 B HVHA A LVLA 3 A LVLA B LVHA 3 B LVHA A HVLA 3 A HVLA B HVHA 3 B LVLA A HVHA 1 A LVHA B LVLA 3 B HVLA A LVHA 2 Total Participants 20

Note. Example movie clip remained the same for all participants, N = Number of participants that took the test in the sequence

In the experiment participants were asked to complete the debug tests used in the previous experiment. Like experiment 1 participants had an option of taking the test in their preferred programming language out of C++, C#, Java and Visual Basic. The result of the participants quiz was presented at the end, with the combination of correct answers from both quiz A and B.

3.4.3 Participants

Participants were invited for the experiment in two ways. Firstly the test application was packaged and sent to one of the participants via his email. This nominated person along with taking the test himself also organized the experiment to be conducted on his PC by 5 other participants as well. The rest of the 14 participants were contacted personally and took the experiment on the laptop of the experimenter. However instructions were presented to all of the participants in only one language (English) as well as all participants used the same English version of the application. A total of 20 participants participated in the study out of which there were only two female participants. The mean age of the participants‘ was 26.6 years with a standard deviation of 2.6 years. The mean experience of participants in the programming field was 2.9 years with a standard deviation of 2.7. About 30% participants took the tests in C#, 40% took the tests in C/C++, and 30% took the tests in Java. Ten percent

programmers were professionals, 40% were post-graduate students, 35% were undergraduate students and 15% were PhD students.

3.4.4 Results

The basic aim of the experiment was to observe the effect of arousal (high, low) induced by movie clips on the debug performance of the programmers and on their subjective observation. However the effect for valence (high, low) was also analyzed in the case of participants where participants watched varying low and high valence movie clip. The first analysis carried out was a MANOVA with as independent with-subject variable arousal with two levels (low arousal and high arousal). The dependent variables were: the numbers of correct answers, the average time in which debug questions were answered and the average score on the four opinion questions. The response on the four questions was taken together as Cronbach‘s alpha examination indicated a high level of consistency between the responses (low arousal condition α= 0.89, and high arousal condition α = 0.89). The multivariate analysis results (Table 3.4) showed a non-significant main effect (F (3, 17) = 1.13, p = 0.36) for arousal.

Table 3.4: Results multivariate and univariate analysis on arousal

Data Item Mean in

HA Mean in LA Hyp. Df Error Df F p Joint MANOVA 3 17 1.13 0.36

Total correct answers 3.30 2.65 1 19 3.77 0.07

Average time taken to answer the debug

question with in time 28.65 28.30 1 19 0.03 0.87

Combined subjective observations 5.92 5.80 1 19 0.24 0.63

Note: HA = High Arousal, LA = Low Arousal.

The univariate analysis (Table 3.4) showed non significant effect of arousal on the programmers‘ subjective observations of their performance. Similarly average time taken to answer questions also emerged to be non significant. The number of correct answers in high and low arousal condition was also not significant (F (1, 19) = 3.77, p = 0.067) however showed a trend toward significance with an average of 3.3 correct answers in the high arousal condition and an average of 2.65 correct answers in the low arousal condition.

A second MANOVA was carried out with as independent with-subject variable valence with two levels (low valence and high valence) on the 9 participants that were assigned to sequence (Table 3.2) in which the two video clips had different valence levels. The dependent variables were again the combined questions score, the

number of correct answers, and the average time in which a debug question was answered. The multivariate analysis results (Table 3.5) showed a non-significant main effect (F (3, 6) = 1.35, p = 0.42) for valence. Examining the results of the univariate analyses showed that valence had no significant effect on the number of correct answers (F (1, 8) = 3.77, p = 0.088) but have a tendency. Note that potential order and arousal effects were controlled by counterbalancing these two factors in the selected sequence in valence analysis.

Table 3.5: Results multivariate and univariate analysis on valence

Data Item Mean in

HA Mean in LA Hyp. Df Error df F p Joint MANOVA 3 5 1.35 0.36

Total correct answers 2.25 3.12 1 7 3.94 0.09

Average time taken to answer the

debug question with in time 7.37 8.29 1 7 0.13 0.73

Combined subjective

observations 5.31 5.15 1 7 0.61 0.46

Note: HA = High Arousal, LA = Low Arousal.