Methods for studying emotional experiences 23

Experiences of the learning environment are retrospective generalizations of how a student sees his or her interaction with the surroundings and personal goals reflect a student’s orientation towards future possibilities. Goals and dispositions may also guide what kinds of emotions are triggered (Ketonen & Lonka, 2013), but emotio- nal experiences take place in the present, at a specific context and time, and they change accordingly. A great part of research focusing on emotions has been con- ducted either in laboratory settings using deductive, quantitative and experimental approaches (Ainley et al., 2005; Mello, Clarridge, & Studdert, 2005; Schutz & DeCuir, 2002). In many studies the participants are exposed to either positive or negative emotional experiences and results for cognitive or academic tasks are thereafter studied (Efklides & Petkaki, 2005; Meinhardt & Pekrun, 2003). While these methods give rather accurate results of the effects of different emotions, they do not shed light on how emotions appear and are experienced in learning situa- tions. Experiences do leave a memory trace and many studies ask participants to list emotions they have felt during “the past few days”(Brunstein, 1993; Buff et al., 2011) or how they usually feel in a specific situation (Goetz et al., 2006; Putwain et al., 2013; Ruthig et al., 2008). However, people tend to use their current experi- ences as a basis for reconstructing past experiences, and this brings about a bias between recalling how and experience felt and what the actual experience was (Shiffman, 2000; Stone, Shiffman, Atienza, & Nebeling, 2007). Therefore, the current affective state often colours the past.

Many studies shorten the time between a learning or test period and the ques- tionnaire by distributing questionnaires immediately after the activity (Järvenoja & Järvelä, 2009; Linnenbrink-Garcia et al., 2011) or they combine different qualita- tive and quantitative methods (Buff et al., 2011; Do & Schallert, 2004). As no sin- gle category of method is suited to answer all questions, a wide variety of ap- proaches should be encompassed to capture the richness of emotions in educational settings. However, very few studies have managed to study emotions in real life settings during the actual learning activities. Pekrun and Schutz (2007) emphasize the need for experiments in field settings and educational intervention research targeting emotions.

Everyday experience methods are approaches where the participants are asked about an ongoing activity in their natural environment (Greenwald & Banaji, 1995; Ross, 1989; Shiffman, 2000). The participant is usually signalled to complete a short questionnaire about his or her current experience. These methods are also referred to as Ecological Momentary Assessment (EMA) (Stone et al., 2007) or Experience Sampling Method (ESM) (Csikszentmihalyi & Larson, 1987). Al- though they are a form of self-report, they lack many of the shortcomings of tradi- tional methods (Barrett & Barrett, 2001; Reis & Gable, 2000). They do not rely on memory, a need for aggregating one’s usual feelings or an artificial context. Instead participants are asked to provide descriptions of their current thoughts, feelings, and behaviours in the situations they encounter in their daily lives, thereby allow- ing researchers to sample a range of variables in different environments. Bringing the instrument into the actual activity offers an accurate and multifaceted portrait of experience in its natural context. However, the complexity of the situation and the difficulty to interpret observations may result in loss of construct validity. That is, it is difficult to say whether a measurement in a study situation measures emotions related to the learning task at hand or to an argument with a friend an hour earlier. Therefore, causal conclusions should be made only after articulating and ruling out possible explanations for results (West & Hepworth, 1991). On the other hand, the possibility to aggregate across multiple occasions and contexts reduces other forms of bias, such as retrospective bias and the need to aggregate across multiple situa- tions (Epstein, 1979). Everyday experience methods allow one to acquire know- ledge not easily obtained with other techniques. They are a promising way to com- plement standard research strategies.

When studying everyday experiences, researchers have used different instru- ments (for review see Bolger, Davis, & Rafaeli, 2003), such as telephone inter- views at the end of each day (Almeida, 2007), diaries using pen and paper (Al- meida, Wethington, & Chandler, 1999), which may be accompanied with a timed beeper to alarm one to complete a questionnaire (Csikszentmihalyi & Larson, 1987; Fave & Massimini, 2005; Moneta & Csikszentmihalyi, 1999). However, pen and paper methods are easy to fake by completing the diary after the fact (Shiff- man, 2000). Fortunately, mobile devices offer a convenient way to carry out the measurement. Mobile devices are especially promising, because they offer several major advantages: time can be recorded to verify compliance with the sampling scheme and data can be uploaded automatically (Barrett & Barrett, 2001). In Study III, participants’ learning experiences were collected with the Contextual Activity

Sampling System (CASS) (Muukkonen et al., 2007; 2008). This research instru- ment has been designed to collect frequent and systematic data on ongoing educa- tional and professional activities. The CASS Query tool is a Java application that runs on 3G mobile phones with a Symbian operating system. It sends queries to participants’ mobile phones, after which the answers are sent to a server database.

Reis and Gable (2000) conclude that there are three general purposes that can be used for everyday experience studies: establishing the commonness or qualities of a phenomena, testing a theoretically generated hypothesis, and serving as a dis- covery technique. In Study III, which focused on student teachers’ experiences on two two-week periods, all of these goals were pursued. The frequency of situations in which students were studying was compared during two different study periods (lecture-based and inquiry-based period). A hypothesis that students’ emotional experiences would differ during the two periods was tested. Thirdly, a time series analysis aggregating emotional experiences at a group level was used to follow and explore the student group’s process.

Experience sampling can occur at regular intervals (interval contingent), in re- sponse to events of interest (event contingent), or randomly throughout the day (signal contingent), which are the three possibilities for sampling strategies (Wheeler & Reis, 1991). In interval contingent recording, participants report their experiences at regular, predetermined intervals. These may be at the end of each day or every four hours. In signal contingent recording, a signal or a vibration from an electronic device prompts the participant to describe current activity at a random or a fixed schedule. In event-contingent recording, participants report whenever a predetermined event has occurred, such as lying or smoking. The selection of a protocol is made based on a research goal and the nature of the phenomenon under study. In Study III, a signal contingent protocol with a three-hour fixed interval was used. This made it possible to compare different domains of activity and men- tal states during different activities.

The multifaceted nature of everyday experience data lends itself well to so- phisticated methods, such as multilevel modelling or within-person regressions (Inkinen et al., 2013; Tolvanen et al., 2011; West & Hepworth, 1991). The many levels include such aspects as the day of week, time of day, place, activity and par- ticipant’s milieu. Reis and Gable (2000) suggest taking into consideration at least two levels: aggregating across different situations, such as work or leisure and tak- ing into consideration between-person variability. In Study III, this was accom- plished by differentiating between those situations in which the participants were studying or when they were not. Moreover, the between-person variability was controlled by standardising observations on a within-person level in order to elimi- nate differences in answer tendencies when making a group level analysis. Al- though time and temporal issues are present in everyday data sets, they are rarely addressed explicitly (Reis & Gable, 2000). West and Hepworth (1991) list different strategies for contending with and identifying serial dependency, such as analysing developmental trends or cyclical mood changes during weekdays In Study III, a simple group level aggregation of data with a temporal sliding mean was used to create a figure accompanied with identifying instances where the students were working collaboratively.

Reis and Gable (Reis & Gable, 2000) emphasized, that when using everyday experience methods the researcher should be careful with the accuracy of the data. The data must be inspected thoroughly and the investigator should rely on multiple strategies before making conclusions. Special focus should be paid on outliers, heteroscedasticity, correlated errors and univariate and multivariate distributions and descriptions of the data handling (Stone & Shiffman, 2002).

Overall, everyday experience methods offer a promising way to supplement traditional research methods or to open new lines of inquiry. In addition to present- ing challenges for data analysis, collecting the data is also resource intensive and technical.