Study 2: BUSA Game

Three surveys were administered to a group of undergraduate business students engaged in a computer-based business strategy game in a public U.S. university where each team

(composed of two to five people) representing a company competing in the shoe manufacturing market was assigned the task of developing and applying strategies to make the company successfully compete in the market and increase market share and profitability over time (where one week is considered one year of the company’s life). The game was a requirement for passing a strategy course.

Feedback on team performance was generated by complex algorithms embedded in the Business Strategy Game (BSG) and not by either the instructor of the course in which the game was a course requirement or the researcher. Each team’s objective results were calculated through the use of these algorithms where 50 percent of the score was based on investor expectations (which in turn was a function of the percentage of investor- determined target measures like return on equity (ROE), credit rating and stock price that was actually achieved) and 50 percent on a best-in-industry score (which in turn is a function of relative performance vis-à-vis the best company in the industry in terms of the same indices used in calculating the investor expectations like ROE, stock price and credit rating). The results were calculated and supplied to the teams after every round which is a week in time but represents a year in the business’s life.

The first survey was administered in the first week of the task (or game) and would measure control variables like age and gender as well as personality characteristics and baseline affect and attitudes. The second survey was given after the team and so individual respondents have been engaged in the task for five weeks, receiving feedback on performance every week; and the third survey was administered after ten weeks of being engaged in the game and in the final week of it. The second survey would measure the major outcomes in this study including

learning behavior, intentions to improve task-related skills, motivation to perform, goals,

expectations, and performance; moderators and mediators were not included except for learning effort due to limitations pertaining to survey length (since this effort was a collective research effort). Learning effort was included here in order to test whether like performance effort (same logic) it has any moderating effect on the relationship between trend and learning behavior.

Learning effort was included here but only in survey 3 (i.e. stage 3 after the 10 rounds) due to length limitations. Learning effort was included in this study because learning behavior could not be measured in study 1; only learning intentions were included in study 1 as explained earlier. Lack of performance control was also included in survey 2 and survey 3 as an outcome and not a mediator in order to test hypotheses 20c on the effect that time since first feedback has on the moderating effect of this feedback on the trend - lack of performance control relationship.

Even though it was originally planned that only two surveys will be distributed, three surveys were administered but survey 3 only included three outcomes: learning behavior, performance satisfaction and task satisfaction. Learning behavior was included in survey 3 because of the nature of the variable and how it was measured: it is more past-oriented than present or future oriented like most of the other main variables and measures included, i.e. it asks respondents to think about what they did while playing the game rather than about their present state or future intentions and plans. Task satisfaction and performance satisfaction were included in order to test the effect of trend and consistency on two of the main outcomes over a longer period of time and evaluate the possibility of different results the longer the time frame used in a study.

This study is a follow-up and supplementary study to study number 1 but is focused on the effects of trend and consistency in feedback while controlling for mean performance and initial valence of the feedback given. This study also addresses all the hypotheses relating the primary outcomes to performance on the team level. So all of the primary outcomes included in this study were averaged across the team in order to test the effect of these averages on team performance since study 1 did not include performance. Because learning behavior was not included in study1, current feedback (in lieu of initial valence in study 1) besides trend and consistency, was used to test for effects on learning behavior as well as the other primary outcomes in the study.

The advantage that this study has over the previous individual –level despite its relatively limited scope is that it tests the hypotheses pertaining to the relationship between the two new independent variables introduced here which are trend and consistency in feedback valence with important work outcomes in a more naturalistic setting that is more similar to an organizational setting in terms of the pressures met, the goals and expectations that need to be set and followed, and the context of competing teams and comparative feedback. Therefore, it should have more external validity than study 1 which can be expected to be higher in internal validity because of its more experimental and so controlled design. Having said however it is important to realize that the two studies are different in important aspects, e.g. the operationalization of trend (as explained later) and the fact that there is no zero inconsistency in the data -unlike in study 1. The difference in the operationalization of trend reflects the difference in methodology- in study1, feedback and task were repeated in very close succession to one another so trend was calculated as average of differences between subsequent feedback scores. On the other hand, in the second study, respondents were asked to think about five rounds together when answering most

questions and so trend was calculated as the slope in line over time. In other words, the two studies can be considered as complementary to one another.

Moreover, even though the feedback provided to each team was mostly objective in nature and included detailed reports and statistics in study 2, the main feedback depended on for this study was the overall score of each team in the round. This is because this is the simplest and most summarized form of information given in the data provided. Also, study 2 enables more valid measurement of effects on actual performance, performance changes and learning behavior than the controlled nature of study 1. Finally, as opposed to study 1 where valence, trend and inconsistency were manipulated and so assigned a value of -1, 0 or 1, in study 2, each of these variables were used in their raw form so numeric or calculated form rather than being assigned a general value that points to the general valence of the variable. The use of values here was meant to explore the effect of valence more specifically. Also, because teams are mainly competing with one another in the same section but not across sections on one hand but compare

performances informally with other teams in other sections due to the universal design of the game (and also because of the interest in the overall not team-based effects of feedback variables), both group-centering and grand-centering will be used to test the hypotheses.

In terms of the sample, in this study 2, which from now on would be referred to as the BUSA study, only 290 of the 349 people who participated in the game responded to the survey. Those respondents belonged to 8 sections and 88 teams; all sections and teams were represented in the data collected. Of the 59 cases not included in the analyses, 9 of the cases only participated in stage 2 of the survey but not stage 3 and another 25 of these cases only participated in stage 3 of the survey but not in stage 2 while the rest participated in none of the stages of data collection. However, of the 290 people who answered the survey, 112 had incomplete data.

In terms of the basic characteristics of the sample (the overall sample of 290

respondents), the average age was 25.6 (and so approximately 26) but age ranged from 19 to 54. In terms of race, there was a lot of missing data, but from amongst the 212 who answered this question, 100 were white, 75 were black, 21 were of Hispanic ethnicity (6 of those also identified themselves as white race) and the rest were of other ethnicities. In terms of job experience, the average in the sample was approximately 11 years; general experience in this study was

measured in terms of total of the years of experience in different business fields like marketing, distribution, finance and so on and not in terms of total number of years of actual job experience (and so for example a 40-year old respondent many have a total of 70 years of experience). GPA ranged from 2.5 to 4.3 with an average of 3.3. Furthermore, in terms of gender, of the 284 participants who answered this question, 135 were male and the rest were female (149).