Pilot Tests on Task Structure Manipulation

The task structure manipulation was pilot-tested and iteratively refined over the course of three months. The same 60 part-time MBA students participated in the pilot­ testing of the experimental task and GDSS environment. The task materials were presented to the first group of 39 test participants and they were asked to complete the task according to the instructions. To assess the task structure manipulation, participants were asked, “Which of the following best describes how you made the decision on the task you have just finished?” (1) Each group member has the same information and the group works together as a team to reach the best solution; (2) Each group member has the same information and the group selects one best solution from an array of solutions championed by individual group members ; or (3) Each group member has different information and each group member contributes to reach the best solution. Of the 39 responses, only 2 were misclassified (see Table 4.4).

Table 4.4: Classification of Experimental Treatments on Task Structure

Experimental Treatments

Classification Additive Disjunctive Conjunctive Row Total

Additive 13 0 0 13 Disjunctive 0 12 1 13 Conjunctive 0 1 12 13 Column Total 13 13 13 39 Chi-Square (X2) df Significant 66.92 4 0.00001

The single-sample Chi-Square demonstrated a good fit between the experimental conditions (additive task, disjunctive task, and conjunctive task) and the participants’ perception of the task structure in the experiment (X2=66.92 df=4, pO.OOOOl). This pilot test demonstrated that participants had no problem in identifying the corresponding task structure they were assigned. They were also asked to comment on

the task materials and the manipulations of the task structures. As a result of this pilot- test, several sentences in the general instructions of task materials were reworded to improve their clarity. The modified task materials were then presented to a second group of 21 MBA students. Like the first group, they were also asked to complete the task according to the task instructions and identify the task structure of their decision task. Of the 21 responses, only 1 was misclassified (see Table 4.5). The single-sample chi-square demonstrated a good fit between the experimental conditions (additive task, disjunctive task, and conjunctive task) and the participants’ perception of the task structure in the experiment (X2=36.75 df=4, pO.OOOOl). This pilot test indicated that task manipulation appeared to have been successful.

Table 4.5: Classification of Pilot-Test Experimental Treatments on Task Structure

Experimental Treatments

Classification Additive Disjunctive Conjunctive Row Total

Additive 7 0 0 7 Disjunctive 0 7 0 7 Conjunctive 0 1 6 7 Column Total 7 6 8 21 Chi-Square (X2) df Significant 36.75 4 0.00001 4.7 Experimental Procedures

A group concurrently reported to a seminar room. The researcher introduced the experiment as a session for assessing the decision making skills of the participants. Then they were told that they would be participating in a simulation exercise and that they would be role-playing the part of a member of a strategic decision making team involved in selecting a new product manager from a set of available candidates.

Although the subjects understood that the experimental session was merely an exercise, they were strongly encouraged to treat the experiment as if it were a real situation within their company. As an incentive to do their best in the exercise, the researcher informed the subjects that individuals who performed well would receive a letter recording their success and they were encouraged to show this letter to their employers.

After the introduction of the experiment, subjects were asked to fill out a questionnaire which requested their personal demographics, educational background, work experience, computer knowledge and attitudes to and experience with group decision making.

Each subject was given an information packet that contained case material, an instruction sheet and candidate’s information sheets according to the experimental group to which they were assigned. All the groups were asked to follow the same general decision-making steps:

1. read and analyse the material individually, 2. discuss the material,

3. generate possible alternatives, 3. resolve any conflict,

4. make a group decision.

This created a structure for the decision making session but allowed free-flowing discussion.

Groups that were supported by GDSS were instructed to use whatever features of the system they thought might help the group decision making. The final decision was written on the decision forms provided. The time they began the experimental task was noted, but groups were not given a time limit for completing the task. They were asked to hand in their decision on the decision forms immediately after the decision

was again noted and the time they had reached a decision was marked on the decision sheet.

After the group handed in their decision sheet, individual group members were asked to fill out a set of questionnaires that asked about how they made decisions on the task they had just finished, and about the group’s decision making environment. These questions served as a check on the successfulness of the experimental manipulation. Individual group members were then asked to fill in two set of questionnaires about their perceived satisfaction with the group’s decision process and outcome. A fourth set of questionnaires were distributed to the GDSS group to measure the perceived usefulness of the system to the assigned task. In terms of questionnaire design, subjects in all six environments received the same items in the questionnaire. The questionnaires are shown in Appendix C.

Finally, after they had filled in the questionnaires, a debriefing session was held for each group. The session served to collect qualitative information regarding any questions, comments or feedback on the decision making session. The group were also asked to comment on the experiment as a whole.