Experiments and data collection

The purpose of this study is to investigate the different mechanisms in terms of their impact on the procurement process and outcomes. The two systems (Imaras and Imbins) are developed to implement the mechanisms and are used to conduct the transactions. The proposed hypotheses are tested with an experimental approach. The experimental design, tasks and procedure for data

collection are described in the following sections. 6.4.1 Experimental design

This study adopts an experimental approach that combined laboratory and online settings to test the research hypotheses. Laboratory experiments have been considered to better establish and test causal relationships by manipulating the independent variables, controlling the environment and procedure, and observing the effects on the dependent variables (Croson 2005; Kagel and Levin to appear). Online experiments have been widely conducted in e-commerce research, which replicate the natural environment and increase the external validity (Kerlinger and Lee 2000; Jap 2003; Bapna, Jank et al. 2008).

Table 6-1 presents the experimental design with two mechanisms in two settings, and thus four treatments were managed in this study.

Table 6-1. Experimental design and treatments (Study 3) Mechanisms Experiment 1

(Laboratory setting)

Experiment 2 (Online setting) Auction Treatment 1 (T1) Treatment 3 (T3) Negotiation Treatment 2 (T2) Treatment 4 (T4)

In Experiment 1, the auctions and negotiations were conducted in the laboratory environment with a number of sessions. Each session lasted two and half hour, including the time for preparation and questionnaires. The buyers in negotiations and the suppliers were set up in different lab rooms, where the facilitators gave instructions and guided them through the experiment. The time slot for bidding and negotiating was also controlled, which lasted 50 minutes. The auctions were in a multi- round setting; each round took five minutes. The bidders obtained the auction updates on the rounds and the revealed information immediately from their computer screen. The buyers and suppliers in negotiations were exchanging offers and/or messages through the system; and, the system automatically sent a notice to their screen for any updates on their negotiations.

In Experiment 2, the auctions and negotiations were running online and lasted 10 days. In auctions, each round was set as one day and thus results in maximum ten rounds. The participants

could log on and off at any time during the auction or negotiation. They were informed by emails about any updates from the ongoing auctions and negotiations, and they were required to log on to access the updates and make offers or bids within the systems.

6.4.2 Experimental tasks

The experimental task was a business case that involved contracting between a milk producer and several transportation service providers. Three attributes of the transportation service were concerned: (1) standard rate; (2) rush rate; and (3) penalty for delay. There were a number of values for each attribute, and the possible ranges were known to each participant. To maintain a relatively complex problem, there were 3,375 alternatives for the contract.

The buyer participants in negotiations represented a procurement officer for the milk producer. The supplier participants played the role of a sales manager for one of the service providers. They were competing with each other to win the contract. Each contract could be awarded to only one supplier. In auctions, the one who made the best bid for the milk producer would be the winner. In negotiations, they were negotiating with the buyer in order to reach an agreement. The service providers wanted to get the contract, while the contract should bring their business profits rather than losses. The participants were given the break-even points of the contract for their company.

The context and background provided in the general information document that was known to every supplier, while the preferences were explained in the confidential information document that was not known to the other suppliers. The reservation and aspiration levels of each company were also indicated in their confidential information, indicating the worst and best deals respectively. A financial calculator is implemented in the system, which can be used by the participants to calculate the revenue and profit for each contract alternative. Appendix D summarizes the business profiles and preferences of the companies involved in the transaction. Based on the preferences of the buyer and each supplier, the utility distribution of alternatives can be identified and used for calculating the

outcome measures.

6.4.3 Experimental procedure

The experimental procedure and the main activities are listed in Table E-1 in Appendix E, which involves several phases: registration, matchup, preparation, interaction and conclusion. Demographics, self-reported data and transaction process and outcomes were collected in different phases.

Before the experiment, the participants first signed up online and their demographical information was gathered via a registration form. The participants were then randomly matched up with different roles and assigned to one of the treatments. In Experiment 1, the participants were also asked to select the session based on their availability, and they were assigned into the chosen session during the experiment.

The participants were also required to watch a video demonstration of the system before they actually used it in the experiment. This was suggested from the participants’ feedback in previous study (Wu and Kersten 2013). It also ensured that the participants had the same baseline knowledge and experience with the system in order to test the effects due to treated conditions (Kumar and Benbasat 2006; Kamis, Koufaris et al. 2008).

During the experiment, the participants first read the general and confidential information for the preparation. This was followed by a case quiz and pre-questionnaire before the auction or negotiation started. Table F-1 in Appendix F shows a sample case quiz for one supplier, which was used to improve the participants’ understanding of the tasks. The participants were then asked to provide their perceptions of the task as well as their expected aspiration and reservation levels (Table G-1 in Appendix G). These measures were used to examine their understanding of the task and their expectations on the contract.

during the transaction. In the interaction phase, the participants constructed and submitted bids or offers on behalf of the companies they represented. In negotiations, they may also send messages with or without offer. Once the transaction was closed, they were asked to fill out a post- questionnaire to report their perceptions and evaluation of the process, outcomes and system (Table G-2 in Appendix G).

The participants’ activities during the transaction process were recorded in a database, which was used to analyze the transaction process and outcomes.