Contributions to Management Science
For further volumes:
Jella Pfeiffer
Interactive Decision Aids
in E-Commerce
Jella Pfeiffer
Johannes Gutenberg-Universit¨at Mainz Lehrstuhl f¨ur Wirtschaftsinformatik und BWL Jakob-Welder Weg 9 55128 Mainz Germany [email protected] ISSN 1431-1941 ISBN 978-3-7908-2768-2 e-ISBN 978-3-7908-2769-9 DOI 10.1007/978-3-7908-2769-9
Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011942413 © Springer-Verlag Berlin Heidelberg 2012
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law.
The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.
Printed on acid-free paper
Physica-Verlag is a brand of Springer
Acknowledgements
At the beginning of my doctorate, I had not anticipated that it would connect me to many people, bring me to many countries, and would follow a very evolutionary process inspiring me not to leave the scientific path. I am grateful for this interesting part of my life and I would like to express my thanks to some special people who have supported me throughout these past years.
First and foremost, I am very grateful to my supervisor Dr. Franz Rothlauf. Franz, you have always supported me no matter how complicated my ideas were and how far they carried me. Furthermore, you have taught me a lot about scientific writing and how to structure my thoughts, and I have the feeling that I will never stop learning from you. Whenever I needed your advice, you were there for me. You have quite simply played the role of “PhD Supervisor” – particularly in the German sense of the word – perfectly! But besides all that, I appreciate your friendship very much.
My second acknowledgment goes to Dr. Ulrich Hoffrage, who was very kind to serve as the second reviewer in the committee. His profound and extraordinary knowledge in the field of decision-making is remarkable. I am very happy to be able to share and discuss ideas with Ulrich and I am very much looking forward to do further research with him in the future.
Most importantly, I would like to say many thanks to my family for all their help and support. They have showed me the importance of being optimistic and earnest, and they have taught me to be curious about the world.
Next, I want to thank all my colleagues from my department for wonderful discussions and their immense support during these last few years. In particular, I would like to thank Heike Kirsch for all her help and support. I would also like to thank Dr. Daniel Schunk, who was the first to inspire me to undertake a research project during my studies in Mannheim, and who has given me valuable feedback on my present work. Daniel, I still remember endless hours of discussing and generating ideas well into the evening in the SFB 504 building. Similarly, I am also grateful to Dr. Martin Meißner, from whom I learned about preference measurement and with whom I had great discussions. Furthermore, I would like to thank Dr. Ren´e Riedl for his good ideas and cooperation in several projects, as
viii Acknowledgements
well as Dr. Eduard Brandst¨atter for very inspiring discussions on decision-making behavior. Moreover, I would like to thank Felix Vogel who helped me a lot with implementing INTACMATO, and Melanie Bloos as well as my brother Thies Pfeiffer who both proofread parts of my dissertation.
I would like to express my thanks to those who participated in the studies and to several students who helped me to conduct the studies. The successful completion of my research is directly related to your support.
One very big thank you goes to Eric Bonabeau, Ph.D., from Icosystem Corpo-ration. He is such an inspiring man, and has made many things possible for me, including allowing me to be part of a company, where both the science-world and the real-world go closely hand-in-hand. I also thank Dejan Duzevik from Icosystem, who is as excited as I am about decision-making behavior and with whom I had great discussions.
I would like to thank all my friends who were always there for me. My close friends, Silke, Susanne and my old friends from Mannheim, the friends I have found among my colleagues, and the friends I have found abroad during my time at Icosystem and Harvard in Cambridge (USA). Last but not least, I would like to thank Mine, who has not been discouraged despite coming into my life during the final and most strenuous stage of my doctorate.
Contents
1 Introduction.. . . 1
1.1 Motivation. . . 1
1.2 Research Question and Contribution. . . 4
1.3 Method.. . . 6
1.4 Structure. . . 9
Part I Analysis of Decision-Making Behavior 2 Fundamentals on Decision-Making Behavior. . . 15
2.1 Choice Tasks and Preferences . . . 15
2.2 Decision Strategies. . . 17
2.2.1 Characteristics. . . 18
2.2.2 Types. . . 20
2.3 Measuring Decision-Making Behavior. . . 22
2.3.1 Outcome-Based Approach.. . . 23
2.3.2 Process Tracing.. . . 25
2.4 Complexity of Choice Tasks. . . 30
2.4.1 Task-Based Versus Context-Based Complexity. . . 30
2.4.2 Variables for Describing Decision-Making Behavior. . . 35
2.4.3 Influence of Task-Based Complexity. . . 37
2.4.4 Influence of Context-Based Complexity. . . 38
2.4.5 Discussion. . . 43
2.5 Conclusions.. . . 45
3 The Influence of Context-Based Complexity on Decision Processes. . . 47
3.1 Theory and Hypotheses. . . 47
3.2 Operationalization.. . . 53
3.3 Experiment.. . . 55
x Contents
3.3.1 Participants. . . 55
3.3.2 Design and Procedure.. . . 55
3.3.3 Eye Tracking. . . 56
3.3.4 Empirical Results . . . 57
3.4 Conclusions.. . . 61
3.4.1 Discussion and Contributions . . . 61
3.4.2 Limitations and Future Work. . . 63
4 The Influence of Task and Context-Based Complexity on the Final Choice. . . 65
4.1 Theory and Hypotheses. . . 66
4.2 Method.. . . 69
4.2.1 Formulation of the Experimental Design as Optimization Problem.. . . 69
4.2.2 A Genetic Algorithm for Finding Robust Choice Tasks with Optimal Mapping. . . 72
4.2.3 Evaluation of the Genetic Algorithm.. . . 75
4.3 Experiment.. . . 79
4.3.1 Participants. . . 79
4.3.2 Design and Procedure.. . . 79
4.3.3 Empirical Results . . . 80
4.4 Conclusions.. . . 86
4.4.1 Discussion and Contributions . . . 86
4.4.2 Limitations and Future Work. . . 87
Part II Decision Support with Interactive Decision Aids 5 Interactive Decision Aids. . . 93
5.1 Types.. . . 94
5.1.1 Recommendation Systems. . . 94
5.1.2 Interactive Information Management Tools (IIMT). . . 96
5.2 Comparison of Recommendation Systems & IIMT. . . 102
5.2.1 Theory and Hypotheses.. . . 102
5.2.2 Experiment. . . 105
5.3 Conclusions.. . . 109
6 INTACMATO: An IIMT-Prototype. . . 111
6.1 Requirements from Information Systems Research . . . 111
6.2 Requirements from Decision-Making Behavior Research. . . 114
6.3 Design of INTACMATO. . . 115
6.4 Qualitative Evaluation of INTACMATO. . . 122
6.4.1 Study 1: Brainstorming with Experts . . . 122
6.4.2 Study 2: Thinking Aloud . . . 123
Contents xi
7 An Effort-Accuracy Framework for IIMT. . . 127
7.1 The Effort-Accuracy Framework by Johnson and Payne (1985).. . . 127
7.1.1 Measurements for Effort and Accuracy.. . . 128
7.1.2 Elementary Information Processes. . . 128
7.2 Extended Effort-Accuracy Framework.. . . 130
7.2.1 Elementary Communication Processes . . . 132
7.2.2 Model Assumptions. . . 132
7.2.3 Related Work. . . 133
7.2.4 Application to IIMT-Prototype. . . 134
7.2.5 Results and Evaluation.. . . 141
7.3 Conclusions.. . . 145
8 Quantitative Evaluation of INTACMATO. . . 147
8.1 Theory and Hypotheses. . . 149
8.2 Operationalization.. . . 152
8.2.1 User Evaluation. . . 152
8.2.2 Design Criteria. . . 153
8.2.3 Determination of Strategies. . . 153
8.2.4 Measuring the Process with Clickstream Analysis. . . 154
8.2.5 Complexity. . . 154
8.3 Experiment.. . . 156
8.3.1 Participants. . . 156
8.3.2 Design and Procedure.. . . 156
8.3.3 Data Cleansing. . . 159
8.3.4 Empirical Results . . . 160
8.4 Conclusions.. . . 177
8.4.1 Discussion and Contributions . . . 177
8.4.2 Limitations and Future Work. . . 178
9 Summary, Conclusions, and Future Work . . . 181
9.1 Summary and Discussion. . . 181
9.2 Implications for Web Stores . . . 184
9.2.1 Usefulness of IIMT. . . 184
9.2.2 Separation into Screening and In-Depth Phase. . . 185
9.2.3 Offering a Variety of IIMT in the In-Depth Comparison Phase. . . 186
9.2.4 Adapting the Set of IIMT to Complexity . . . 186
9.3 Future Work . . . 188
A Details on Decision Strategies.. . . 191
B Details on IIMT-Prototype. . . 217
C Details on Empirical Studies. . . 225
References. . . 235
List of Figures
Fig. 1.1 Rigor-, relevance-, and build-cycle.. . . 7
Fig. 2.1 Choice process. . . 16
Fig. 2.2 Value functions.. . . 18
Fig. 3.1 Example of a choice task. . . 48
Fig. 3.2 Experimental procedure. . . 56
Fig. 3.3 Effect of SM on information acquisition (PCPM). . . 60
Fig. 3.4 Effect of SM on information acquisition (CBC). . . 60
Fig. 4.1 Two mappings of alternatives to decision strategies.. . . 70
Fig. 4.2 Construction of a choice task from a solution string. . . 73
Fig. 4.3 Controlling for the attribute range.. . . 77
Fig. 4.4 Controlling for the attractiveness difference.. . . 77
Fig. 4.5 Controlling for the correlation of attribute vectors. . . 78
Fig. 4.6 Influence of the independent variables on the relative frequency of the observed strategies.. . . 81
Fig. 4.7 Interaction effects for MAJ. . . 85
Fig. 4.8 Interaction effects for EBA. . . 85
Fig. 5.1 Welcome screen on myproductadvisor.com. . . 95
Fig. 5.2 Types of interactive decision aids. . . 95
Fig. 5.3 Product-comparison matrix on cdw.com.. . . 96
Fig. 5.4 Screening phase with IIMT on cdw.com.. . . 97
Fig. 5.5 Screening phase without IIMT on panasonic.com . . . 97
Fig. 5.6 Product-comparison matrix on myproductadvisor.com. . . 99
Fig. 5.7 Industries among the 100 analyzed websites. . . 101
Fig. 5.8 Websites per industry offering a product-comparison matrix.. . . 102
Fig. 5.9 Attribute importance dialog on myproductadvisor.com.. . . 106
Fig. 5.10 Filter on myproductadvisor.com. . . 106
xiv List of Figures
Fig. 5.11 User navigation on myproductadvisor.com.. . . 107
Fig. 5.12 Results of Study: IIMT vs. RA. . . 109
Fig. 6.1 IIMT: REMOVE. . . 118
Fig. 6.2 IIMT: PAIRWISE COMPARISON and MARKd iff er e nces. . . 119
Fig. 6.3 IIMT: MARK and REMOVEmar ked. . . 119
Fig. 6.4 Choice process. . . 120
Fig. 6.5 IIMT: SORT and FILTER. . . 121
Fig. 6.6 First IIMT-prototype. . . 122
Fig. 6.7 Results from the thinking aloud usability study. . . 124
Fig. 6.8 Final design of INTACMATO. The tooltip explains the IIMT functionality. . . 125
Fig. 7.1 Positions of decision strategies in the effort-accuracy space. . . 130
Fig. 7.2 The technical environment as additional factor influencing decision making. . . 131
Fig. 7.3 Effort for WADD without support of IIMT. . . 136
Fig. 7.4 Effort for WADD with support of IIMT. . . 137
Fig. 7.5 Effort for EBA without support of IIMT. . . 138
Fig. 7.6 Effort for EBA with support of IIMT. . . 139
Fig. 7.7 Effort for LEX without support of IIMT. . . 140
Fig. 7.8 Effort for LEX with support of IIMT. . . 141
Fig. 7.9 The extended effort-accuracy framework.. . . 145
Fig. 8.1 Experimental Design. . . 157
Fig. 8.2 Screenshot of webstores for group few IIMT. . . 158
Fig. 8.3 Screenshot of webstore for group no IIMT . . . 158
Fig. 8.4 Evaluation of design criteria. . . 161
Fig. 8.5 Results of hypothesis testing . . . 163
Fig. 8.6 Mean of clicks for group all IIMT . . . 167
Fig. 8.7 IIMT used at least once for group all IIMT. . . 168
Fig. 8.8 Mean of clicks for group few IIMT . . . 169
Fig. 8.9 IIMT used at least once for group few IIMT. . . 170
Fig. 8.10 Applied strategies (analysis of final choices). . . 172
Fig. 9.1 Number of decision strategies which are supported by each decision aid. . . 187
Fig. 9.2 Cumulative number of decision strategies which are supported by each decision aid. . . 187
Fig. A.1 Characteristics of decision strategies. . . 192
Fig. A.2 Effort for ADD without support of IIMT. . . 198
Fig. A.3 Effort for ADD with support of IIMT. . . 199
List of Figures xv
Fig. A.5 Effort for COM with support of IIMT. . . 201
Fig. A.6 Effort for CONJ without support of IIMT. . . 201
Fig. A.7 Effort for CONJ with support of IIMT. . . 202
Fig. A.8 Effort for DIS without support of IIMT. . . 202
Fig. A.9 Effort for DIS with support of IIMT. . . 203
Fig. A.10 Effort for DOM without support of IIMT. . . 204
Fig. A.11 Effort for DOM with support of IIMT. . . 205
Fig. A.12 Effort for EQW without support of IIMT. . . 206
Fig. A.13 Effort for EQW with support of IIMT . . . 206
Fig. A.14 Effort for FRQ without support of IIMT. . . 207
Fig. A.15 Effort for MAJ without support of IIMT. . . 208
Fig. A.16 Effort for FRQ with support of IIMT. . . 209
Fig. A.17 Effort for MAJ with support of IIMT. . . 209
Fig. A.18 Effort for MCD without support of IIMT. . . 210
Fig. A.19 Effort for MCD with support of IIMT . . . 211
Fig. A.20 Effort for LED without support of IIMT. . . 212
Fig. A.21 Effort for LED with support of IIMT. . . 213
Fig. A.22 Effort for SAT without support of IIMT. . . 213
Fig. A.23 Effort for SAT with support of IIMT. . . 214
Fig. A.24 Effort for SAT+ without support of IIMT. . . 214
Fig. A.25 Effort for SAT+ with support of IIMT. . . 215
Fig. B.1 Final design of IIMT: FILTER. . . 218
Fig. B.2 Final design of IIMT: MARKdiff. . . 219
Fig. B.3 Final design of IIMT: MARKmanually(positive). . . 220
Fig. B.4 Final design of IIMT: MARKmanually(negative).. . . 221
Fig. B.5 Final design of IIMT: SORThierarchically. . . 222
Fig. B.6 Final design of IIMT: SORTmanually. . . 223
List of Tables
Table 1.1 Guidelines for the design-science approach.. . . 8
Table 2.1 Product-comparison matrix. . . 17
Table 2.2 Examples of correlation of attribute vectors. . . 33
Table 2.3 Attribute ranges and attractiveness differences. . . 35
Table 3.1 Design of preference measurement used in CBC and PCPM. . . 57
Table 3.2 SM for the three stages. . . 58
Table 3.3 Alternatives with at least one fixation. . . 58
Table 3.4 Correlations between measures of complexity and breadth and depth of search. . . 59
Table 4.1 Attributes and attribute levels used in the experiment. . . 69
Table 4.2 Quality of solutions found by GA versus randomly generated choice tasks (mean and standard deviation). . . 76
Table 4.3 Correlations of complexity measures. . . 78
Table 4.4 Observed versus expected usage of strategies. . . 80
Table 4.5 Wilcoxon tests on the relative frequencies for task-based measurements. . . 83
Table 4.6 Wilcoxon tests on the relative frequency for the context-based measurements.. . . 83
Table 4.7 Binary logit models testing interaction effects between AR and AC, AD and AC, respectively.. . . 84
Table 4.8 Relative frequencies of the observed strategies for different combinations of AR and AC . . . 85
Table 4.9 Results of cluster analysis. . . 86
Table 5.1 Number of products in the product-comparison matrices. . . 100
Table 6.1 List of IIMT implemented in INTACMATO. . . 117
xviii List of Tables
Table 7.1 Elementary Information Processes.. . . 130
Table 7.2 Elementary Communication Processes. . . 132
Table 7.3 Effort-reduction when using IIMT . . . 142
Table 8.1 Example of a pairwise Hamming distance of two. . . 155
Table 8.2 Mean values for evaluation criteria. . . 163
Table 8.3 Results of hypotheses tests with effect sizes. . . 164
Table 8.4 Frequencies of strategies which explain choices.. . . 172
Table 8.5 Occurrences of mixed strategies. . . 176
Table A.1 Decision strategies and alternative name conventions.. . . 191
Table A.2 Summary of studies on choice task complexity.. . . 193
Table C.1 Pages with highest Google PageRank.. . . 225
Table C.2 Measures (A): RA vs. IIMT. . . 228
Table C.3 Measures (B): RA vs. IIMT. . . 229
Table C.4 Measures (A): evaluation of INTACMATO.. . . 230
Table C.5 Measures (B): evaluation of INTACMATO. . . 231
Table C.6 Measures (C): evaluation of INTACMATO. . . 232
Acronyms
ADD Additive difference rule ANOVA Analysis of variance COM Compatibility test CONJ Conjunctive strategy DIS Disjunctive strategy DOM Dominance strategy DSS Decision support system(s) EBA Elimination by aspect strategy
ECP Elementary communication process(es) EIP Elementary information process(es) EQW Equal weight heuristics
EV Expected value
FRQ Frequency of good and/or bad features heuristic GA Genetic algorithm
IDA Interactive decision aid(s)
IIMT Interactive information management tool(s)
INTACMATO Prototype for interactive information management tools LED Minimum difference lexicographic rule
LEX Lexicographic heuristic LTM Long-term memory
MAJ Simple majority decision rule
MD Median
M Mean
RA Recommendation agent(s) SAT Satisficing heuristic SATC Satisficing-plus heuristic SD Standard deviation SE Standard error
xx Acronyms SI Search index SM Strategy measure STM Short-term memory TTF Task-technology fit vs. Versus
Symbols
˛ Cronbach’s alpha
aij Attribute level of attribute i and alternative j
Ai Vector of possible attribute levels of attri
Attrw Vector of attributes ordered decreasingly according to attribute weight
altk D .a1k; : : : ; amk/ Alternative vector
attrl D .al1; al2; : : : ; aln/ Attribute vector
asp./ Aspiration level function (equals 0 in case aspiration level is met)
ˇi k Part-worth utility of occurrence k of attribute i
c Fitness for correlation of attribute vectors
ct Choice task
d.altj/ Deterministic component of u.altj/
df Degrees of freedom
ds Decision strategy
DSu The set of strategies without multiple mappings
"j Error term of u.altj/
F Fitness
O
f Effect size for ANOVA
Fr obust Robust fitness
H.ct/ Entropy of a choice task
l The length of the genotype
n Number of alternatives
m Number of attributes
mp Fitness for mapping
p Mutation probability
.altj/ The probability that alternative j is chosen
r Pearson’s correlation coefficient
rattr Number of attribute-wise transitions
xxii Symbols s Fitness for attribute range/attractiveness difference
t .x/ t-value of the T-test, x: degrees of freedom
u.altj/ Overall utility value of alternative j
v.aij/ Attribute value of attribute i and alternative j
wi Attribute importance, attribute weight
Xj i k Binary variable is 1 if altj contains occurrence k of