2015
Hybrid user perception model: comparing users’
perceptions toward collaborative, content-based,
and hybrid recommender systems
Mengqi Wu
Iowa State University
Follow this and additional works at:https://lib.dr.iastate.edu/etd
Part of theOther Communication Commons, and thePsychology Commons
This Thesis is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please [email protected].
Recommended Citation
Wu, Mengqi, "Hybrid user perception model: comparing users’ perceptions toward collaborative, content-based, and hybrid recommender systems" (2015).Graduate Theses and Dissertations. 14739.
Hybrid user perception model: Comparing users’ perceptions toward collaborative, content-based, and hybrid recommender systems
by Mengqi Wu
A thesis submitted to the graduate faculty
in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE
Co-majors: Journalism and Mass Communication and Human Computer Interaction Program of Study Committee:
Jan Lauren Boyles, Major Professor Daniela Dimitrova
Jonathan Kelly
Iowa State University Ames, Iowa
2015
TABLE OF CONTENTS Page LIST OF FIGURES ... iv LIST OF TABLES ... v ACKNOWLEDGMENTS………. ... vi ABSTRACT………. ... vii CHAPTER 1 INTRODUCTION ... 1
CHAPTER 2 LITERATURE REVIEW AND THEORETICAL MODEL ... 3
Big Data ... 3
Algorithmic Power ... 5
Unpacking Recommender Systems ... 7
A Typology of Recommender Systems ... 9
Recommender Systems and User Perception ... 11
Theoretical Model ... 12
Research Questions ... 18
CHAPTER 3 METHODOLOGY ... 19
Population and Sample ... 21
Procedure and Questionnaire Design ... 22
Pretest ... 25 Data Analysis ... 25 CHAPTER 4 RESULTS ... 27 Results ... 27 CHAPTER 5 DISCUSSION ... 36 Conclusions ... 41
Limitations and Future Research ... 42
REFERENCES ... 44
APPENDIX B APPROVAL OF IOWA STATE UNIVERSITY’S
LIST OF FIGURES
Page Figure 1 Theory of Planned Behavior (TPB) ... 14 Figure 2 Original Technology Acceptance Model (TAM) ... 15 Figure 3 Hybrid User Perception Model for The
Use of Recommender Systems ... 17 Figure 4 Graph of The Mean and Standard Deviation of
LIST OF TABLES
Page Table 1 Description of Section 2
(Questionnaire Source for Online Shopping Scenario) ... 24
Table 2 Demographic Profile (Descriptive Statistics) ... 28
Table 3 Descriptive Statistics for The Independent Variables ... 32
ACKNOWLEDGMENTS
I would like to express my appreciation to my major professor, Dr. Jan Lauren Boyles, and my committee members, Dr. Daniela Dimitrova and Dr. Jonathan Kelly, for their general guidance and support as well as providing helpful feedback throughout this one-year study.
Additionally, I would like to thank my classmates, friends and the faculty and staff in Greenlee School of Journalism and Mass communication for helping me successfully
complete my master’s study at Iowa State University. I would also like to thank the staff in the Registrar’s Office and the Information Technology group for sending out my survey via mass email service, as well as all the schoolmates participated in my survey for providing valuable information in this study.
Finally, special thanks to my parents and my boyfriend for their love, which encouraged me to keep moving on my life journey.
ABSTRACT
This study examines users’ perceptions toward three types of recommender systems by employing a hybrid user perception model combining with Theory of Planned Behavior (TPB) and Technology Acceptance Model (TAM) in order to specifically explain a message-attitude-use process. Recommender systems, as an innovation applying big data ideas and algorithmic power, have been widely applied to multiple Internet industries. In order to further investigate how users perceived the use of recommender systems and the differences among users’ perceptions toward the use of different recommender systems (collaborative filtering, content-based filtering, and hybrid filtering), three perception variables (perceived usefulness, perceived behavioral control, and perceived enjoyment) were specifically assessed by using an online survey of college students. Overall, the results indicated that there were some statistically significant differences among the user perceptions towards different types of recommender systems. In addition, users generally feel positive about the use of these recommender systems, and users’ perceptions toward hybrid-filtering system were rated higher than perceptions toward collaborative filtering and content-based filtering.
Keywords: recommender systems, big data, algorithms, user perceptions, collaborative filtering, content-based filtering, hybrid filtering, Theory of Planned Behavior, Technology Acceptance Model
CHAPTER 1 INTRODUCTION
Big data refers to “things one can do at a large scale that cannot be done at a smaller one, to extract new insights or create new forms of value, in ways that change markets, organization, the relationship between citizens and governments, and more” (Meyer-Schonberger & Cukier, 2013, pg. 6). In the last decade, this term has come onto the stage of digital society, thanks to the prevalent use of computer-based technologies, which are generating large volumes of data with huge potential to be exploited. Within the big data environment, the results based on data analysis are able to make predictions on users’ preferences or interests (Manyika et al., 2011). For example, the Obama campaign applied the analysis of big data to presidential elections, and won in 2012 (Lampitt, 2013). They persuaded individual voters by using various strategies, which were specifically designed according to the predictions on voters’ interests generated by big data analysis (Issenberg, 2012). Similarly, police officers can make predictions on criminal activities that are going to happen by means of analyzing big data (Collins, 2013). And UPS has employed big data to analyze drivers’ performance and to re-organize their route plans, which have greatly saved fuel costs (Davenport & Dyche, 2013). In general, big data, as a source of information, can be applied to diverse areas and can generate new economic value (Meyer-Schonberger & Cukier, 2013).
This aggregation of data provides both opportunities and some new problems, however. The development of computation-based systems has caused an environment of information overload that has negatively impacted the efficiency of user information when searching (Huang et al., 2004). Scholars have become increasingly interested in investigating how to manage this
problem (Pu et al., 2011). In particular, recommender systems were designed to solve this problem.
Recommender systems refer to a computational technology invented in early 1990s, (Konstan & Riedl, 2012) which helps to filter information by predicting preferences and offering suggestions via a series of algorithms (Pu et al., 2011). Most prior studies related to
recommender systems have specifically contributed to strategies about optimizing this technical functions or updating software algorithms (Shinde & Kulkarni, 2012) or researching how
recommender systems play an important role in an e-commerce environment (Senecal & Nantel, 2004). Prior literature, however, barely mentions the types of recommender systems, and how these systems impact users’ perception or decision-making. It is important to understand user experiences and improve systems in order to better satisfy users’ needs.
To fill this gap in the literature, I explore whether there is any difference among users’ perceptions toward varying types of recommender systems and how user perceptions operate in using these systems. To look at this phenomenon, this study employs a combination of two theoretical models, derived from the theory of planned behavior and technology acceptance model. These models are integrated into the research because they can generally explain the relationship between technology use and user perceptions, as well as the psychological process involving attitude-intention-behavior. To better understand user perceptions, I used a survey as the main research method. Generally, because there are very few studies on the relationship between the types of recommender systems and user perceptions, through this study, I hope to create new knowledge in the field by applying a combinative framework that will contribute to the further development of recommender systems or other innovations by comprehensively considering human factors.
CHAPTER 2
LITERATURE REVIEW AND THEORETICAL MODEL
This chapter outlines previous scholarship related to big data, algorithmic power, and recommender systems. This section also introduces a pair of theoretical models (Theory of Planned Behavior and Technology Acceptance Model), which explain the relationship between the use of technology and a message-attitude-behavior process. These models provide solid background and theoretical supports for this study. In this study, I specifically explore whether there is any difference among user perceptions toward three types of recommender systems, and how user perceptions operate when interfacing with the different systems.
Big Data
Because big data cross multiple disciplines, the definitions of big data can be varied according to the actual cases, and has not yet unified into a mutually shared definition (Meyer-Schonberger & Cukier, 2013). The definition of big data, centered within the computer science community, focuses more on structured data – information that is well organized and distributed into a common category or file (Arasu & Garcia-Molina, 2003). But not all data fit into
structured formats. In fact, most of information in the world dwells in some unstructured forms, defined as data that are unable to be fixed into neatly organized databases. Because of this
disorganization, data in unstructured forms are often processed very slowly (Kaisler et al., 2013). A growing number of researchers have expanded the definition of big data beyond its initial use in computer science. Subsequently, other scholars have redefined big data as the datasets that
current technology is incapable to store, manage and process efficiently because of the exceeding amount of data (Kaisler et al., 2013).
What causes the big size of data? According to Manyika et al. (2011), each action people take in the digital world is creating personalized data, such as cookies when browsing a website or purchasing histories when shopping online. Nowadays, the prevalence of technological products, electronic devices, and application systems are all driving the generation of data. Data, in fact, are recording the whole process of users operating technology. Each move made by users could be transferred into each separated dataset and stored somewhere in the device. As Manyika et al. (2011) reported, there are 5 billion mobile phones used in 2010. Using these devices, 30 billion texts or pictures are communicated around on Facebook monthly. Culling through these data points forms “an ocean of data” (Lewis et al., 2013, pg. 35).
Big data were initially considered as a technical problem because of “its volume, variety, and velocity” (Russom, 2011, pg. 12), which were thought to impede the operations and reduce efficiency of technological systems. Today, further scholarship has explored the values implicit in data-rich environments. In some situations, the more data collected, the more accurate results generated by data analysis (Russom, 2011). Marketing researchers, for example, could somehow analyze consumer behaviors and experimentally test relevant decisions based on the data
collected from consumers’ purchasing process (Kaisler et al., 2013). Data can also help to handle some complex situations when the human brain sometimes fails to process information, in favor of rationally adjusting human perception (Brooks, 2013). Some reports posit that big data can assist target campaigns in meeting some specific needs by accurately grouping information or users (Manyika et al., 2011).
Not all people can clearly understand their own preferences, however. To make a decision on selecting one preference or choice is a tough task that commonly happens in our daily life. Hence, big data analysis can be a tool to help people by predicting their preferences. It is possible to filter information with the support of big data tools and algorithms. Recommender systems, referred as a computational technique that can provides recommendations via a series of algorithm processes (Pu et al., 2011), are one innovation that can help people make decisions. Even though it is unlikely to empower big data to make all the decisions for humans anytime anywhere, algorithmic filtering based on big data is still a useful tool for problem solving, to some degree. Additionally, big data are boosting new business forms in multiple areas (Manyika et al., 2011) and inspiring more new ideas. For instance, the user experience toward a product is an important index for organization assessment, which can be transformed into data by analyzing consumers’ previous records (Russom, 2011). Then organization can improve the existing
product or innovate future products, and better enhance the user’s experience. In this business case, big data analytics is undoubtedly one of the best helpers (Manyika et al., 2011).
Algorithmic Power
An algorithm, defined as “a series of steps undertaken in order to solve a particular problem or accomplish a defined outcome” (Diakopoulos, 2014, pg. 3), is constantly exerting influences in society, particularly in the context of recommendation systems. In most cases, an algorithm is technically designed with a certain purpose of accomplishing the needs of solving existed problems or promoting relevant strategies (Lohr, 2012). Specifically, algorithmic power, as the technological foundation of recommender systems, represents a programing strategy to manage big data from computational perspectives (Diakopoulos, 2014). In the interaction
between big data and algorithmic use cases, several challenges exist (Manovich, 2011). Because the modern computer is capable to collect and disperse countless amounts and types of data (Cohen et al., 2011), it enables data to be fixed to either different forms or disciplines. Likewise, algorithmic power can be regarded as an innovative power evoked by big data that can accelerate the processing of data analysis. For instance, datasets could be efficiently filtered via the input algorithmic system and bring out more logical information to users.
Related to decision-making, algorithms are not only devoted to making autonomous conclusions by efficient computer programs (Diakopoulos, 2014); they also play a crucial role on human decision-making tasks, particularly those intersecting with recommendation systems. Considering the decisions made by algorithmic application, the term named “filtering algorithm” one of the types of algorithms proposed by Diakopoulos (2014) essentially presents the idea of a recommender system. Diakopolous defines the filtering algorithm as “including or excluding information according to various rules or criteria” (Diakopoulos, 2014, pg. 8). In this study, even if there are other different filtering approaches of recommender systems, the basic idea of this system still follows the concept of the filtering algorithm, which is about computational information-selection for users based on relevant data analysis. In the recommender system example, when a user is shopping online, useful recommendations or decisions can be filtered out after classifying related items, associated with other similar users’ choices, and prioritized to the user. A filtering algorithm is more inclined to be an integrated algorithmic approach and widely applied to recommender systems (Diakopoulos, 2014).
Unpacking Recommender Systems
A recommender system, which is also named as recommendation system (e.g. McDonald & Ackerman, 2000), recommender agent (e.g. Hostler et al., 2011) or recommendation algorithm (e.g. Linden et al., 2003), refers to a computational technology that is able to offer suggestions to users via a series of algorithm processes based on users’ previous searching history or other behaviors (Pu et al., 2011). In other words, it is a tool used to make assumptions on user preferences. These systems are widely applied to various areas, such as e-commerce (e.g. Amazon.com), movie or music websites (e.g. Youtube.com) and hotel restaurant service websites (e.g. TripAdvisor.com), among others. It means that once a user spends time on the Internet, these algorithmic programs probably have interacted with that user (McSherry & Mironov, 2009), including collecting data about the user, processing his or her information, predicting his or her interested items, presenting these personalized suggestions, and/or attracting his or her attention.
The use of recommender systems is considered as a profitable engine for Amazon and other online business companies. Thanks to the assistance of this technology, users are able to search target items in seconds. Sellers or organizations benefit from it when users purchase additional potential items, which they were not originally seeking (Pu et al., 2011). For example, let us say a user planned to buy a non-stick pan on Amazon.com, and searched numerous pans, comparing different features. Based on the user’s searching history, then, the recommender system behind Amazon.com would suggest several items marked as “related to items you’ve viewed” or “inspired by your browsing history,” (Amazon.com) like other cookers or kitchen accessories. At the end, the user might not only purchase a preferential non-stick pan, but also a set of baking pans. This case illustrates the power of recommender systems and algorithms. In
other cases, this system is not only applicable to an online retailer, but also other types of websites. Another example surrounds friend recommendations on social media, such as Facebook or Twitter. This system will attempt to match a user’s interests with other users. If successfully matched, the system would automatically offer friend recommendations to reciprocally matched users as well.
The use of recommender systems is considered as a profitable engine for Amazon and other online business companies. Thanks to the assistance of this technology, users are able to search target items in seconds. Sellers or organizations benefit from it when users purchase additional potential items, which they were not originally seeking (Pu et al., 2011). For example, let us say a user planned to buy a non-stick pan on Amazon.com, and searched numerous pans, comparing different features. Based on the user’s searching history, then, the recommender system behind Amazon.com would suggest several items marked as “related to items you’ve viewed” or “inspired by your browsing history,” (Amazon.com) like other cookers or kitchen accessories. At the end, the user might not only purchase a preferential non-stick pan, but also a set of baking pans. This case illustrates the power of recommender systems and algorithms. In other cases, this system is not only applicable to an online retailer, but also other types of websites. Another example surrounds friend recommendations on social media, such as Facebook or Twitter. This system will attempt to match a user’s interests with other users. If successfully matched, the system would automatically offer friend recommendations to reciprocally matched users as well.
By 2015, recommender systems have advanced as part of growing industries that enable companies to make huge profits (McSherry & Mironov, 2009). Netflix is a successful example related to the application of the recommender system. This company has set up a $1 million prize
to award the team that is able to best optimize its recommender system (McSherry & Mironov, 2009). According to data from McSherry & Mironov (2009), more than half of Netflix movie rentals are based on the suggestions provided by its personalization service.
An increasing number of companies across multiple industries have adopted and employed this popular invention. Generally speaking, recommender systems that are based on filtering algorithms can quickly segment recommendations to assist users in making decisions efficiently.
A Typology of Recommender Systems
Prior scholarship has identified three steps in the operation of a recommender system. First, the system purposively collects the data related to users’ preferences. Next, the system analyzes and calculates recommendations using algorithms. Finally, the outcomes are displayed to users (Wei, Huang & Fu, 2007). Recommender systems are categorized into the three
following approaches, a typology adapted from Wei, Huang and Fu:
Collaborative filtering (CF): As one of the most common approaches (Wei, Huang & Fu, 2007), CF comes to a recommendation by matching the records of a user’s behavioral history with other alike users’ histories (Jones, 2013). The central idea of CF is to search other target users with similar interests or preferences as the current user, and then group or generalize the information among these like users, constructing the prediction based on user preferences. Such engagement with CF systems is commonplace. For instance, users often prefer to adopt our friends’ recommendations instead of using filtering alternatives. This type of recommender system is widely applied to online retailers (which recommend products) or social media (which recommend friends with overlapped social network or similar interests).
Content-based filtering (CBF): Without the context of the user’s social network, it is impossible to associate data with other users’ information. In this situation, CBF can arrive at recommendations based on the data of her or his prior behavior history (Costa-Montenegro et al., 2012). There are some similarities between this approach and traditional searching methods. Generally, CBF generates recommendations by matching the relative content with the current user’s behavior. For example, a news website can provide potentially attractive news to a user based on his or her previous browsing history. If he or she relatively reads more food science news in the past, then the CBF-based recommender system is able to find similar contents that match with this topic. This example illustrates that digital media expertly create content and store messages as data. With the help of algorithmic systems, media can provide personalized
information to different users (Beam, 2014).
Hybrid filtering (HF): The purpose of the hybrid filtering approach is to avoid or improve the disadvantages of other recommendation technologies. The most common hybrid approach is the combination of CF and CBF approaches. The cooperation of these two
approaches is regarded as a way to enhancing the efficiency and accuracy of recommendations by allowing the outcomes to be processed from CBF at first, and then shifted to CF for further treatment (Jones, 2013). Netflix is a good example for this approach by combining CBF and CF (McSherry & Mironov, 2009). Another example is about adding users’ context factors into the common-used system types especially for mobile applications, which was proposed by Woerndl et al., (2007).
Taken together, these three types of recommender systems separately describe how our informational data are disposed by algorithmic programing in diverse ways. In practice, each type of recommender systems is respectively following different algorithmic principles, and
probably creating different results. As a result, it is worthwhile to examine whether any difference exists between users perceptions toward this typology or any factor that exerts influence on the user’s perception process.
Recommender Systems and User Perception
Within computational disciplines, a wealth of studies has illustrated the power of recommender systems (Cosley et al, 2003; Pathak et al, 2010; Hostler et al., 2011; Mandl et al, 2011; Costa-Montenegro et al., 2012; Cremonesi, et al., 2012). Particularly, this algorithmic technology plays a crucial role in the user’s attitude change and decision-making (Gretzel & Fesenmaier, 2006). As a result, it is reasonable to assume that the suggestions computed by recommender systems are conveying useful messages to audiences, to some degree. On the one hand, it helps users to make better choices by filtering overloaded information or matching relevant information (Costa-Montenegro et al., 2012). On the other hand, as the process of eliciting preferences by presenting refined messages (Gretzel & Fesenmaier, 2006), it does enhance the likelihood of persuading users to purchase unanticipated items. Based on a
computational thinking approach, recommender systems can be considered as both scientific and persuasive strategies, presenting a successful application of algorithmic power as well. I will now discuss how this computational technology exerts its power in users’ perceptions by setting up a theoretical model, which is necessary to explain the relationship between the use of
Theoretical Model
In this study, I establish the model of user evaluation toward recommender systems by combining two theoretical models: the technology acceptance model (TAM) and the theory of planned behavior (TPB). Theoretically, both TAM and TPB are derived from the theory of reasoned action (TRA), which was addressed by Martin Fishbein and Icek Ajzen (1975), for the sake of explaining a message-attitude-use process and predicting behavioral outcomes. The main purpose of all of these models is to figure out what drives an individual to conduct a given behavior, including the summative explanation of the psychological activities and some other external factors
Theory of planned behavior (TPB)
There are two key elements underlined in the theory of reasoned action construct: attitude towards behavior (ATB) and subjective norms (SN) (Ajzen, 1991). Respectively, ATB is defined as the extent to which a person makes an assessment of the target performance as positive or negative, consisting of behavioral beliefs and outcome evaluations (Ajzen, 1991; Benoit & Benoit, 2008). SN is defined as “the perceived social pressure” (Ajzen, 1991) that can help shape decision-making, including normative beliefs and the motivation to comply (Benoit & Benoit, 2008). In this model, external variables cannot directly lead to the planned behavior, but
behavioral intention can be regarded as a transition between external variables and final behavior. It shows the intention whether to conduct the related behavior or not. The likelihood that an individual will perform a behavior mostly depends on the degree of intention toward
performance (Ajzen, 1991). In short, the stronger the intention; the higher possibility to perform (Benoit & Benoit, 2008).
Subsequently, Icek Ajzen (1991) further developed TPB (shown as Fig.1) by adding perceived behavioral control (PBC) into the TRA construct (Benoit & Benoit, 2008). PBC is referred to as the extent to which a person perceives himself or herself capable to conduct the behavior or not, on the basis of previous experience and the prediction of unknown difficulties in the future (Ajzen, 1991). In this TPB model, Ajzen (1991) explained PBC with two key factors: one was “control belief” that indicated a belief toward someone’s ability to complete a plan. The other is “potential control factors” that indicated some other factors were possible to influence on the actual implementation of this to-do plan (Benoit & Benoit, 2008). This variable emphasizes self-assessment as well as confidence towards the future plan. For example, let’s say a person plans to run in a marathon race. Before participating in this race, he or she may have a checklist in mind, such as whether he or she has enough willpower or physical energy or even a good pair of running shoes. Items in the checklist can be visible or just mental. The result after
self-assessment would be somehow presented as PBC. Hence, in order to more carefully explain the process of how an external message exerts an effect on actual behavior, there are three
significant variables presented in the TPB model (shown as Figure.1), which are attitude towards behavior (ATB), subjective norms (SN), and perceived behavioral control (PBC).
Figure 1. Theory of planned behavior (Icek Ajzen, 1991)
Technology acceptance model (TAM)
However, applying the TPB model alone in this study is insufficient for the reason that it is a broad framework to explain and predict the behavioral process, rather than a situational application specifically in the technological use case. Hence, I attempt to add the TAM model into our application model, which is able to interpret perception variables in the case of recommender system.
Consistent with TRA, Davis (1989) proposed two determinants for a user’s technological adoption in TAM: perceived usefulness (PU) and perceived ease of use (PEU). Literally, PU is defined as the degree to which a user perceives the likelihood of improvement on his or her job performance by the use of a certain technological system (Davis et al. 1989); PEU relatively
refers to “the degree to which the . . . user expects the target system to be free of effort" (Davis et al. 1989, pg. 985). These two determinants mentioned above can well expand the concepts of behavioral beliefs and outcome evaluations in TPB (Ajzen, 1991).
Figure 2. Original technology acceptance model (TAM) (Davis, 1989)
According to the literature review on TPB and TAM, both of these models are able to explain or predict the attitude-intention-use process (Mathieson, 1991; Venkatesh & Davis, 2000; Legris et al., 2003) and explain users’ willingness toward the use of technology (Chen &
Dimitrova, 2008). In our study, in order to specifically explore user’s evaluations toward different types of recommender systems, I will use elements of these theoretical models, applying elements of the TPB and TAM models.
Considering the definitions of PBC (in TPB) and PEU (in TAM), PEU is reasonable to be viewed as a subcategory of PBC, because PBC explains the impact of perceived self-efficacy toward attitude and use (Ajzen, 1991), which is composed of the degree to which a user feels easy to adopt the recommendations provided by recommender systems as well as the degree to
which he or she perceives his or her condition or ability is eligible to follow these
recommendations. Therefore, in the case of recommender system use, PBC can generally cover the concept of PEU and better describe self-efficacy belief.
In addition, both PU and PBC primarily contribute to the explanation of planned behaviors. But unintended behaviors are possible to emerge during the use of recommender system. When browsing websites, it is unlikely to avoid being attracted by some unexpected information (Madhavaram & Laverie, 2004). Therefore, another determinant will be proposed in our application model: perceived enjoyment (PE), defined as “the extent to which the activity of using a specific system is perceived to be enjoyable in it’s own right, aside from any
performance consequences resulting from system use” (Venkatesh, 2000, pg. 351; Davis et al., 1992). The founders of TAM, Davis et al. (1992), raised PE as a significant “intrinsic motivation” on behavioral intention to use, which potentially exerts power on user’s adoption process and motivates user’s intent on use. The information generated by a recommender system seems to be more powerful to attract users’ attention. PE can somehow reveal a user’s preference toward technology use. In particular, PE addressed here is for the purpose of explaining the occasional situation, which a user can be instantaneously motivated by enjoyable or interesting information provided by recommender system, and then decide to adopt the unintended behavior, such as impulsive online purchasing.
Hybrid user perception model: the application model
Generally, the application model for the use of recommender system is shown as Figure. 3, which is a combinational model by synthesizing the Theory of Planned Behavior and
will examine the three types of recommender systems (CF, CBF, and HF) and the external variables shown in the Technology Acceptance Model. For the human perception phase, there are three determinants hypothesized: PU, PBC and PE. As a result, attitude towards use and behavioral intent will be evaluated as outcomes.
Figure 3. Hybrid User Perception Model for the use of recommender system
Prior studies have been insufficient in answering how different types of recommender systems can exert influences on user perceptions or decision-making process. This scholarship has not fully explored the relationship between types of system and user perceptions. It is necessary to fill this gap because the advanced development of technologies should carefully take human factors into account in order to continually optimize user experiences and create more user-friendly technological products (Wickens et al., 2004). In this case, user perception, as an essential psychological factor, can be a source to reflect user experience toward use of
recommender systems (Agarwal & Prasad, 1998). Based on the results generated by the current study, system developers or relevant campaigns may deeply understand their target audiences and enhance the usability of system. This study can be regarded as an application example for future researchers within the domain of Human Computer Interaction. And in future studies, the new theoretical framework employed in this study can be flexibly applied to further explicate the relationship between the use of technology and user perceptions.
Research Questions
In general, the purpose of this study is to assess users’ perceptions toward the three types of recommender systems - CF, CBF, and HF. User perceptions are evaluated by three
perceptions variables- PU, PBC, PE. Research questions for this study are concluded as follows: RQ1. What differences exist among users’ perceived usefulness (PU) toward three types of recommender systems (Collaborative Filtering, Content-Based Filtering, and Hybrid Filtering)?
RQ2. What differences exist among users’ perceived behavioral control (PBC) toward three types of recommender systems (Collaborative Filtering, Content-Based Filtering, and Hybrid Filtering)?
RQ3. What differences exist among users’ perceived enjoyment (PE) toward three types of recommender systems (Collaborative Filtering, Content-Based Filtering, and Hybrid Filtering)?
CHAPTER 3 METHODOLOGY
In this research, I specifically probe how three types of recommender systems (CF, CBF, and HF) influence user perceptions, which would be measured by three variables (PU, PBC, and PE). The relationship between the types of recommender systems and user perceptions has been rarely investigated in prior studies. As a result, this study would fill a gap of knowledge in this field because user perception is a significant psychology determinant on attitude-behavior relations (Fazio & Williams, 1986), and a new theoretical framework employed in this study can be applied to future studies. Understanding user perception is a crucial step when studying the interactive relationships between human and technology. Moreover, the three types of
recommender systems with different algorithmic approaches are possible to exert different powers on user perception. To fully understand this relationship, I conducted a survey of college-age users of recommendation systems.
Within scholarly research, a survey is frequently employed because researchers can collect a large number of data about the characteristics, behaviors, or perspectives from countless participants (Tanur, 1982). As a quantitative research method, the primary strategy for using a survey is to gather information by asking participants questions (Pinsonneault & Kraemer, 1993). The use is commonplace in decision-making studies (e.g. Stewart, 1992; Amason, 1996;
Hoffmann & von der Schulenburg, 2000; Trevino, 1986), which directly applies to the context of this research. Because the questions in a survey are open to a variety of people without
geographic limitation (Fowler Jr, 2008), it is possible that to collect a large number of data and generally analyze perspectives from varied publics (Wimmer & Dominick, 2013).
Although a survey was chosen as the most appropriate method in this research, it possesses several limitations. It is impossible, for instance, for researchers to confirm whether someone who took the survey is the recruited participant. In addition, it is hard to ensure the quality of responses because self-report answers could be unreal (Wimmer & Dominick, 2013). While this study empirically examined the differences between three types of recommender systems by means of three kinds of perceptions, self-perception was not enough to represent actual behavior or use, to some degree. Although three kinds of perceptions are considered as the main variables during the process of using recommender systems, other unmentioned variables are still possible to exert power to change users’ attitude or behavior intention, such as cultural background or computer skills. In order to eliminate confounding factors, I tried to manipulate statistical strategies, such as enhancing sample randomization or selecting appropriate
approaches to analyze collected data. This method enabled reaching a wide population of college students to uncover their interactions with recommender systems.
In this research, an online survey was chosen because this topic was related to
computational technology and Internet users of college students, who were easier to approach via a web-based survey instrument. Although in-person or telephone surveys allow interviewers to cover a large, geographic area, an online survey is able to maximally eliminate geographic limitation because of the wide diffusion of Internet access (Wimmer & Dominick, 2013; Fowler Jr, 2008; Kaplowitz et al., 2004; Cobanoglu et al., 2001; Dommeyer & Moriarty, 1999).
Because this research involves human subjects, I submitted the study through Iowa State University’s Institutional Review Board (IRB) process by submitting an exempt study review form. As an exempt review, this approval granted approval to conduct a survey method with adults. The IRB approval documentation is attached as Appendix B in this thesis.
Population and Sample
Aligning with the purpose of this study, the research population includes Iowa State University students who are over 18 years old. A random sample of Iowa State University undergraduates, graduates and new admits to the university were recruited because they are commonly regarded as the most active users of web-based technologies (Morahan-Martin & Schumacher, 2000). Operationally, this survey was programed using an online survey tool, Qualtrics. This approach is appropriate for an academic survey because it allows researchers to construct a questionnaire with a formal design.
To gather enough valid responses to make statements about recommender use within the selected population, I decided to randomly sample students at Iowa State University. According to the data listed on the institution’s website, more than 34,000 students are currently enrolled at ISU (Iowa State University, 2014). To best represent the target population, I randomly sampled 3,000 students. To administer the survey, I requested that the registrar office’s provide a random list of 3,000 student emails. Participants received an email that contained a link to the Qualtrics survey. The first wave of responses was sent on March 20, 2015. Because of the spring break holiday, responses received in this first wave of respondents were less than 80. Given the low response rate, I requested a second list of 3,000 student emails from the registrar office and sent out the survey link again on April 1, 2015. The second round of survey was in the field for roughly one week. In the second wave, 320 participants took part in this study. The analysis, therefore, only encompasses the second wave of the study.
Procedure and Questionnaire Design
Participants were asked to complete an online survey, which included 3 sections of questions: (1) use of the Internet; (2) perceptions toward the use of recommender systems (including perceptions toward the use of CF; perceptions toward the use of CBF; perceptions toward the use of HF); and (3) demographic questions. Taken together, the survey consisted of 32 questions (including 9 filtering questions), and took users, on average, about 15 minutes to complete.
In the survey’s introduction, participants were able to choose whether took part in the survey or not. Subjects were informed that their participation was completely voluntary, and all responses obtained would be anonymous. There were no foreseeable risks for participating in this research.
In case of participants’ preconceptions might lead to deviation of results, three types of recommender systems were explained by three different scenarios rather than definition. Operationally, each type of recommender systems was designed into each scenario table and each table encompasses three variables. The use of scenarios was to clearly present examples for three system types and tried to avoid pre-informing participants. In this section of the survey, the recommender system was described as “an online service.” I explained three types of systems with three different scenarios, such as online shopping (CF), news browsing (CBF), watching movies or television shows online (HF). A five-point Likert scale (1-strong disagree; 2-somewhat disagree; 3- neither agree nor disagree; 4-2-somewhat agree; 5-strongly agree) was arranged into three tables for each type of recommender system. Likert scales, created in 1932, have been regarded as a common-used tool to measure the respondents’ views or attitudes (Allen & Seaman, 2007; Clason & Dormody, 1994) because it is capable reflect strength or importance
of their opinions by a quantitative scale (Maurer & Pierce, 1998). This scale was selected as an acceptable tool to assess user perceptions towards the recommender systems for the reasons that it is easily gather descriptive data and better measure perception-related questions (Maurer & Pierce, 1998).
Because there were three dependent variables (PU, PBC, and PE) each variable was operationally described with three statements and measured by a five-point Likert scale. Each statement was adapted based on original definitions of each perception and some other literature sources that were shown in Table 1. Totally, there are nine statements designed to evaluate user perceptions. In particular, PU was measured as the speed of making a decision with the help of suggestions, usefulness of suggestions and advantages of suggestions. PBC was measured by the ability to adopt suggestions, comprehension of suggestions, and the simplicity of suggestions. PE was measured by whether new information was inspired, the appeal of suggestions, and the enjoyment of suggestions.
Table 1. Description of Section 2 (Questionnaire Source for Online Shopping Scenario)
The survey concluded with demographic questions including: user age, gender, ethnicity origin, education level, population of city/town, location (state) and family income.
Additionally, for the purpose of assessing validity, most of questions in this survey were closed-ended providing some alternative response options (Forman & Damschroder, 2008; Jean
Item Measure Source
PU1 Using these suggestions enables me to make a decision more quickly about what should I purchase online.
Adapted based on the definition of PU (Davis et al., 1989)
PU2 The suggestions offered by an online shopping website are often useful for me.
Adapted based on the definition of PU (Davis et al., 1989)
PU3 I find these suggestions are advantageous in choosing products that I may be interested in.
Adapted based on Wu & Wang, (2005)
PBC1 I feel able to adopt the suggestions provided by an online shopping website.
Adapted based on the definition of PEU (Davis et al., 1989)
PBC2 I generally find the suggestions provided by an online shopping website to be understandable.
Adapted based on Pikkarainen et al. (2004)
PBC3 I find the suggestions provided by an online shopping website to be simple.
Adapted based on Pikkarainen et al. (2004) PE1 The suggestions often lead me to new products and services that I
wouldn’t have otherwise discovered.
Adapted based on the definition of PE (Davis et al., 1992)
PE2 Generally speaking, I find the suggestions provided by an online shopping website to be appealing to me.
Adapted based on Venkatesh, (2000) and Davis et al. (1992) PE3 I enjoy using the suggestions from this type of shopping website. Adapted based on
Venkatesh, (2000) and Davis et al. (1992)
& Presser, 1986). The survey featured two open-ended questions, in which respondents provided suggestions to improve the recommender system experience, and if they viewed this system either positive or negative. This type of question was necessary because it allowed participants to leave their perspectives freely without being pre-informed, and enabled the researcher to collect new information related to the topic (Jackson & Trochim, 2002). Open-ended questions were required to be coded on the responses for further analysis on qualitative textual data. Using both open-ended and closed ended questions enabled a broader range of potential responses.
Pretest
After drafting an acceptable questionnaire for the survey and receiving IRB approval, a pretest was conducted among 15 college students (roughly 0.5% of the final sample 3,000), which were purposively sampled at Iowa State University, for the purpose of testing the survey instrument. All pretest participants had Internet experience on recommender systems, and completed all the questions listed in the questionnaire. Based on the feedback after the pretest, I revised and optimized the questionnaire. Specifically, one question was added about the level of Internet use (heavy/medium/light user). In addition, two open-ended questions were added to broaden the subject’s ability to respond to the topic. The pretest participants did not take part in the final survey.
Data Analysis
In this study, data were numerically collected from the answers of the survey, and analyzed using SPSS. Final results of each question were categorized and interpreted by distribution of probability. Inferential statistical analysis was conducted to describe the
evaluations of different types of recommender systems and the analysis of users’ perceptions. Because a five-point Likert scale was conducted in the main part of survey questionnaire, the summative scores on responses could be directly processed by statistical analysis. In order to test the differences among three types of perception (measured by Likert scales as continuous
variables) under each scenario of recommender systems (categorical variables), one-way ANOVA was an appropriate statistical approach in this study to test significance of group differences when equal or more then three categorical independent variables (Tabachnick & Fidell, 2001). In addition, correlation test was also necessary to measure the relationship between the level of Internet use and the points responded in the Likert scale questions about perceptions (Tabachnick & Fidell, 2001; Lawrence & Lin, 1989). The p-values for each factor are indicated, and the statistical significance highlighted. By use of statistical factor analysis, we could provide detailed data outcomes as scientific evidence, illustrating how users engage with recommender systems.
CHAPTER 4 RESULTS
Results
In this survey, 320 responses were received. Hence, the response rate was 10.67% (320 out of 3000). Of the responses, 308 (96.3%) could be regarded as valid data (N = 308), in which subjects responded to most of questions in the survey. In total, 12 responses were removed because they answered less than 28 (out of 32) questions in the survey. Nearly half (46%) of the respondents were male, whereas 54% of them were female (see Table 2; the full text of the survey is available in Appendix A). Because the sampling frame was drawn from college students at Iowa State University, 83.6% of the respondents were from the 18-25 years old age group, and 48.5% of them have completed some colleges. Additionally, this survey was conducted in Iowa. As a result, the majority of the respondents were Midwest residents,
especially Iowa residents (79.9%), and 77.9% of them were living in a city/town where had less than 100,000 residents. In particular, 8% of them were from Illinois; 7% of them were from Minnesota; and the rest of respondents were from outside the Midwest area. In viewing the demographics, 86.9% of respondents were white. For the annual total household income, 38.5% of respondents indicated that their annual incomes were less than $25,000 and 20.7% of them indicated that their annual income were more than $100,000. Since the respondents were college students, it was possible that they would report their parents’ incomes in this question that makes this demographic variable biased.
Table 2. Demographic Profile (Descriptive Statistics)
Variable Frequency Valid percentage
Age 18-25 254 83.6 26-35 43 14.1 36-45 5 1.6 Over 45 2 0.7 Gender Female 166 54.0 Male 142 46.0 Race White 266 86.9
Asian / Pacific Islander 28 9.2
Hispanic or Latino 8 2.6
Native American 2 0.7
Black or African American 1 0.3
Education level
Some college 151 48.5
Bachelor’s degree 57 18.4
High school graduate 42 13.6
Master’s degree 32 10.4
Associate’s degree 18 5.8
Doctorate 6 1.9
Some high school completed 2 0.6
Local population
Less than 50,000 residents 131 42.4
99,000-50,000 residents 109 35.5
249,000-100,000 residents 29 9.5
499,999-250,000 residents 20 6.6
1,000,000-500,000 residents 11 3.6
Table 2. Demographic Profile (Descriptive Statistics)
Variable Frequency Valid percentage
Annual household income Less than $25,000 115 38.5 More than $100,000 62 20.7 $50,000 - $74,999 44 14.7 $75,000 - $100,000 39 13.0 $25,000 - $49,999 39 13.0 Area Iowa 243 79.9 Out of Iowa 61 20.1
Internet use and recommender system
According to the responses to the study’s Internet use questions, 99.4% of the
respondents indicated that they used the Internet several times a day. In order to further define the level of Internet users’ engagement online, respondents were how many hours they used the Internet per day. Nearly half (47.6%) of respondents evaluated themselves as a heavy user of the web (using the Internet more than 6 hours a day). And the other half of users (46.0%) of them evaluated themselves as a medium user (using the Internet 3-6 hours a day). As a result, few users defined themselves as “light” users of the web. During the last month, most subjects used the Internet to send email (97.7%), listen to music (94.2%), assess a social networking site (e.g. Facebook, Twitter) (89.3%), and follow news stories (79.9%). In assessing the types of devices used to connect to the Internet, 93.9% of all respondents used laptop computers, and 90% of all respondents used smartphones.
The survey’s core purpose was to evaluate how users approach recommender systems. To address this topic, three scenarios were described in the survey -- each representing a type of recommender system. It should be noted that the results are self-reported data.
The first scenario surrounded online shopping, which served as an example for the CF system. The CF approach is defined as a type of recommender system that can provide a
recommendation by matching the records of a user’s behavioral history against other like users’ search histories (Jones, 2013). This system type is widely applied to online retailers. In this scenario, 92.5% of respondents had participated in online shopping experiences, and 66.6% of these respondents indicated that they have shopped online at least monthly. In their interactions with online shopping, nearly all respondents had been provided with suggestions by
recommender systems during online shopping (97.9%). In evaluating the frequency of these recommendations, nearly nine in 10 respondents (88.5%) had been offered with CF-based suggestions every time or most of the time they engaged in online shopping.
The second scenario centered upon news browsing, as an example for the CBF system. CBF is defined as a type of recommender system that can arrive at recommendations based on the data of her or his prior behavior history (Costa-Montenegro et al., 2012). This system type is commonly used in online searching tools, especially for searching personalized news. In this scenario, nearly nine in 10 (93.2%) of the respondents had browsed news online, and nearly three-quarters of them (74.3%) indicated that they have browsed news online daily or weekly. Specifically, three in four users (76%) were provided with suggestions by a recommender system while browsing news online, indicating that they have been offered with suggestions every time or most of the time (72.6%).
The third scenario focused on watching movies or television shows, as an example for the HF system. HF is defined as a type of recommender system that can combine different system types into one, in order to enhance the efficiency and accuracy of recommendations. This system type allows the outcomes to be processed from CBF at first, and then shifted to CF for further treatment (Jones, 2013). In this scenario, more than nine in 10 (91.9%) respondents watched online movies/television shows online, with three in four users (74.2%) watching online movies/television shows daily or weekly. The majority of users had been provided with
suggestions by recommender systems while watching movie or television programming (85.9%), indicating that they have been offered with suggestions every time or most of the time (80.8%).
Analysis of perception variables by ANOVA
The main purpose of this study was to explore the differences among user perceptions toward each type of recommender system. A one-way repeated measures ANOVA and paired-sample t-tests were conducted to compare means of relevant variables under different system type conditions. The survey had three scenarios that represented for the three system types, and three types of perceptions assessed. All the perception variables (independent variables) were measured by the five-point Likert scale (1-strong disagree; 2-somewhat disagree; 3- neither agree nor disagree; 4-somewhat agree; 5-strongly agree). Operationally, PU was computed by
averaging responses to questions 12_1, 2, 3; questions 16_1, 2, 3; and questions 20_1, 2, 3. Similarly, PBC was computed by averaging responses to questions 12_4, 5, 6; questions 16_4, 5, 6; and questions 20_4, 5, 6. And PE was computed by averaging responses to questions 12_7, 8, 9; questions 16_7, 8, 9; and questions 20_7, 8, 9 (see Appendix A).
The descriptive statistics for the independent variables (see Table 3) evidenced that all the perception variables were higher than the midpoint of the five-point Likert scale. In other words, these responses seemed to be positive toward the use of recommender systems. PBC was rated slightly higher than PU and PE. And all the perceptions toward CF were rated higher than the perceptions toward CBF and HF. In Figure 4, it can be seen that means of PU, and means of PE were very similar, but the means of PBC was different from the other two.
Table 3. Descriptive statistics for the independent variables
Variable Type of system Mean Std. Error of
Mean Perceived usefulness (PU) Collaborative Filtering (CF) 3.10 .057
Content-based Filtering (CBF) 3.50 .062 Hybrid Filtering (HF) 3.53 .060 Total 3.34 .047 Perceived behavioral control (PBC) Collaborative Filtering (CF) 3.60 .039 Content-based Filtering (CBF) 3.71 .049 Hybrid Filtering (HF) 3.79 .046 Total 3.68 .037
Perceived enjoyment (PE) Collaborative Filtering (CF) 3.16 .055 Content-based Filtering (CBF) 3.55 .066
Hybrid Filtering (HF) 3.66 .058
Figure 4. Graph of the mean and standard error of dependent variables
As a result, PU was analyzed in a one-way repeated measures ANOVA, comparing the three types of recommender systems (CF, CBF, and HF). The main effect of system type was statistically significant, F (2,304) =12.172, p<.001. Follow-up tests indicated that PU toward CF was different than PU toward CBF, t(192)=5.096, p<.001, and PU toward CF was different from PU toward HF, t(219)=5.458, p<.001. No other comparisons were statistically significant.
PBC was analyzed in a one-way repeated measures ANOVA comparing the three types of recommender systems (CF, CBF, and HF). The main effect of system type was statistically significant, F (2,302) =3.478, p=.032. Follow-up tests indicated that PBC toward CF was different than PBC toward HF, t(218)=3.193, p=.002. No other comparisons were statistically significant
PE was analyzed in a one-way repeated measures ANOVA comparing the three types of recommender systems (CF, CBF, and HF). The main effect of system type was statistically significant, F (2,302) =14.152, p<.001. Follow-up tests indicated that PE toward CF was different than PE toward CBF, t(191)=4.225, p<.001, and PE toward CF was different from PE toward HF, t(218)=6.401, p<.001. No other comparisons were statistically significant.
Additionally, I selected gender, as possible demographic factors impact on users’ perceptions toward recommender systems, to be further analyzed by ANOVA because the distribution of gender was almost half and half - 46% of the respondents were male and 54% of them were female (see Table 2). There were two subgroups of comparison that were statistically significant: PU toward CBF, F (1, 215) =6.717, p=.010, and PU toward CBF, F (1,215) =5.827, p=.017. No other comparisons were statistically significant.
Regarding views on user experience related to the recommender system, most respondents considered the site’s visual design (81.6%) and the relevance of information provided (88.3%) as important factors to attract them to use web-based suggestions.
By running the reliability statistics, the Cronbach’s Alpha of PU statements was calculated as .802, the Cronbach’s Alpha of PBC statements was calculated as .771, and the Cronbach’s Alpha of PU statements was calculated as .823, which indicated the measurement model was reliable (Wimmer & Dominick, 2013).
Two open-ended questions addressed the positive/negative elements of online
suggestions. After coding, seven categories emerged for each question (see Table 4). It can be seen that many respondents considered “new ideas are inspired” (29.9%) and “relevant
information affiliated with users’ interests” (27.9%) as positive elements, while 41.7% of them considered “too many information provided can be distracting or annoying” as negative elements.
Table 4. Coding Sheet for Open-ended Questions
Question Frequency Valid
percentage Q24. What are some
positive elements of online suggestions? Total responses: 201
1. New ideas are inspired 60 29.9 2. Relevant information affiliated
with users' interests
56 27.9
3. More information is provided 26 12.9 4. Helpful for decision-making or
information searching
25 12.4
5. Allow to compare products or prices
14 7.0
6. Convince 11 5.5
7. Others 9 4.5
Q25. What are some negative elements of online suggestions? Total responses: 204
1. Too many information provided can be distracting or annoying
85 41.7
2. Wrong or irrelevant information 41 20.1 3. Waste of time or money 29 14.2 4. Information provided is useless
or unnecessary
24 11.8
5. Not interested in 11 5.4
6. Others 9 4.4
7. Privacy problem, data mining 5 2.5
In sum, there existed some differences among perception variables (PU, PBC, and PE) toward each type of recommender systems (CF, CBF, and HF), and these differences were statistically significant.
CHAPTER 5 DISCUSSION
This study tested the differences among perceptions toward three types of recommender system by employing an application model based on the Theory of Planned Behavior (TPB) and the Technology Acceptance Model (TAM). In order to offer more insights about the relationship between users and technology use, this section introduces some implications of the theoretical model, perception variables and demographic variables. based on the findings of this study.
Several prior studies (e.g. Venkatesh, 2000; Koufaris, 2002; Dickinger et al., 2008; Igbaria et al.,1995) related to the Theory of Planned Behavior (TPB) and Technology Acceptance Model (TAM) have discussed how the use of technology influences perceptions, attitude or behavior. To some extent, the results of this study supported the theoretical ideas of these studies. The current work suggests that the use of recommender systems has an interrelationship with user perceptions (PU, PBC and PE). Although these prior studies have investigated the
relationship between the use of technology and human perception, they rarely centered upon how the use of the recommender system was related to user perceptions and the comparison of system types. Recommender systems, which apply concepts of big data ideas and algorithmic power, have been widely used in human online activities (Pu et al., 2011). It is worthy to be particularly discussed as a human computer interaction topic due to its universality. The results of this study showed statistically significant differences among perception variables toward different system types, which can fill this space in future research.
Theoretical implications
This study applied a new theoretical model (Hybrid User Perception Model) by combining the Theory of Planned Behavior (TPB) with the Technology Acceptance Model (TAM). This combinational model could be an alternative for the future studies related to technology use. It was a reasonable attempt to combine these two models because both of them were derived from the theory of reasoned action (TRA) and human psychological factors were emphasized in these models. The results of this study showed the relationship between the use of recommender system and users’ perceptions and explained the differences among PU, PBC and PE toward different system types. For the future studies, these perceptions could be important variable related to the use of technology.
Implications of perception variables
According to the results, most of users cared about whether the recommendations provided were useful or relevant for them. In other words, PU was important in users’
information process and decision-making process, especially when users needed to select an item from a variety of options. PBC was relevant to whether the information clearly communicate to users and whether they would adopt the recommendations based on their capabilities. PE was viewed as a motivation to attract users’ attentions or arouse users’ intentions. For example, many respondents indicated that some new options were presented when users were offered
recommendations. These perception variables seemed to work separately, but the combination could greatly impact on users’ behavioral intention or actual behavior. Hopefully, the findings of this study could also encourage more scholars to explore the interrelationship between
In general, users were positive toward the use of recommender systems. More
specifically, three types of perceptions were examined in three scenarios (or system types). There were statistical differences between the perception types toward CF and the perception types toward HF. In particular, PU and PE were rated similarly in each type of recommender system. But PBC was rated differently from the other two perception variables.
Generally speaking, perception variables toward CF, on average, were rated lower than perception variables toward CBF and perception variables toward HF, which indicated that users felt less satisfactory toward CF comparing with the other two system types. Regarding to the definitions of each system type, information provided from CF was generated by matching the records of a user’s behavioral history with the other alike users’ histories (Jones, 2013), while information provided from CBF was generated based on the data of a user’s prior behavioral history (Costa-Montenegro et al., 2012), while the HF was combined with the two system types mentioned above. To match data based on a single user’s prior behavior history was perceived as more beneficial than matching data with the similar users’ histories. In particular, HF was
generally rated highest. Technically speaking, algorithmic programing could greatly determine the differences among multiple system types by manipulating different strategies (McSherry & Mironov, 2009). In other words, algorithm can issue instructions requiring the recommender system to generate collaborative recommendations or content-based recommendations. In terms of HF, it absorbed the advantages of multiple system types with the help of algorithms and enhanced the quality of recommendations, which may improve users’ satisfaction by better matching with users’ preferences. It seems that excessive information provided by recommender systems was an annoying problem for users. To some extent, HF can be regarded as a useful
approach to further refine recommendations via the combinative filtering process and reducing the amount of information.
Moreover, perception variables, on average, were rated higher than the midpoint of the five-point Likert scale (1-strong disagree; 2-somewhat disagree; 3- neither agree nor disagree; 4-somewhat agree; 5-strongly agree), which indicated that users were optimistic toward the use of recommender system and the information provided by the system. They believed that the recommendations provided by this technology were generally helpful to filter information or offer inspirations.
Implications of demographic variables
After testing the relationship between demographic variables and the use of different recommender system types, the results indicate that gender variable is a possible demographic factor influencing users’ perceptions. Compared to CF and HF, the perception differences between male and female users are more visible under the CBF condition. Generally, male users perceived lower PU and lower PE than female users. Prior studies have shown that some gender differences exist when processing online information (Kim et al., 2007). And these differences were reflected in users’ attitude and behaviors (Kim et al., 2007). Compared to female users in this study, male users seemed to be less satisfied the use of recommender systems in terms of the content-based online activities they commonly used. Otherwise, since the results did not show enough variances in the sample of college students, the other demographic variables could not be evaluated in the current study.
Based on the results of questions 21-25 (see Appendix A), most of respondents referred the relevance of information provided and the amount of information as the important factors
during when using the recommender system. Based on the responses to these two open-ended questions (see Table 4), some benefits of use can be identified. For instance, users viewed recommender systems as a positive tool to inspire new ideas and to provide personalized information matched with their interests. On the other hand, users cited that recommender systems can overwhelm users with too much information, which can be annoying. Overall, users felt positive about recommender system possibly due to the relevance of information and new inspiration. In the era of big data, users could somehow get useful suggestions from a variety of information and solve information overload problem with the help of system. Following from these findings, recommender systems that will be most engaging to the user should balance the volume of suggestions and the interests of users.
Human computer interaction (HCI)
Generally, this study is related to human perception variables and computational factors. As mentioned, previous studies of recommender system have mainly focused on the algorithmic programing level (e.g. Shinde & Kulkarni, 2012) but rarely focused on human factors. In fact, user-centered design could be a powerful approach to optimize the system features and improve the usability or effectiveness of a recommender system (Swearingen & Sinha, 2001). In the last two decades, user-centered design and HCI standards have been applied to multiple
technological practices (MacKenzie, 1992; Soloway et al., 1994; Bevan, 2001). Human factors, as important elements in human computer interaction (Wickens et al., 2004), can provide direct and valuable information to analyze the usability of a product. This study, from a HCI