Applying the Technology Acceptance Model to Evaluation of Recommender Systems using Machine Learning Approach

(1)

International Journal of Emerging Technology and Advanced Engineering

Website: www.ijetae.com (ISSN 2250-2459, ISO 9001:2008 Certified Journal, Volume 5, Issue 10, October 2015)

359

Applying the Technology Acceptance Model to Evaluation of

Recommender Systems using Machine Learning Approach

Oruan Memoye Kepeghom

1

, Madhu B.K

2

, Orie Maduabuchukwu John

3 1_{PhD Scholar, Dept. of Computer Science, Jain University Bangalore India.}

2_{Research Guide, Jain University & Professor and Head, Dept. of ISE, RRIT Bangalore India.} 3_{HOD Dept. of Computer Science, Federal College of Education (Technical) Omoku-Nigeria}

Abstract— Recommender systems unlike acceptance of

systems provide seemly accurate recommendations rather than factors influencing users acceptance of system. Moreover, accuracy on itself cannot account for absolute user satisfying outcome or experience. With the differences between the two concepts in the research, it adopt the technology acceptance model (TAM) to analyze user acceptance of a recommender system in the teaching and learning domain. A new latent variables representing ICT affection and availability (acceptance) to use a recommender system are introduced. The experiment involves 105 lecturers in four universities in the southern geopolitical region of Nigeria who responded to the shortlisted variables above after using ICTs recommended teaching facilities. The outcome substantiate that perceived ease of the system has more impact than perceived usefulness to motivate acceptance of recommendations. More so, users affection towards the system strongly influence perceived ease of use, which directly impacts on perceived usefulness of the system. The stated result can assist developer and administrator of recommender system in their attempt to maximize users' experience.

Keywords—Machine learning algorithms, Perceived

Affection, Perceived availability, Recommendation System, Technology Acceptance Model.

I. INTRODUCTION

The numerous and existing high tech applications available within the market place through the popularization of the internet has given room unlimited alternative of software and hardware applications available for the users especially in teaching and learning and has call for a higher stress of effective user acceptance. On this note it is valid for an effective recommender systems. Recommender Systems (RSs) are software tools and techniques providing suggestions for items to be of use to a user. Recommender system relates to various decision- making- processes, such as what items to buy, what music to listen to, or what news to read (F. Ricci et al 2010).

Recommender system mainly focuses towards individuals who lack sufficient personal experience or competent to evaluate the potentially overwhelming number of alternative items. RSs relatively new compared to research into classical information system tools and techniques has made tremendous impacts. Recommender system play an important role in the online movie rental service, awarded a million dollar prize to the team that first succeeded in improving substantially the performance of its recommender system (Netflix), the same trend if properly enhance can improve and effectively facilitate teaching and learning. Research has started to explore the factors that might have a direct impact to explore the factors that might have a direct impact on a user's acceptance of a recommendation technology. As earlier mentioned RSs are based on user's satisfaction, item attractiveness, accurate understanding of the user's preference etc. K. Swearingen and R. Sinha 2001 among others were the first to highlight that the effectiveness of RSs depends on factors that go beyond the quality of the prediction algorithm. The arising problem tends to be handled by various models, nonetheless TAM has standout and is the most widely used (M. Chuttur 2009).

This work focus at explaining potential user acceptance issue on a recommender system, applying TAM. TAM model as a baseline we incorporate some new latent variables as perceived affection, perceived availability to use a recommender system. An empirical user study using a classroom teaching recommender system as a test bed, as well as a questionnaire application to any recommender system in teaching and learning.

II. BACKGROUND

(2)

International Journal of Emerging Technology and Advanced Engineering

360

M.G. Armentano (2015) states that user acceptance is a complex concept that goes far beyond having an attractive and easy-to-use user interface. It has been proven that two systems with identical user interface might be perceived differently by users if the underlying recommendation algorithm is changed. F. D. Davis (1989) in an attempt to answer the question of what factors influences the acceptance or rejection of an information technology discovered that people will use an application if they believe it will help them to perform a given task better than when not using the application. More so, he found that even if users believe that a given application is useful, if the application is hard to use, then the perceived benefits of using the application are outweighed by the effort needed to use it. Both variable were regarded "perceived usefulness" and "perceived ease of use respectively. These finding resulted to the popular model adopted frequently by researchers known by Davis as the Technology Acceptance Model (TAM) an adaption of the Theory of Reasoned Action (TRA) which purpose is to handle issues of prediction of the acceptability of an information system. Various dimension or variables application of this model has been used, however, the purposed was to use the model to predict the acceptability of a tool and to identify the modifications that must be brought to the system in order to make it acceptance to users.

III. PREVIOUS WORK

TAM with recommender systems has been fused together to evaluate user acceptance of systems. R. Hu and P. Pu (2009) the authors evaluate an existing personality-based recommender system using the technology acceptance model. In their work focusing on music consider that when recommending music other factors such as emotion and mood have to be considered. Burke and Ramezani provided a new classification of recommender systems taken an AI-centric approach, and focus on the knowledge sources required for different recommendation approaches and the constraints related to them as a primer guideline to choosing the algorithm. Another area notable enhanced is the recommender approach towards recommender systems for technology- enhanced learning (TEL). TEL covers technologies that support all forms of teaching and learning activities, aims at designing, developing and testing new methods and technologies to enhance learning practices of both individuals and organizations. F. Ricci et al (2010), comment that TEL may benefit greatly from integrating recommender systems technology to personalize the learning process and adjust it to the user's former knowledge, abilities and preferences.

Technology enhanced learning which aims to design and test socio-technical innovation that will support and enhance learning practices combined with TAM will produce a remarkable outcome.

IV. METHODOLOGY

The research was conducted with an invitation to lecturers from four universities in the southern region of Nigeria. An introduction of a new variable for teaching recommender system. The work seem to balance the study by drawing dataset from lecturers in the area of Computer Sciences and other areas like Economics, Law and Education disciplines. Question formulated using the Likert-5 scale with 1 corresponding to " Strongly Disagree" and 5 corresponding to "Strongly Agree". The questions presented to participants along with the associated TAM variable are detailed as follows;

-AICT1. Internet services provided by the university (Afrihub & Others) are adequate.

-AICT2. Internet services provided by the university are reliable.

-AICT3. The university’s digital library is efficient. -AICT4. Links to educational resources websites like e-journals, e-books can be found on the College’s website. -AICT5. Computers and other ICTs are adequately provided.

-AICT6. Digital Video Disk prayers, Flash drives/External Hard drives and software are adequately provided

-PITL1. Effective utilization of ICT facilities improves students’ performance.

-PITL2. The use of ICT facilities for teaching and learning give better understanding to students.

-PITL3 Effective teaching will improve if all teachers have access to Internet facilities in their offices.

-PITL4. Teaching is very interesting when performed with any ICT equipment such as laptops, power point projector, clever board etc.

-PITL5. The practical approach of ICT in teaching and learning increases students’ learning/achievement.

-PITL6. ICT facilities provide all the materials needed for the students at the right time.

-PE1. Computer/internet can be easily used for teaching/Learning

-PE 2. Computer/Internet are efficient to use

-PE3. Sourcing for academic information through the internet is preferred to books

(3)

International Journal of Emerging Technology and Advanced Engineering

361

-PE5. Refer students to the internet to solve assignment

-PE6. Use computer simulations to aid teaching and learning

-PPAD1. Teaching and learning is more controlled with ICT facilities

-PPAD2. There is arousal in teaching or learning in the use of ICT facilities

-PPAD3. Using ICT facilities in teaching or learning gives me energy to proceed on and on

-PPAD4. ICT facilities create pleasure when using it in teaching or learning

-PPAD5. I have effective control when using ICT facilities in teaching or learning

[image:3.612.336.524.264.424.2]

-PPAD6. Using ICT facilities in teaching or learning gives me joy and pleasure.

Figure 1. Technology Acceptance Model ((Davis, Bagozzi and Warshaw, 1989, P985)

V. EXPERIMENT

TAM variables are often unobserved variables ( Latent Variables). The "Perceived Usefulness", "Perceived ease of use", "Perceived Affection" and "Perceived Availability" are variables that cannot be directly observed, but can be inferred from some indicators, and this is reviled by the questionnaire. These latent variables are modeled by specifying a model and a structural model. The classification model specifies the relationships between the observed indicators and the latent variables while the structural equation model specifies the relationships amongst the latent variables. More so, an analysis that consisted in x-raying the reliability and validity of the classification model and examine the significant and prediction of path coefficients in the structural model.

Reliability as noted by (Marcelo G. Armentano et al 2015) refers to the consistency of the item-level errors within a single factor.

A "reliable" set of variables will consistently load on the same factor. Cronbach’s alpha is considered to be a measure of scale reliability or internal consistency. Cronbach’s alpha can be written as a function of the number of test items and the average inter-correlation among the items. This metric measures how closely related a set of items are as a group.

Table I shows the Cronbach’s alpha coefficient correlation. The factors Cronbach-alpha is 0.887 which is higher than 0.7 for all factors, indicating that the reliability of data can be considered to be sufficient.

Table 1:

Cronbach's Alpha for Correlated Factors

VI. SVMANALYSIS

Support Vector Machines (SVM) are an analytical tool that can be used for both classification and regression purposes (Vladimir N. Vapnik, 2000). The analytical tool has gained considerable popularity within years. SVM are particularly known for their ability to tackle the standard problem of over fitting, especially in multivariate settings.

Support Vector Machine algorithms are used in classification, such classification can be viewed as the task of optimizing boundary separating classes in feature space. Usually SVM uses three or more support vectors to obtain the boundary.

The equation for the hyper plane that discriminates the positive class from the negative class is given by;

∑

and the separating hyper plane equation

y = x + b

where is the hyper plane b is an offset

AICT PE PITL PPAD

AICT 1

PE 0.780 1

PITL 0.742 0.877 1

PPAD 0.713 0.792 0.858 1 Perceived

Usefulness

Actual System Use

Perceived Ease of Use

Behavioral Intention External

[image:3.612.47.298.327.438.2]

(4)

International Journal of Emerging Technology and Advanced Engineering

362

Table 2:

[image:4.612.349.579.141.691.2]

Decisions mined from SVM analysis

Figure 2: Snap shot of SVM Analysis " Perceived Affection"

AICT - Availability of ICT infrastructure PE - Perceived Ease of Use of ICT

PITL - Perceived Impact of ICT in Teaching and Learning PPAD - Perceived Pleasure/Arousal/Dominance of ICT facilities

[image:4.612.43.284.172.595.2]

Figure 3. Random Forest Tree

Table 3:

Rules mined from Random Forest Tree

IF RATING THEN PROBABILIT

Y (%)

PE <= 2.0 IAUE 50

PE >2.0 RSUST 38.6

AICT & PE =>4.5 & >2.0 RSUST 42.9

AICT & PE <=4.5 &

=>2.0

RSUST 37.8

AICT & PE & PPAD

<=4.5 &

=>2.0 &

<=4.5

RSUST 34.8

AICT & PE & PPAD

<=4.5 & >2.0 & =>4.5

RSUST 80 ***

AICT & PE & PPAD

<=4.5 & >2.0 & <=3.5

IAUE 44.8

AICT & PE & PPAD

<=4.5 & >2.0 & (3.5,4.5)

FCET 27.5

AICT & PE & PPAD

<=4.5 & >4.5 & (3.5,4.5)

FCET 60 ***

AICT & PE & PPAD

<=4.5 &

2.0,4.5 &

(3.5,4.5)

RSUST 31.4

AICT & PE & PPAD

<=2.5 &

2.0,4.5 &

(3.5,4.5)

UNIPORT 57.1 ***

AICT & PE & PPAD

(2.5,4.5) & (2.0,4.5) & (3.5,4.5)

RSUST 39.3

AICT & PE & PPAD

<=3.5 & >2.0 & <=3.5

IAUE 50

AICT & PE & PPAD

(3.5, 4.5) & >2.0 & <3.5

RSUST 44.4

ATTRIBUT -ES

RATING CLASS META

ATTRIBUTE-S

PITL PITL PILT

4 -Agree 5 -Strongly Agree 5 -Strongly Agree

UNIPORT RSUST FCET

SCIENCE SCIENCE ARTS

PE PE PE PE

4 -Agree 4 -Agree 4 -Agree 4 -Agree

FCET IAUE RSUST UNIPORT

SCIENCE ARTS ARTS SCIENCE PPAD

PPAD PPAD PPAD

4 -Agree 1-Strongly Disagree 4 -Agree 4 -Agree

ARTS SCIENCE SCIENCE SCIENCE

AICT AICT AICT AICT

3-Undecided 3-Undecided 3-Undecided 4 -Agree

[image:4.612.347.577.359.680.2]

(5)

International Journal of Emerging Technology and Advanced Engineering

363

VII. DISCUSSIONS OF RESULTS

Table 2 and Figure 2 gives SVM analysis and breakdown regarding the four universities where the analysis was conducted. From table 2 the decision mined portrays that "perceived usefulness (impact)" is "agree and strongly agree" upon by the class of three universities with two science faculties and arts respectively, while "perceived ease of use" is agreed by all the institutions. Except for one institution that "strongly disagree" with "perceived affection" other universities agreed on the influence of perceived affection as a determinant for adopting TAM. Availability of ICT infrastructure from three institution indicates inadequate and low proportion, except for University of Port Harcourt (UNIPORT) lecturers that agreed on availability.

More so, Table 3 and Figure 3 representing the random forest tree and rules mined from random forest tree reveals that strong classification between "perceived ease of use of ICT" and "perceived pleasure arousal driven from the use of ICT" for effective teaching and learning with 80% probability while the same percentage is short of "ICT availability" in Rivers State University of Science and Technology (RSUST). Federal College of Education Technical Omoku (FCET) also shows a closed tie classification with "perceived ease of use" with "perceived pleasure arousal" with 60% strongly agreeing respectively. The same result is reveals of University of Port Harcourt (UNIPORT) lecturers with 57.1% strongly agreeing a classification on both concepts. Except for Rivers State University of Science and Technology (RSUST) with acceptance rate 42.9% regarding ICTs availability, all other institutions portrays a low rate of ICTs availability acceptance with undecided as an option.

VIII. RECOMMENDATION AND CONCLUSION

This work is a lecturer version of an ongoing research work to study four higher institutions in the southern geo-political region of Nigeria to reveal the impact of a recommender system (ICTs) adopting a new latent variables based on TAM to enhance the effectiveness teaching and learning if timely and properly used. With a similar research conducted for students in the stated institutions, the researchers performed an experiment with some commonly used ICT facilities to enhance teaching and learning. Participants responded to a post treated questionnaire related to a set of variables that influence each latent variable in TAM and new latent variables corresponding to "Perceived affection or arousal, availability and inhibitors" were in use of the recommender system.

The machine learning algorithms adopted show a confirmatory evidence that validate the fact that the data fit adequately in the proposed model though some new latent variables in some universities varies as the case may be. The outcome of the experiments confirmed that perceived usefulness plays a predominant role for users to accept a new recommender system, as proposed in TAM. ICT availability is a key player in the institutions evolution to improve teaching and learning as perceived ease of use is agreed upon by these institution in the use of ICTs. The result also reveals that at least an institution reflex the fact that perceived affection has a strong correlation with perceived impact that is usefulness in the analysis. There are strong inhibitors that tends to mediate on the application of the model to effectively impact teaching and learning as reveals by the questionnaire finding are teachers phobia to use the recommender system, lack of manpower (well trained teachers), electricity problems etc. These findings would be found effective and useful to recommender systems developers both in the academic and system designers for commercial use.

REFERENCES

[1] Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–339.

[2] F. D. Davis, ―Perceived usefulness, perceived ease of use, and user

acceptance of information technology,‖ MIS Q., vol. 13, no. 3, pp. 319–340, Sep. 1989.

[3] F. Ricci, L. Rokach, and B. Shapira, Introduction to Recommender

Systems Handbook. Springer, 2011, ch.1, pp. 1–35.

[4] K. Swearingen and R. Sinha, ―Beyond algorithms: An HCI

perspective on recommender systems,‖ in ACM SIGIR Workshop on Recommender Systems, vol. 13, 2001, pp. 393–408.

[5] R. Hu and P. Pu, ―Acceptance issues of personality based

recommender systems,‖ in Proc. of ACM RecSys’09. New York, NY, USA: ACM, 2009, pp. 221–224.

[6] Marcelo G. Armentano, Ingrid Christensen and Silvia Schiaffino,

"Applying the technology acceptance model to evaluation of recommender systems 2015. http://dx.doi.org/10.17562/PB-51-10 * pp. 73-79.

[7] M. Chuttur, ―Overview of the technology acceptance model:

Origins, developments and future directions,‖ Working Papers on

Information Systems, vol. 9, no. 37, pp. 1–22, 2009.

[8] M. G. Armentano, R. Abalde, S. Schiafﬁno, and A. Amandi, ―User

acceptance of recommender systems: Inﬂuence of the preference elicitation algorithm,‖ in 9th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), Nov. 2014, pp. 72–76.

[9] Vladimir N. Vapnik. The nature of statistical learning theory.