7.2
Achievement of the Research Objectives
This section will discuss the extent to which each of the research objectives presented in Section 1.2, has been achieved.
Identify the challenges within e-Learning recommendation by performing a critical review of research in recommendation with a focus on issues in e-Learning recommendation. The semantic gap and intent gap faced by learners have been identified as two main issues that make e-Learning recommendation challenging. The semantic gap presents itself through the lack of a shared vocabulary between learners and domain experts. While the intent gap happens because learners lack sufficient knowledge about what topics are suitable for them when searching for relevant learning materials.
Standard recommendation approaches should be implemented differently when considering e-Learning recommendation. Typically, collaborative filtering systems rely on the preferences of users with similar interests for making predictions. However, in e-Learning recommendation, the need of a learner captured through a query would have to be considered for suitable recommenda- tions to be made. In content-based systems, users are often recommended items that are similar to those previously consumed. For e-Learning systems, you do not wish to recommend more of the same learning materials that a learner has read. Instead, the recommendation should be based on the learner’s current query.
The item-user pair in recommender systems can be mapped to the learning resource-learner pair in e-Learning recommenders. However, finding relevant materials is challenging due to the complex features that are often used to describe learning materials and learners. Learning re- sources are largely text which presents challenges of dealing with unstructured data and indexing the learning resources for retrieval. Further, the vocabulary used in the resources is often different from that used by learners, thus making it difficult to find and retrieve relevant learning resources for learners. A key feature that is important for learners is the learning goal, which is often cap- tured through a query. This query should be taken into account when making recommendations because it is supposed to capture what a learner wishes to learn. However, learners often find it difficult to craft an effective query because they lack sufficient knowledge of the domain. This research explores approaches that support e-Learning recommendation to enable learners find rel- evant learning resources.
7.2. Achievement of the Research Objectives 118
Address the semantic gap by exploring how to provide a shared vocabulary between domain experts and learners in order to enable learners find relevant materials
The semantic gap is addressed by introducing a novel method that automatically creates back- ground knowledge in the form of a set of rich learning concepts related to the selected learning domain. The identified concepts provide a vocabulary and focus that is based on teaching materi- als with provenance. The concepts in the background knowledge represent important topics that learners should be interested in. The background knowledge is employed to influence retrieval in the recommendation of new learning materials by leveraging the vocabulary associated with the concepts during the representation process.
A CONCEPTBASEDdocument representation approach employs the concept vocabulary only for representing documents. However the initial CONCEPTBASED document representation ap- proach had a limited number of concepts, so its vocabulary was too restricted for concept-based distinctiveness to be effective. An augmented document representation approach leverages a vo- cabulary from both concepts and documents for representing documents. The augmented approach exploits differences between distributions of document terms in the concept and document spaces, in order to boost the influence of terms that are distinctive in a few concepts. The vocabulary from both concepts and documents is focused using the vocabulary from the concept space. Evaluation results show that augmenting the representation of learning resources with the concepts addresses the semantic gap by providing a shared vocabulary between learners and experts. This work won the Donald Michie Memorial Award for the Best Technical Paper at the BCS AI International Conference (Mbipom et al. 2016).
The background knowledge is enhanced by refining the method used to generate the domain concepts. The output is a richer set of domain concepts which is used to develop the enhanced CONCEPTBASED* document representation method. The richer concept vocabulary in CONCEPT- BASED* provides a better coverage of the domain when employed in the representation and re- trieval of documents. The benefit of the enhanced background knowledge is evaluated using a col- lection of Machine Learning and Data Mining documents. Our approach outperforms benchmark methods, demonstrating the advantage of using background knowledge for representing learning materials which enables learners to find relevant materials during e-Learning recommendation. This work has been published in a Special Issue of the Expert Systems Journal (Mbipom, Craw & Massie 2018).
7.2. Achievement of the Research Objectives 119
Address the intent gap by exploring effective methods to help learners identify relevant top- ics in order to support learners to ask useful queries when finding learning materials The development of an approach that refines learners’ queries by identifying important learning topics is a key contribution that addresses the intent gap. Query refinement is often done ex- plicitly, where a learner has to choose which topics are relevant. Such approaches are difficult because learners do not usually know what topics are relevant. The approach in this research is done implicitly so that we do not rely on learners, who often have insufficient knowledge about what they are looking for. A knowledge-rich representation containing important learning topics has been generated from learning materials in the form of concepts in our background knowledge. The approach employs background knowledge by leveraging concepts that are similar to queries and distinctive concept terms for refining learners’ queries. This allows the search using a refined query to focus on topics that should be relevant for a given learner’s query. So, the refined queries enable learners to ask effective queries and find relevant learning materials.
A recommender system is built to demonstrate the recommendation of learning materials. The developed system allows us to evaluate the effectiveness of the query refinement approach. A collection of queries and a dataset of Machine Learning and Data Mining documents are used for evaluation. The evaluation is not a standard user trial with learners, because the users had to be knowledgeable in the chosen domain, to be suitably qualified to give relevance judgements. Relevance judgement is subjective because it depends on the opinions of the users taking part in the evaluation. However, we had a good level of consensus on the relevance judgements provided by users with different levels of expertise. The results from experts, competent users and beginners all showed that using learning concepts to refine queries achieved effective queries. The search using our refined queries produced documents that were consistently more relevant than when the standard method was used.
An investigation of the coverage of relevant topics across the query-recommendation pairs showed that most of the recommendations covered topics that were relevant to the query. A com- parison of the relevance and coverage scores generally showed that documents that had high rating scores associated with them also had good coverage scores. The evaluation results demonstrate the effectiveness of our approach to support e-Learning recommendation, by recommending relevant learning materials that contain a good coverage of topics that are relevant to the queries evaluated. This work was presented at the EAAI symposium at AAAI (Mbipom, Massie & Craw 2018).