Initialised GBRT using Random Forests (IGBRT)

This application is a hybrid pointwise LTR technique (Mohan et al., 2011). It uses the ranking model produced by the pointwise RF technique in the initialisation procedure of

the GBRT technique. Mohan (Mohan et al., 2011) developed this technique with point- wise GBRT and pointwise RF in rt-rank package (Mohan et al., 2011). This technique has been introduced in Yahoo challenge for LTR problems (Chapelle and Chang, 2011). It was in comparison with LambdaMART, AdaBoost and YetiRankand approaches. The LambdaMART outperformed IGBRT technique regrading reciprocal rank evaluation val- ues in this challenge. This comparison did not consider the computational time of each technique. However, IGBRT outperformed GBRT and RF in terms of receiprocal rank. From our findings, IGBRT technique also consumes considerable computational runtime for large LETOR datasets.

4.3

Chapter Summary

In this chapter, the related works ofEvolutionary Computation (EC)andMachine Learn- ing (ML) in TVM and FVM are presented. The chapter started with the TVM and in- troduced the limitation of applying EC and ML in TVM for evolving term weights. The limitation is summarised in the problem size and the computational run-time for evolving the whole document representations. Besides to the relevance judgement limitation for applying EC or ML techniques at the beginning of establishing a new IR system. These cause the need for proposing a new methodology for evolving document representations using EML techniques. This Chapter also introduces the various LTR techniques on FVM. In this research domain, the EML techniques are used to learn the most suitable ranking model weight for the training data and testing the performance of the techniques on the test data. The literature research was not stated the experimental settings nor the training computational run-time in their comparison. Furthermore, most of the EML techniques give high accuracy more than other heuristic techniques. The following chapter presents a new TWS called TF-ATO and it shows the heuristic issues caused by the pre-processing procedure in IR.

Term Frequency With Average Term

Occurrences (TF-ATO)

5.1

Introduction

In the context of Information Retrieval (IR) from textual documents, the term-weighting scheme (TWS) is a key component of the matching mechanism when using the TVM (Term Vector Model) representation. At the beginning of establishing an IR system, there is a need for using a non-learning (mathematical) term-weighting scheme (TWS). This is because there are no relevance judgement values provided by the users for the IR test collection. The preferable non-learning TWS is the term-weighting function that considers most of non-noisy document words as index terms and assigns a discriminate weight value for it. As discussed in Section2.2.2, an effective TWS is crucial to make an IR system more efficient. There are various TWS proposed in the literature and some have been implemented in search engines. To the best of our knowledge, the most widely used approach is the term frequency-inverse document frequency (TF-IDF) as a non-learning TWS. However, TF-IDF and its variations may remove some significant keywords before user relevance feedback is gathered by the IR systems. This may cause the bias of relevance judgement values based on the used TWS in the IR system (Buckley et al.,2007;Urbano,2016).

demonstrated in subsection3.1.3. This analysis shows that test collections are not fully judged as achieving that is expensive and may be unfeasible for large collections at the early stage of establishing the IR system. A test collection being fully judged means that every document in the collection acts as a relevant document to a specific query or a group of queries. Some Evolutionary Computation (EC) techniques have been used for evolving TWS or evolving term weights using those test collections (Cummins, 2008; Cordon et al., 2003). However, such approaches have an important drawback. These EC approaches usually use the relevance judgements for the test collection on their fitness functions for checking the quality of the proposed solutions. The relevance judgement of a collection gives the list of relevant/irrelevant documents for every query. Furthermore, the real IR test collection have not relevance judgement values at the beginning of IR systems. This means that TWS can not be evolved at the beginning of establishing IR system. This provokes that when using EC techniques most documents have random term weight representations. In addition, TWS evolved with Genetic Programming (GP) as in (Cummins and O’Riordan,2006;Cordon et al.,2003) are based on the characteristics of the test collections and hence, not easily generalisable to be effective on collections with different characteristics.

This is what motivates the work presented in this chapter on the development of such the proposed TWS. In this work, theTerm Frequency With Average Term Occurrence (TF- ATO)is proposed which computes the average term occurrences of terms in documents and uses aDiscriminative Approach (DA)based on the document centroid vector to re- move less significant weights from the documents. This TWS does not require any prior knowledge about relevance judgement values.

This chapter evaluates the performance of TF-ATO and investigates the effect of stop-words (or negative words) removal (Fox, 1992) and the DA as procedures for removing non-significant terms and term weights in heuristic TWSs. The performance of the proposed TF-ATO and the well-known TF-IDF approach are compared in this chapter. It is shown that using TF-ATO results in better effectiveness in both static and dynamic test collections. In addition, this chapter investigates the impact that stop- words removal and our DA have on TF-IDF and TF-ATO. The results show that both,

stop-words removal and the DA, have a positive effect on both term-weighting schemes. More importantly, it is shown that using the proposed DA is beneficial for improving IR effectiveness and performance with no information in the relevance judgement for the col- lection. The intended contributions of this chapter are contributions 2 and 3 in section1.4.