Summary of the Work

This research has proposed three Thai word similarity measures, two Thai word benchmark datasets, one Thai sentence similarity measure, and one Thai sentence benchmark dataset. These are the outcomes of the work to answer the research questions as follows:

• Can a semantic-based Conversational Agent be developed in Thai?

Chapter 2 established that this research question cannot be given an immediate answer ‘YES’. The Thai language simply does not yet have the resources to support this. Therefore, the main focus of this work is to create a suitable framework to support future work in the development of Conversational Agents. This chapter provided a background to this thesis that introduced related research, including English word similarity measures, non-English similarity measures, English sentence similarity measures, non-English sentence similarity measures, the fundamentals of the Thai language and the current state of research in Thai WordNet. Also, the potential for a Thai similarity measure was reviewed and discussed. This found no research about Thai similarity measures. Therefore, as a starting point to develop a new Thai similarity measure, the STASIS architecture was selected.

• Can an English word similarity measure be developed for the Thai language by translating Thai words into English?

Chapter 3 proposed the first Thai word measure (TWSS), which was developed directly from Li’s measure (Li et al., 2003). This work answers this research question. TWSS was created based on the conversion of Thai words to English for Li’s measure to be applied. Moreover, a 30 Thai word pair benchmark dataset (TWS-30) was also presented in this chapter. In an evaluation of TWSS with TWS-30, a correlation coefficient of 0.823 (P- value < 0.01) was obtained, providing supporting evidence for the research question. This result was promising. However, this measure could not be used to fully predict those word pairs that relate to Thai culture as TWS-30 was built based on an English dataset

172

(Rubenstein and Goodenough, 1965). Thus, to experiment on words relating to the Thai culture, a more effective evaluation is needed.

• Can a WordNet based English word similarity measure produce a similarity rating between words based on Thai culture?

Chapter 4 presents the methodology for the creation of a 65 Thai word pairs benchmark dataset based on Thai culture (TWS-65) which addresses this research question. The evaluation of a subset of TWS-30 and TWS-65 human ratings has shown that both datasets are not significantly different. In addition, a correlation coefficient of 0.807 was obtained between TWS-65 human ratings and TWSS ratings. TWSS uses the English-based WordNet to perform the rating. Hence, it results in an inefficient rating performance of these word pairs which are related mainly to the Thai culture. Therefore, the limitations of TWSS mean it should be considered as a pathway to a final Thai word similarity measure.

• Can a search engine provide an alternative natural language resource for a Thai word similarity measure?

Chapter 5 presented the investigations undertaken in considering this research question. This chapter proposed a word similarity measure based on a lexical chain that was created from a mini corpus produced by a search engine (LCSS). The aim of this algorithm is to overcome the problem with TWSS. A training dataset (TWS-30) and a testing dataset (TWS-51) were also presented. The training dataset was used to find the most suitable

Alpha parameter in LCSS. The testing dataset was used to evaluate the LCSS algorithm. A

correlation coefficient of 0.723 (P-value < 0.01) was obtained. Both the TWSS and LCSS perform quite well on their own. However, evidence shows that each contributes a different insight into the similarity process. Therefore, a combination of TWSS and LCSS may be more effective.

• Can a combination of TWSS and LCSS provide a better model of human perception of Thai word semantic similarity than either separately?

Chapter 6 proposed a word measure that was created from a combination of TWSS and LCSS, called nTWSS, to addresses this research question. The correlation coefficient between nTWSS ratings and TWS-51 human ratings was r = 0.867 (P-Value < 0.01), a significant improvement on TWSS or LCSS alone. Accordingly, nTWSS can be used to develop a Thai sentence similarity measure.

173

Chapter 7 and Chapter 8 presented the investigations undertaken in considering the research question

Chapter 7 presents the first Thai sentence benchmark dataset (TSS-65). Following O’Shea’s procedure (O’Shea, 2008), TSS-65 was created by replacing the words with a definition, shown in Section 7.2.1. Comparing TSS-65 and TWS-65, a correlation coefficient of 0.896 (P-value < 0.01) was obtained indicating general consistency between derived sentences and the word. This paved the way for the development of the new Thai sentence similarity measure.

Chapter 8 proposed the first Thai sentence semantic similarity measure (TSTS). The sentence measure is developed from STASIS; nTWSS is used to calculate the semantic similarity between two words. Word order is not taken into account. In this measure, the correlation coefficient between TSS-65 and TSTS was r = 0.809 (P-value < 0.01).

• Is the developed Thai sentence similarity measure feasible for use in developing Thai Conversational Agents

In Chapter 8, an experiment was conducted to answer the research question: ‘Is the developed Thai sentence measure feasible to use to develop Thai Conversational Agents?’ In this simple experiment, TSTS predicted the categories low, medium, and high similarity with 80% accuracy between sentence pairs from a Conversational Agent log file. Furthermore, the crucial function of STSS in a Conversational Agent is to find rules that capture attributes accurately; therefore, the higher performance of TSTS in this circumstance is important. Medium and Low similarity matches against rules normally leading to disambiguation of user meaning or the firing of off-topic ‘chat’ rules. Because Conversational Agents inherently do multiple interactions to disambiguate misunderstandings, this is approaching a usable performance. Therefore, this supports the view that TSTS could be used to create Thai semantic-based Conversational Agents.