Despite our great effort to improve argument mining and uncover its potentials in educational application, the thesis still exposes limitations that we hope to resolve in future work.
First of all, our research has not answered how the quality of argument/domain word lexicons might affect the argument mining performance. In Chapter 5, we compared the extracted argument words with the set of argument seed words to test the lexicon cover- age. However, such test did not handle the precision of the lexicon. In other words, we did not evaluate the quality of our extracted argument words, and how the argument min- ing accuracy will be if we adjusted the lexicon extraction algorithm. Second, our lexicon extraction algorithm is semi-supervised in that it needs a set of argument seed words to
start. This makes our algorithm not easy to adapt to other writing genres such as online debates or product reviews. In the future, we first plan to revise the argument and domain word extraction algorithm by first automating the argument keyword selection phrase. One solution is to use argumentative discourse markers to initiate the process. Second, we will compare our lexicon extraction algorithm with other approaches for argument and domain word learning proposed in prior studies (Madnani et al.,2012;S´eaghdha and Teufel,2014) in terms of how the extracted argument/domain words help improve argument mining models. It is also worth measuring the relative precision and coverage of lexicons learned by different approaches to see how their quality impacts argument mining accuracy.
Secondly, there is still room for improvement in window-context features for argumen- tative relation mining. This thesis has proposed two approaches to create context-windows including window-size heuristics and text segmentation. Experimental results (Table 17) showed that each approach worked better for one of the two data sets. Thus, one question to investigate in future work is whether quality of text segmentation output has impact on the effectiveness of window-context features. To answer this, we plan to manually compare text segmentation output in persuasive essays and academic writings and conduct an error analysis for argumentative relation mining with respect to window-context features. Beside hard-boundary context windows as we have studied, soft-boundary window (i.e., shaped windows) is an interesting idea to explore next. The basic idea is that each context sentence will have a probability of belonging to the context window of an argument component. This membership probability can be inferred from the data and may depend on the distance, discourse relation and content relatedness between the context sentence and the argument components. We see research in probabilistic topic models, text segmentation and discourse parsing are valuable resources for us to develop an approach for this idea.
Our other follow-up plan is a joint model that labels argument components and argu- mentative relations simultaneously to take advantage of mutual information between the two problems. Joint prediction has shown great successes in argument mining and we believe such a mechanism will further improve our context-aware argument mining models. Also, an interesting direction is to apply deep learning in argument mining. Researchers have used word embeddings as feature vectors for argument component classification. We are thinking
of exploiting deep learning to model argumentative discourse relations in an unsupervised way. With great availability of free texts, we believe to have more than enough pairs of sentences/clauses with explicit discourse markers that likely signal argumentative relations, e.g., because, therefore, but, however. We now need a learning mechanism to create an relation embedding that can represent the deep semantics of the relevance in each pair. For this ambition, we see recent research on sentence embedding, representation learning for discourse parsing as valuable resources.
In parallel with continuously improving argument mining models, deploying argument mining in real-world tasks is also of interest. A possible direction is to use argument min- ing for a complete assessment of student argumentative writing in the SWoRD peer review system. However, this would need a larger corpus of academic writing annotated for argu- mentation structures, and also the same or other writings graded by experts. We believe that with larger data sets for argument mining and essays with expert scores, we can have a stronger evidence of the effectiveness of argumentation features for academic essay score prediction.
Finally, expanding our context-aware argument mining research to different writing gen- res beyond persuasive essays, and with broader concepts of context is our long-term vision. We believe our proposed topic- and window-context features are applicable to wide range of text genres, e.g., online debate and product reviews. In fact, researchers have studied argu- mentative relations between user comments in a debate but have not yet considered the full potential of discourse structures among comment sentences. Moreover, topic information is generally available in such texts, e.g., debate topics, product keywords, user opinions. Thus, we can adapt our algorithm to extract argument and domain words from texts. On the other hand, the scope of context can be expand to beyond the documents. To identify argument moves in online debates, or evidences in user comments, we should not limit ourself to the textual domain. External contexts such as thread structure, user activity history, and other metadata can be useful. Therefore, an ultimate model should be the one that exploits as much context as the data provides to get the most insights of the data.
APPENDIX A
LISTS OF ARGUMENT WORDS