6.1.1 Conclusions
Recent performance improvements in supervised learning call for the improvement of the interpretability of such models and their results. Therefore, much research has proposed ways to facilitate the understanding and explanation of models. However, considerably less attention has been given to the development of interpretability in unsupervised settings. This dissertation takes the initiative to look at the interpretability under the umbrella of unsupervised learning, and we propose a M3 framework towards the interpretability in unsu- pervised pattern mining. We select multi-aspect data because an increasing amount of data generated has various multi-aspect characteristics. We use tensor factorization to demon- strate the proposed framework since it is one of the most popular techniques for uncovering
patterns in multi-aspect data.
The need to interpret for unsupervised mining is critical because of several challenges. 1) Mining with the mismatch between human information need and reconstruction errors. Simply applying an off-the-shelf mining toolkit does not respect the particular information need of the users. 2) Mining with insufficient evaluation criteria. Current evaluation schemas undergo either qualitative examination of outputs (e.g., topic modeling) or using downstream tasks as a proxy to measure mining quality (link prediction in graph representation learning). 3) Mining with the mismatch between experts’ domain knowledge and data-driven models. Data can be noisy. Even if a model is tuned to the users’ information need, the results may not readily translate to something domain experts’ can agree upon. 4) Mining with the mismatch between underlying multi-aspect pattern and human understandability. Patterns from multi-aspect data require its interpretation simultaneously from multiple perspectives. Different presentations of a pattern can vary in experts’ ability to understand them.
These challenges stimulate this dissertation. A M3 framework of pattern discovery from
multi-aspect data was proposed to address each challenge. To ease the mismatch between human information need and reconstruction-oriented factorization, we propose the multiplex pattern discovery component. In this component, the information need is operationalized through a regulative tensor factorization model that is tuned to users’ information needs. To ease the evaluation of patterns from tensor factorization, we design a multifaceted pattern evaluation schema, where patterns are validated from multiple perspectives: quality, validity, and utility, where quality stands the overall quality of tensor factorization, the validity suggests how well the patterns can be explained by the experts’ domain knowledge, and the utility evaluates the applicability of patterns in downstream tasks. To further close the gap between human interpretability and interaction with the factorization process, we propose a visual analytic system as a united approach to simultaneously address all the challenges.
This thesis introduced three studies to build up the components of the interpretation framework and demonstrate its effectiveness. In the first study, we situated the requirement of a carefully crafted model to cater to the information need in an urban space in the af- termath of the major events. Compared to a participatory assessment of the impacts of events [176], there is a crucial need for a data-driven model to reveal the impacts quickly.
With the increasing amount of multi-aspect data becoming available, we formulate the in- formation need as a contrasting pattern discovery problem given multi-aspect mobility data from before and after major events in the city. We design a collective tensor factorization model, PairFac, to uncover the shared phenomena and discriminative phenomena. Pair- Fac takes multi-aspect data as input and split into two groups, before and after certain events. We apply PairFacin two case studies and demonstrate its capability to reveal per- sistent and changing mobility patterns following events of interest. For example, in our first case study, using data from the terrorist attacks in Paris of 2015, we see that activities around professional life and food venues experienced the fewest changes.
In the second study, we target the information need in the domain of understanding contrasting user behavior patterns in massive online courses (MOOC), with a particular interest in identifying the relationships between underlying behavioral patterns and perfor- mance outcomes. We propose a tensor-based learning method, iDisc, that discovers the common and discriminative learning patterns at multiple levels. Based on this, it projects users to a latent space (i.e. embedding for the downstream prediction tasks) to identify the association between the multi-way interaction of the features and the students’ performance. The empirical studies with the dataset from different MOOC platforms have shown that iDisc yields promising results on the effectiveness and efficiency.
In our third study, we propose a visual analytic system, FacIt, to simultaneously address all the mismatches we identified with pattern discovery from multi-aspect data. The system is built to meet common requirements, such as model selection, model refinement, results scrutinization, and interpretation for its real-world applications. We develop a suite of model scrutinization and inspection tools that empowers the model selection process. A novel weakly semi-supervised tensor factorization algorithm is proposed to allow human-in-the-loop tensor factorization discovery. In addition, we provide an interactive design that caters to experts’ different exploration strategies, such as characteristics- and content-driven pattern discovery. The effectiveness and usefulness of FacIt have been evaluated through usage scenarios across different domains, followed by in-depth interviews with domain experts.
6.1.2 Contributions
In general, I believe that there are several key contributions of this dissertation:
• An understanding of the challenges of interpretability in unsupervised pattern discovery. To the extent of my knowledge, while interpretability in machine learning has become an increasingly known issue, this dissertation is the first to explore the problems of interpretability in unsupervised settings.
• A framework to address the challenges of interpretability in unsupervised pattern discov- ery. By understanding of challenges, this dissertation provides the first attempt to create a framework of interpretable pattern discovery from multi-aspect data. The M3 frame-
work consists of three components to address the identified challenges: 1) multiplex pattern discovery to close the gap between human information needs and standard pat- tern discovery tools; 2) multifaceted pattern evaluation to validate patterns via multiple approaches; and 3) multipurpose pattern presentation to close the gap between pat- terns and human understandability, and allow experts to provide feedback in the loop of pattern discovery.
• A demonstration of the effectiveness of the proposed framework with three studies. The dissertation contributes three studies that are incrementally organized to demonstrate the use of the framework in solving real-world problems of interpretability where human information needs intersect with pattern discovery from multi-aspect data.