Here, we see the benefits of considering entities and terms separately. While an increased context window size is sensible for entities, it would introduce noise for terms that are less likely to be related across sentence boundaries. We see this as confirmation of the potential behind an approach that uses different window sizes for entities and terms.
Overall, as a result of this comparison, we find implicit networks to be well suited for the exploration of entity relations in document collections. In the following, we therefore consider this exploratory aspect on the network level, which embeddings cannot offer.
3.6 Summary and Discussion
In this chapter, we introduced the implicit network representation of large document col- lections as the foundation of subsequent, more application-driven contributions in the remainder of this thesis. We did not go into application-specific details or data, and in- stead applied the network model for generic tasks. Specifically, we evaluated and tested the approach on Wikipedia, primarily because it is an enormous and free source of un- structured text that can be cleaned to remove noise, and thereby allows for an assessment of the scalability of our model. However, note that the implicit network model itself can be used to represent an arbitrary document collection, since it makes no assumptions about the documents’ structures or origins. In particular, any entity-centric approach is bound to perform better on a corpus or document collection with a narrower scope of topics due to the improved performance of named entity recognition tools that can capitalize on the re- duced ambiguity. Therefore, the performance of retrieval tasks on the network also stands to improve in such settings, in contrast to the very diverse Wikipedia data.
Practical implications
Recall the journalists and data analysts from our introduction in Chapter1that are work- ing on such data as the Panama Papers or the FBI’s investigation into collusion during the U.S. presidential election of 2016. For them, implicit networks offer the potential of looking at their data from new angles. By modelling their unstructured textual data as an implicit networks, they can now extract potential relations between arbitrary entities in the documents and consider them in context. Unlike our experiments, where we used typ- ical named entities and standard entity annotators, such practical applications can include any type of (named) entity and any type of annotation process. For small data sets, manual annotation might be viable. However, a more likely scenario is gazetteer-based annotation,
3 Implicit Entity Networks
in which lists of entities are matched against the text to annotate their occurrences and construct a network. Considering the practical application scenarios, it becomes apparent how comprehensive the notion of entity can be, and how it can include anything from the traditional names of persons or locations, to financial concepts, individual groups of people or factions, or even to relevant parts of technical terminology.
The network-centric view on any type(s) of such entities then provides the analysts or journalists with the ability of uncovering latent connections between focal concepts. By focusing on some entities or terms as input queries, they can explore the implicit re- lations in their neighbourhood and discover relations between them. However, not only does the network allow them to uncover and visualize such non-obvious relations, the fact that implicit networks also include structural elements, such as sentences or documents, even provides provenance information for these mentions. As a result, it becomes possi- ble to start at entities or terms of interest, and identify documents with relevant content that mention implicit relations and contexts of which these entities are a part. Thus, the network model reduces the insurmountable task of reading thousands of documents to browsing a handful of documents that serve as starting points for uncovering previously unknown relations and affiliations.
Outlook
Beyond the domain of predominantly static input data that we have considered so far, such as Wikipedia dumps, an application of the model in a streaming or online setting is possible. There, it stands to benefit from the underlying graph representation, which allows incremental updates to the graph as new documents arrive in the stream. The addition of new documents to the model is possible in real-time, simply by processing the document and adding the resulting subgraph to the main graph structure. Even edits, additions, and deletions in the individual sections of a document can easily be accounted for. Since the adjustment of edge weights happens locally within the graph structure, a change graph can be computed for a document before and after an edit, and then used to update the implicit network. Thus, the model is able to handle the processing of frequently edited document collections as well as streaming sources such as news feeds.
Based on these deliberations, we focus on applications and refinements of the model in the following chapters. We begin by presenting further applications for implicit net- works in practice on the example of two use cases in Chapter4. Afterwards, we discuss an extension of the model to dynamic document collections and consider possible applica- tions for implicit network models of streams of news articles in Chapter5, where we also
3.6 Summary and Discussion
address partial edge aggregation techniques and document time stamps. In Chapter 6, we investigate how implicit networks can provide a novel angle on topic detection in dy- namic document collections, and how they support an interactive exploration. Finally, in Chapter7, we consider a generalization of entity and term cooccurrences to a hypergraph network model and provide a formal basis for expressing queries to such a representation of documents.
4
Applications of Implicit Networks
Using the implicit network representation of entities and terms in documents that we introduced in the previous chapter, we now consider applications of this document model. To this end, we focus on two application scenarios in particular. First, we demonstrate the efficiency and effectiveness of the representation as both an index structure and a data source for localized explorations around entities and terms in the network by showcasing an entity-centric search engine that works on top of the network representation. Second, we focus on the extraction of sentences for the task of extractive summarization, and show how such descriptive sentences can be used to identify complex relations that cannot be detected by current knowledge extraction and knowledge base population techniques. Contributions. In this chapter, we thus make the following three contributions.
I We demonstrate how an implicit network representation can be used to construct an entity-centric search engine with an interactive user experience.
II We propose, expand, and evaluate sentence ranking schemes for the extractive sum- marization of descriptive sentences from implicit networks.
III We investigate the extraction of complex entity relations beyond the capabilities of traditional relation extraction approaches on the example of toponyms.
References. Parts of this chapter are based on the peer-reviewed publications:
A. Spitz, S. Almasian and M. Gertz. “EVELIN: Exploration of Event and Entity Links in Im- plicit Networks”. In: Proceedings of the 26th International Conference on World Wide Web (WWW), Companion Volume. 2017, pp. 273–277. doi:10.1145/3041021.3054721
A. Spitz, G. Feher and M. Gertz. “Extracting Descriptions of Location Relations from Implicit Textual Networks”. In: Proceedings of the 11th Workshop on Geographic Information Retrieval (GIR). 2017, 1:1–1:9. doi:10.1145/3155902.3155909
4 Applications of Implicit Networks
4.1 Overview and Motivation
As we have seen in our exploration of the Wikipedia implicit network in Chapter 3.4, implicit networks can be used to describe and extract latent entity relations from large document collections. Thus, it stands to reason that they can support a variety of tasks in information retrieval, knowledge extraction, and search. While such implicit relations offer less insight into the types of connection between entities than traditional knowledge bases, they are much easier to extract from unstructured textual sources, and can therefore be constructed from a wider variety of texts. The derived relationship strengths between entities can then be used to identify and leverage important co-mentions, based on which complex constructs of semantically related entities can be assembled with ease. In this chapter, we therefore explore applications for such implicit networks beyond the examples that we have previously considered.
For the first application scenario, we focus on the typical task of searching a document collection. Here, the entity-centric focus of implicit networks provides us with a unique an- gle and perspective that we can use to extract information through entity relations. Thus, we describe and demonstrate an entity-centric search engine that enables us to search for any type of node (meaning entities or structural elements such as sentences or documents), and rank them according to the user’s input. In particular, since we can link nodes in the network to the entries of a knowledge base, we consider the scenario in which the input also consists of typed entities in addition to plain terms, thereby providing the user with additional input opportunities for a faceted entity search, and additional feedback about his selection of query entities.
Based on these options for ranking arbitrary nodes in the neighbourhood of query enti- ties or terms in the network, we then focus not just on the relations between entities, but on describing these relations. As our second application scenario, we therefore investi- gate the ranking and extraction of descriptive sentences from the network representation, which can serve to describe or summarize relations. Using these relation descriptions, we then perform a contrastive comparison to traditional knowledge graphs and the discrete relations that they contain.
Structure. In Chapter4.2we give a broad overview of the network model’s capabilities by presenting a user interface that can be used to query an entity-annotated version of Wikipedia for different types of input and output entities. Following this general explo- ration, we focus on sentence nodes in the network in Chapter4.3, where we consider the task of extractive summarization and descriptive sentence extraction.