Design Decisions

The main design goal of VEX is enabling the user to quickly identify entity linking and clustering errors. Because a naive approach to entity visualization by drawing edges between all possible pairings of mention spans quickly leads to a cluttered graph as shown in 3.17a on the facing page, we instead visualize entities using Euclidean minimum spanning trees, inspired by Martschat and Strube’s 2014 use of spanning trees in error analysis for coreference resolution.

An Euclidean minimum spanning tree is a minimum spanning tree of a graph whose vertices represent points in a metric space and whose edge weights are the spatial distances between points. In our case, the metric space is the two- dimensional pixel space in which the HTML document is rendered by the web browser, a point is the top-left corner of a text span element, and the distance met- ric is the pixel distance between the top-left corners of text span elements. Since the minimum spanning tree spans all graph vertices while minimizing total edge length, it allows for a more concise visualization as shown in 3.17c on the next page.

Since the actual positions of mention span elements on the user’s screen de- pend on various user environment factors such as font size and browser window dimensions, the minimum spanning trees of displayed entities are computed in

19_{Also known as Hungarian algorithm.}

(A) Complete graph

(B) Sequential order graph

(C) Euclidean minimum spanning tree

FIGURE 3.17: Cluttered visualization of an entity via its complete graph, drawing all pairwise connections between mentions (a), less cluttered visualization connecting entity mentions in sequential order (b), and a more concise visualization of the same entity using an Eu- clidean minimum spanning tree, connecting all mentions while mini- mizing total edge length (c).

real time using a client-side JavaScript library21 and are automatically redrawn if the browser window is resized. Drawing of edges is performed via jsPlumb22, a highly customizable library for line drawing in HTML documents.

In order not to overemphasize mention detection errors when displaying enti- ties, VEX assumes a system mention span to be correct if it has a non-zero overlap with a gold mention span. For example, consider the first gold mention “Vulcan Inc” in Figure 3.15 on page 67, which has not been detected correctly by the system, as it detected “Vulcan Inc.” instead. While a strict evaluation requiring perfect men- tion spans will give no credit at all for this partially correct result, seeing that this mention detection error is already visually signified by the red square bracket, VEX treats the mention as detected correctly for the purpose of visualizing the entity graph, and counts it as a true positive instance if it has been linked correctly.

While VEX provides a carefully chosen default configuration, the visualization style can be easily customized via CSS, e.g., in order to achieve a finer-grained cat- egorization of error types such as off-by-one mention detection errors, or classifica- tion of linkable mentions as NIL mentions and vice-versa.

3.3.8

Summary

In this section, we have introduced the Visual Entity Explorer (VEX), a tool for vi- sual error analysis of entity linking systems. We have shown how VEX can be used for quickly identifying the components of an entity linking system that appear to have a high potential for improvement, as well as for finding errors in the gold standard annotations. Since visual error analysis of our own entity linking system revealed several issues and possible improvements, we believe performing such an analysis will prove useful for other developers of entity linking systems, as well.

21_{https://github.com/abetusk/euclideanmst.js. This library uses Kruskal’s algorithm}

(Kruskal, 1956) to find minimum spanning trees.

Chapter 4

Selectional Preferences for

Coreference Resolution

In this chapter, we study the application of another aspect of coherence: the seman- tic agreement between a predicate and its arguments. This agreement is an essential property of language and is one of the factors that distinguish a coherent text from an incoherent one. For example, in the phrase

(47) the ship sinks

the predicate sinks and the subject argument the ship agree semantically, since, ac- cording to our knowledge of the world, it is plausible, albeit unfortunate, that a ship sinks. In contrast, the phrase

(48) ? the ship writes a dissertation

lacks semantic agreement, since, to the best of our knowledge, ships cannot write, and dissertations cannot be written by inanimate objects.

4.1

Preference and Affordance

The notion of semantic agreement between a predicate and its arguments has mainly been approached from the perspective of the predicate. In this view, a predicate se- lects specific arguments: The predicate sink selects subject arguments that are not buoyant – either by default such as stones, or by accident such as ships. Similarly, the predicate write selects subjects that can write, such as PhD students. The precise mechanism by which this selection happens has been subject of extensive scholarly debate.1 In this work we adopt the view that “a predicate preferentially associates with certain kinds of arguments” (Resnik, 1993, p. 53). Resnik calls the prefer- ential association between a predicate and its arguments the predicate’s selectional

preference. A predicate’s selectional preference expresses what kind of arguments it typically chooses. This choice is “less a yes-or-no decision and more a function of how easily the predication can be accommodated given information about word meanings and context” (ibid., p. 59). For example, the predicate sink has a high preference for the first of the following subject arguments, but the preference for the subsequent subjects becomes smaller:

(49) the ship sinks (50) the stone sinks (51) the person sinks (52) the airplane sinks (53) the house sinks (54) the island sinks (55) ? the balloon sinks (56) ? the maple leaf sinks (57) ? the gas sinks

(58) ?? the election sinks (59) ?? the choice sinks

(60) ?? the quadratic equation sinks

This gradual change in selectional preference reflects the ease or difficulty of imag- ining contexts in which a specific pairing of predicate and argument makes sense. There are many contexts imaginable in which a ship, a stone, or a person sink. Less typical, but still plausible are situations involving a sinking airplane, house or is- land. Not typical, but also not impossible are scenarios that lead to the sinking of a balloon, a maple leaf, or gas. Least preferred are the bottom three subjects, since it is difficult to imagine a context that meaningfully combines an event like an election or abstract concepts like choice and quadratic equation with the literal sense of sink that is active here.

Complementary to the analysis of a predicate’s preferred arguments is the anal- ysis of an argument’s preferred predicates. Instead of asking what kind of object typically sinks, we now ask what a ship typically does, or what is typically done to a ship. Typical things a ship does are:

(61) the ship set sail (62) the ship arrived (63) the ship sank

and things that are typically done with or to a ship are: (64) the ship was built

(65) the ship was christened (66) passengers boarded the ship (67) the captain is steering the ship (68) high waves rocked the ship

Besides (48), examples of things that are not typically done by or with/to ships are: (69) ? the ship thawed

(70) ? the ship thought

(71) ? the ship was taught algebra (72) ? the passengers bored the ship (73) ? the captain is wearing the ship

Our model of selectional preferences, which we introduce in section 4.3 below, cap- tures both a predicate’s preferred arguments and an argument’s preferred predi- cates. While the former has an established name, namely selectional preferences, the latter does not. Erk and Padó (2008) call what is typically done by/to/with an ob- ject the object’s inverse selectional preferences. Borrowing a term originally coined in perceptual psychology, Attardo (2005) calls it affordance. Taking an example by Attardo, the difference in meaning between

(74) He ran to the edge of the cliff and jumped. and

(75) He ran to the trampoline and jumped.

lies in the difference between what a cliff allows doing and what a trampoline al- lows doing. The edge of a cliff affords jumping off (once, into the sea), while a trampoline affords jumping on it, usually repeatedly.