Figure1.3 provides a schematic overview of the structure of my thesis. Chapters 2 and 3 are concerned with the identification and classification of explicit lexi- cal and punctuational markers of syntactic complexity in English sentences: the signs of syntactic complexity. These chapters present a human annotated corpus containing text of three registers and an automatic classifier derived using a ma- chine learning method applied to this corpus. This sign tagger is exploited by the additional syntactic analysis method and the sentence simplification methods presented in Chapters 4 and 5, respectively.
Chapter 4 presents a new machine learning method to identify the spans of compound clauses and complex constituents in English sentences. It includes a de- scription of the annotated resources developed to support development and train- ing of the method. The chapter includes an evaluation of its accuracy when classi- fying tokens in input sentences as occurring within compound clauses or occurring within several different types of complex constituents, including complexRF NPs.
Chapter 5 presents my approach to automatic sentence simplification. This approach is based on several sentence transformation schemes to simplify sen- tences containing compound clauses and complexRF NPs. The schemes are im-
plemented as rules comprising rule activation patterns associated with transfor- mation operations that exploit those patterns. Two systems are presented, one which implements handcrafted rule activation patterns (Section 5.2.2) and one which exploits the machine learning approach presented in Chapter 4 to imple-
ment machine-learned rule activation patterns (Section 5.2.3).
Chapter 6 presents intrinsic evaluation of both sentence simplification sys- tems. This evaluation is made using overlap metrics which compare automatically simplified sentences with human-simplified sentences (Section 6.1) and using au- tomated assessments of the readability of system output (Section6.2). In Section 6.3, the system exploiting handcrafted rule activation patterns is also evaluated via surveys of the opinions of human readers with respect to the grammaticality, comprehensibility, and meaning of its output. Chapter7of the thesis presents ex- trinsic evaluation of that system. The extrinsic evaluation is made via automatic NLP applications for multidocument summarisation, semantic role labelling, and information extraction.
Chapters2–5and Chapter7of the thesis each contain surveys of related work. These chapters and Chapter6 also include sections detailing contributions made to the previously listed research questions. Finally, Chapter 8 synthesises this information and discusses the extent to which the main goal of the thesis was achieved. It includes indications of directions for future work relevant to each of the preceding chapters.
Sentence Analysis of English
The research described in this chapter addresses research question RQ-1 of the thesis, which is concerned with the existence of explicit textual signs indicating the occurrence of compound clauses and complexRF NPs in English sentences.
My response to this research question is one part of the more general task of providing a shallow syntactic analysis of English sentences. In this context, the signs are explicit markers of a potentially wide range of compound constituents and subordinate clauses modifying complex constituents. In this chapter, I spec- ify the set of signs and their syntactic linking and bounding functions. In my research, I supervised development of a corpus annotated with information about these signs of syntactic complexity. I describe and present an analysis of this corpus.
Compound clauses are one type of compound constituent occurring in natural languages, including English. Compound constituents are those that contain two or more syntactic constituents linked by coordination. Quirk et al.(1985) define coordination as a paratactic relationship that holds between constituents at the same level of syntactic structure. The linking function occurs between conjoins1
1In this thesis, I employ the terminology used by Quirk et al. (1985). In related work, the
term conjunct has been used rather than conjoin, but Quirk et al. use the former term to denote “linking adverbials”.
(7) She knew the risks [and] still insisted the operation should go ahead, Dr Addicott said.
Compound constituents are structures containing conjoins and their linking co- ordinators.
Subordination is defined as a hypotactic relationship holding between con- stituents at different levels of syntactic structure, referred to as superordinate and subordinate constituents. Sentence (8) contains a non-finite subordinate prepo- sitional clause linked to a superordinate noun phrase (NP) in the main clause of the sentence.
(8) McKay[,] of Wark, Northumberland, denies five charges of contaminating food.
Grammatically, subordinate constituents are clausal.3 Relative clauses are sub-
ordinate constituents that modify their superordinate constituents and depend for their meaning on those constituents (i.e. they are not independent clauses). ComplexRF NPs are those which are modified by non-restrictive relative clauses.
2In example sentences containing signs of syntactic complexity, signs in focus are indicated
using square brackets while coordinated conjoins or subordinate constituents are underlined. Where appropriate, the location of elided elements is indicated using . Occurrences of may be co-indexed with their antecedents.
3In many cases the extant parts of the subordinate clause are subclausal, as in the case of
The aim of this chapter is to determine whether or not there are reliable and explicit textual signs which indicate the occurrence of compound constituents (including compound clauses) and relative clauses (including those which modify complexRF NPs) in English sentences. If so, the aim of the chapter is also to
specify the forms and functions of these signs (RQ-1).
In this thesis, I use the term signs of syntactic complexity to denote words and punctuation marks that bound subordinate constituents and that coordinate the conjoins of compound constituents in sentences. The presence of these syntactic structures in a sentence is indicative of two commonly cited determinants of text processing difficulty: syntactic complexity and propositional density.
The syntactic functions of conjunctions, complementisers, relative adverbs, relative pronouns, and punctuation marks have been described in numerous lin- guistic studies of English (Chomsky, 1977; Quirk et al., 1985; Nunberg et al.,
2002). For this reason, I posit a subset of words and punctuation marks of these categories as potential signs of syntactic complexity.
In this chapter, I present an annotation scheme to encode the linking func- tions of coordinators (conjunctions, punctuation marks, and pairs consisting of a punctuation mark followed by a conjunction)4 and the bounding functions of
subordinate clause boundaries (complementisers, wh-words, punctuation marks, and pairs consisting of a punctuation mark followed by a lexical sign).5 The
4Restricted to a relatively unambiguous subset of coordinators to facilitate both the manual
annotation task and the automatic tagging process (Chapter5)
5With regard to punctuation, my research concerns the annotation of what Nunberg et al.
(2002) refer to as secondary boundary marks. Due to practical resource limits, I focus on this subset, considering the annotation of other types of punctuation such as primary terminals, parentheses, dashes, punctuation involved in quotation, citation, and naming, capitalisation,
annotation scheme also encodes information about false signs that do not have coordinating or bounding functions (e.g. use of the word that as an anaphor or specifier). The chapter includes an analysis of the annotated corpus. It is expected that the encoding of this information can be gainfully exploited in the development of the sentence analysis and sentence simplification methods pro- posed in this thesis (Chapters 3–5).
2.1 Related Work
The main aim of the research described in this chapter is to produce annotated re- sources supporting development of a tool to automatically classify signs of syntac- tic complexity with specific information about their syntactic linking and bound- ing functions (Chapter 3). This sign tagger is a key component of the pipeline for automatic sentence simplification proposed in this thesis. Analysis of these annotated resources provides insights into RQ-1, concerning the form and char- acteristics of signs which may indicate the occurrence of compound clauses and complexRF NPs in English sentences.
In view of these aims, the most relevant topics in previous work include the development of syntactically annotated resources, proposals to improve the qual- ity of these resources, and the development of syntactic parsers that can automate the process.
There are currently a wide range of Treebanks available, providing access to syntactically annotated resources in many languages (Brants et al., 2002; Simov
et al., 2002; Haji˘c and Zemánek, 2004). In English, one of the most widely-used is the Penn Treebank (Marcus et al., 1993) which has been exploited for the de- velopment of supervised syntactic parsers (Charniak and Johnson, 2005; Collins and Koo, 2005). Despite several criticisms of this resource, the Penn Treebank continues to be widely exploited in the field of supervised parsing because syn- tactically annotated data is scarce and expensive to produce. In addition, the Penn Treebank has been enhanced with other types of annotation, as described below.
Maier et al. (2012) observed that one shortcoming of the Penn Treebank is that punctuation symbols (commas and semicolons) are not tagged with infor- mation about their syntactic functions. If present, information of this type would facilitate the training of syntactic parsers that were better able to analyse sen- tences containing compound structures in which conjoins are linked in asyndetic coordination (Quirk et al.,1985). To address this shortcoming,Maier et al.(2012) propose the addition of a second layer of annotation to disambiguate the role of punctuation in the Penn Treebank. They present a detailed scheme to ensure the consistent and reliable manual annotation of commas and semicolons with information to indicate their coordinating function.
An advantage of the approach described by Maier et al. (2012) is that the addition of an annotation layer is more cost-effective than the development of new annotated resources from scratch. By leveraging the original layer of annotation, minimal human effort and expertise is required. However, there are two main criticisms of this methodology. First, the scheme encodes only coarse-grained
information, with no discrimination between subclasses of coordinating and non- coordinating functions. Second, although production of the second annotation layer is inexpensive, application of the proposed scheme is costly as it depends on the availability of the original syntactic annotation layer. This limits the portability of the approach.
The annotation scheme developed in my research tags coordinators with more detailed information about their conjoins. It also encodes syntactic information about the extant constituents bounded by subordinate clause boundaries.6 Re-
sources produced using this scheme and the scheme proposed by Maier et al.
(2012) can thus be regarded as complementary.
As noted earlier in this section, the Penn Treebank has been exploited in the development of supervised approaches to syntactic parsing. Given that this type of processing, if done with sufficient accuracy, could serve as the basis of any syntactic processing or sentence simplification system, there has been considerable research in improving the performance of syntactic parsing. Much of this involves techniques specifically designed to improve the parsing of coordinated structures (Charniak and Johnson, 2005; Ratnaparkhi et al., 1994; Rus et al., 2002; Kim and Lee, 2003; Nakov and Hearst, 2005; Hogan, 2007; Kawahara and Kurohashi,
2008; Kübler et al., 2009). However, supervised methods trained on the Penn Treebank are likely to generate syntactic analyses subject to the shortcomings of that dataset. A better prospect is to exploit such traditional resources in
6Here, I use the term extant constituent to refer to the constituent that remains in the
text when the rest of the clause has been elided. Examples of extant consituents would be noun phrases in non-finite nominal clauses and prepositional phrases in non-finite prepositional clauses.
combination with others, such as the annotation layer proposed by Maier et al.
(2012).
The new scheme presented in this chapter is derived from the one proposed by Evans(2011), which aimed to improve performance in information extraction by simplifying sentences in input documents. In that scheme, members of a small set of textual markers of syntactic complexity were considered to belong to one of two broad classes: coordinators and subordinators. These groups were annotated with information on the syntactic projection level and grammatical category of conjoins linked and subordinate constituents bounded by those signs. The annotation of these markers, called potential coordinators, was exploited to develop an automatic classifier used in combination with a part-of-speech tagger and a set of rules to convert complex sentences into sequences of simpler sentences. Extrinsic evaluation showed that the simplification process evoked improvements in information extraction from clinical documents.
One weakness of the approach presented byEvans(2011) is that the classifica- tion scheme was derived by empirical analysis of rather homogeneous documents from a specialised source. Their consistency, together with the restricted range of linguistic phenomena manifested, imposes limits on the potential utility of the resources annotated. The scheme is incapable of encoding the full range of syntactic complexity encountered in documents of other registers.