In S ection 3. 6 we took a detailed look at verb-final constructions with the sister- head parser. Here, we will replicate this analysis for the smoothed G F parser and we will also present a similar analysis for another construct found in G erman and not in English: topicalization ( cf. S ection 1 . 1 . 1 ) . In main clause constructions, the verb is in the second position of a S rule, and the sub ject is usually in the first position ( see Example 1 . 2 ) .
Example 4. 1 .
Ich esse Schinken in dem Hau s I eat ham in the house
NP V NP P P
po sitio n i ii iii
However, a modifier can be to picalized and occupy position i. In these cases, the sub ject moves to position iii, using the verb an an axis. O ther complements and modifiers come after the sub ject, as in Example 1 . 3.
Example 4. 2 .
in dem Hau s esse ich Schinken in the house eat I ham
PP V NP NP
po sitio n i ii iii
For formal grammar writers, topicalization ( and flexible word order in general) has been the source of much research, and a number of techniques have been devised to handle this phenomenon. These techniques include movement, soft constraints in LP / ID rules and topological fields.
However, the situation for treebank grammars is slightly different. In cases when the sub ject moves from position i to position iii are both to be found in the treebank grammar. P resumably, this ought not to make parsing much harder, as the grammar can learn both orders from the treebank.
S BAR? all vf novf to pic no to pic Avg. # of nodes 7. 5 1 1 . 2 6. 4 8 . 9 6. 5 S tandard F-S core × 72 . 7 68 . 1 75 . 0 71 . 8 73. 6 72 . 6 67. 8 75 . 0 72 . 2 72 . 9 Weighted F-S core × 72 . 7 68 . 3 73. 7 72 . 0 72 . 8 72 . 6 68 . 7 73. 7 72 . 4 72 . 1
Table 4. 9 . Performance of the unsmoothed model on various syntactic constructions
4. 4. 1 M etho d
We proceed in a manner similar to S ection 3. 6, reporting both standard F-scores, and a weighted F-score measure that attempts to remove the influence of longer and more complicated sentences. We test results using four parsers. Two of these four parsers are the best performing unsmoothed model and the best performing smoothed model. O ne of these parsers includes the S BAR ( verb-final clause) marking modification, the other does not. To round out the comparison, we ensure that models both with and without the S BAR marking modification are included in both the unsmoothed and smoothed cases.
4. 4. 2 Results
We show the results in Table 4. 9 for the parser without smoothing, and Table 4. 1 0 for the parser with smoothing. The ‘ S BAR? ’ column indicates if the model in question contains the S BAR re-annotation. Most of the other entries should be self-explanatory. O nce again, though, there are too many results to discuss all of them in detail, so we will simply point out some of the key findings. Just as in S ection 3. 6, we find sentence with ‘ special’ constructions have lower F-scores. Adding the S B AR annotation made little difference in the performance non-verb- final ( novf) sentences in both the smoothed and unsmoothed grammars. In the unsmoothed grammar, this annotation led to mixed results in all conditions except the novf case, where performance was unchanged. In the smoothed grammar, on the other hand, it improved the performance in the vf condition.
O ddly enough, the unsmoothed parser did about 1 point better when the sub- ject was not in position i ( to pic) than when it was ( no top ic) , despite the fact that sentences are longer in the top ic case. The smoothed grammar was more accurate in both the s ub j and no s ub j conditions, but the relative performance swapped, with the no top ic case giving the higher result.
S BAR? all vf novf to pic no to pic Avg. # of nodes 7. 5 1 1 . 2 6. 4 8 . 9 6. 5 S tandard F-S core × 76. 1 72 . 8 77. 9 75 . 1 77. 2 76. 3 73. 2 77. 8 75 . 6 76. 9 Weighted F-S core × 76. 1 74. 6 76. 8 75 . 9 76. 4 76. 3 75 . 2 76. 8 76. 0 76. 2
Table 4. 1 0. Performance of the smoothed model on various syntactic constructions
4. 4. 3 D iscussion
It appears that verb-final clauses are almost as difficult for the unlexicalized G F parser as for the sister-head parser. In S ection 3. 6, we hypothesized that parsers ought to have some sp ecial mechanism for dealing with these constructions. The S BAR annotations did improve the smoothed grammar’ s performance on verb- final constructions, but apparently not enough to close the gap with non-verb- final constructions. However, adding this annotation also made the weighted per- formance of both parsers in the top ic condition comparable to their weighted performance in the notop ic condition ( although partly by decreasing the F-score in top ic condition) . It is interesting that, for both parsers, the difference between to p ic and notop ic is much smaller than the difference between vf and novf. Part of this might be explained by the smaller difference in average sentence lengths, but the change is impervious to weighting, which ought to account for part of the sentence length effect. We conjecture this is because a fronted P P has less attachment ambiguity than one which occurs in the verb’ s argument field. Another possibility is that ( accorinding to some dependency grammar theories) , verb-final clauses involve crossing dependencies, whereas topicalization does not.
Based on our initial exp eriments in S ection 4. 1 , it seemed as if horizontal M arkovization was a general technique, and not specific to the Penn Treebank. But our experiments here suggest that M arkov grammars have some difficulty in modeling flexible word order constructions. G iven that the parsers have a harder time with verb-final constructions, a possible solution is to give a better treatment of long-distance dependencies. This wouldn’ t help with topicalization, however. There, the problem may be that the M arkov histories are not long enough. If we are considering which node to add to the partial rule S → NP-OA V NP-SB , the probability of addding an accusative ob ject would be too high: the existing NP-OA is lost in the M arkov history, and we could not tell if it was an ob ject or a modifier which had been fronted.
4. 5 C onclusion
O ne of the goals of this chapter has been vindicated: a finely tuned grammatical function parser performs better than the fully lexicalized parsers of C hapter 3. Unfortunately, we did not succeed in matching the performance of the G F parser from S ection 3. 2 . However, the coverage of the G F parsers here was much higher.
O verall, we found that Markovization and smoothing help overcome coverage problems, and increase performance. Lexical sensitivity also helps with low-cov- erage problems at the cost of reducing performance, although some of this lss can be overcome by using a smart suffix analyzer when sparse data is not a problem.
While ( horizontal) M arkovization worked well in NEG RA, other techniques which have been shown to be useful for English, including higher-order vertical M arkovization, appear to be specific to the annotation style of the WS J, and do not generalize to NEG RA.
The two main results are ( i) that M arkovization may not be helpful in han- dling phenomena of flexible word order, such as the sub ject movement we saw in S ection 4. 4 ( although difficulties with the evaluation precludes us from making a claim with any degree of certainty) ; and ( ii) that including more knowledge about things such as noun declension is one of the factors that allow the unlexicalized parser to be comp etitive with the lexicalized parser.
This second point brings us back to our main argument: that linguistic fea- tures play an important role in languages with a rich morphology, and we cannot depend on lexicalization alone. For good measure, it is interesting to note that many of the features we annotated dealt with case, and would not even be rele- vant for an English parser.