summarization evaluation
Automated Pyramid Summarization Evaluation
... human summarization skills, and their de- velopment throughout secondary school and be- ...2010 summarization challenge used automatic caseframe analysis, and found essen- tially these same properties in ...
15
Summarization Evaluation meets Short Answer Grading
... ROUGE-S* using the standard skip of 4 to- kens between the elements of a bigram works best for ASAP and Beetle, while LCS outperforms it slightly for SEB. In addition to the standard skip of 4 tokens, we also ...
7
Better Summarization Evaluation with Word Embeddings for ROUGE
... Another immediate goal is to study the use of better compositional embedding models. The gen- eralization of unigram word embeddings into bi- grams (or phrases), is still an open problem (Yin and Sch¨utze, 2014; Yu et ...
6
Combining Coherence Models and Machine Translation Evaluation Metrics for Summarization Evaluation
... and summarization evaluation: while MT systems always generate new sentences, most summarization systems focus on locating existing salient ...
9
Revisiting Summarization Evaluation for Scientific Articles
... text summarization approaches have been mostly based on metrics that measure similarities of system generated summaries with a set of human written gold-standard ...in summarization evaluation has ...
8
Multilingual Summarization Evaluation without Human Models
... Text summarization evaluation has always been a complex and controversial issue in computational ...the summarization eval- uation field. Various evaluation frameworks have been established ...
9
Results of the 2016 ENtity Summarization Evaluation Campaign (ENSEC 2016)
... entity summarization evaluation campaign, is the first step taken towards realizing that goal, and we present the results of the systems participating in the ...
6
Studying Summarization Evaluation Metrics in the Appropriate Scoring Range
... an evaluation metric becomes standard, it is optimized, either directly by supervised meth- ods or indirectly via repeated comparisons of un- supervised ...for summarization (Peyrard and Eckle-Kohler, ...
8
The TIPSTER SUMMAC Text Summarization Evaluation
... However, for the generic summaries in the cat- egorization task which was hard even for hu- mans with full-text, in the absence of a topic, the summarization methods in use by these syst[r] ...
9
The Feasibility of Embedding Based Automatic Evaluation for Single Document Summarization
... automatic evaluation for summarization relies on token over- lap between reference and system ...on summarization evaluation (Zhou et ...
6
Question Answering as an Automatic Evaluation Metric for News Article Summarization
... extrinsic summarization evaluation metric; (2) We show APES strength through an analysis of its correlation with Pyra- mid and Responsiveness manual metrics; (3) we present a new abstractive model which ...
11
Challenging Issues of Automatic Summarization: Relevance Detection and Quality-based Evaluation
... The evaluation of this method was performed with the ROUGE tool, which made possible the comparison between automatic summaries and reference ...the summarization evaluation have, a novel manner of ...
8
Multi document multilingual summarization and evaluation tracks in ACL 2013 MultiLing Workshop
... track, summarization evaluation, is related the corresponding, prominent research problem of how to automatically evaluate a sum- ...automatic evaluation of summaries, without human “ model” ...
9
Some Examinations of Intrinsic Methods for Summary Evaluation Based on the Text Summarization Challenge (TSC)
... automatic summarization. Evaluation methods for text summariza- tion can be divided into two categories: intrinsic and extrinsic(Sparck-Jones and Galliers, ...a summarization sys- tem in a given ...
8
HighRES: Highlight based Reference less Evaluation of Summarization
... Our evaluation framework comprises three main components: document highlight annotation, highlight-based content evaluation, and clarity and fluency ...
12
The Heterogeneity Principle in Evaluation Measures for Automatic Summarization
... The HBR (Heterogeneity Based Ranking) method proposed in this paper does not represent the “best automatic evaluation measure”. Rather than this, it promotes the development of new measures. What HBR does is ...
8
A Machine Learning Approach to Sentence Ordering for Multidocument Summarization and Its Evaluation
... We also performed a human evaluation of our orderings. We asked two human judges to grade the summaries into four categories. The four grades are; perfect: no further adjustments are needed, acceptable: makes ...
12
Hierarchical Summarization: Scaling Up Multi Document Summarization
... multi-document summarization (MDS) can handle 10-15 documents and create a short flat summary, but are insufficient for large-scale ...large-scale summarization, we need summa- rizers that organize the ...
11
Detection of Topic and its Extrinsic Evaluation Through Multi Document Summarization
... LDA and MACD. We estimated the number of k ′ and d ′ in LDA, i.e., we searched k ′ and d ′ in steps of 100 from 200 to 900. Figure 3 illustrates en- tropy value against the number of topics k ′ and documents d ′ using 30 ...
6
What's the issue here?: Task-based evaluation of reader comment summarization systems
... on completion of the task. Our approach has many similarities to this. In particular we too ask subjects to answer questions about source doc- uments under different summarisation conditions. We also follow up the ...
8