spoken dialogue system evaluation
Design of the VICO Spoken Dialogue System: Evaluation of User Expectations by Wizard-of-Oz Experiments
... Steadily increasing dissemination of computer applications has resulted in an ever-growing functional complexity of electronic services and devices. Hence the utilization of natural language is highly desirable to ...
6
Mining Spoken Dialogue Corpora for System Evaluation and Modelin
... evaluating spoken language systems in the context of human-machine dialogs. Spoken di- alog corpora allow for a multidimensional anal- ysis of speech recognition and language under- standing models of ...
8
On the Use of Prosody for On-line Evaluation of Spoken Dialogue Systems
... technology, spoken dialogue systems are especially prone to error, for instance because of user utterances that are misrecognized (Oviatt et ...the system has the option of changing its strategy (for ...
8
A Three Tiered Evaluation Approach for Interactive Spoken Dialogue Systems
... the system fails to recognize the user signing onto the radio net- work, it will ignore all subsequent utterances until the user successfully logs ...periodic evaluation of component performance focuses ...
5
Quantitative and Qualitative Evaluation of Darpa Communicator Spoken Dialogue Systems
... As mentioned above, our goal was to de- velop metrics that captured differences in dia- logue strategies. The positive coefficient of the Econ metric appears to indicate that an explicit confirmation strategy overall ...
8
Opinion Mining Using The Prosodic Information In The Spoken Utterance’s For Efficient Dialogue Management In A Spoken Dialogue System
... the dialogue manager in selecting the dialogue strategy for effective interaction and influencing the final ...the system responses in a student evaluation information system and ...
6
A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System
... optimal dialogue strategy can be obtained with a small number of training samples and has the potential for significant improvements in robustness compared to hand- crafted and grid-based ...approaches. ...
8
Semantic Information and Derivation Rules for Robust Dialogue Act Detection in a Spoken Dialogue System
... on Hidden Markov Model (HMM) with 32 Gaus- sian mixture components per state. For the lan- guage model, SRILM toolkit (Stolcke, 2002) was employed to estimate a bi-gram model with the Q- data. The average word accuracy ...
6
Examining the Impacts of Dialogue Content and System Automation on Affect Models in a Spoken Tutorial Dialogue System
... In this post-hoc analysis, we will analyze the im- pact of content differences by comparing the perfor- mance of models built with WOZ-07 and WOZ-08, and automation differences by comparing models built with WOZ-08 and ...
7
Intrinsic and Extrinsic Evaluation of an Automatic User Disengagement Detector for an Uncertainty Adaptive Spoken Dialogue System
... on system domain and applica- ...influences dialogue patterns during in- teractions with an embodied agent that gives ad- vice about healthy ...coded dialogue acts based on the SWBDL-DAMSL scheme ...
12
Effects of Game on User Engagement with Spoken Dialogue System
... For long-term effects, we averaged the number of utterances in a week over new users. Then we plotted it against elapsed weeks as shown in Figure 2, where Played and Non-played rep- resent users who had played and had ...
5
Evaluating a Trainable Sentence Planner for a Spoken Dialogue System
... on evaluation of natural lan- guage generation has utilized three different ap- proaches to evaluation (Mellish and Dale, ...subjective evaluation methodology such as we use here, where human ...
8
An Evaluation Framework for Natural Language Understanding in Spoken Dialogue Systems
... casts dialogue act identification as a supervised learning ...resulting system is typically high (Gupta, ...a dialogue act label may be book-request, while an example of one too fine-grained may be ...
6
Galaxy-II as an Architecture for Spoken Dialogue Evaluation
... We have developed various mechanisms for dealing with these problems. One such problem is the changing nature of the information we provide to users. Weather in- formation for our Jupiter domain (Zue et al., 2000), ...
6
Expert Evaluation of a Spoken Dialogue System in a Clinical Operating Room
... the dialogue, the procedural part follows an exact surgery schedule which has been modelled in the Spoken Dialogue ...The system listens to each of the surgeon’s instructions and increments ...
6
Spoken Dialogue System based on Information Extraction using Similarity of Predicate Argument Structures
... experimental evaluation to com- pare the effectiveness of the two significance mea- sures (TF-IDF and Naive Bayes (NB)) in the Japanese professional baseball ...
8
Developing and Testing General Models of Spoken Dialogue System Peformance
... of system usability, however there are several ways in which this work should be ...COMMUNICATOR evaluation data to examine these differences (Walker et ...
8
DATE: A Dialogue Act Tagging Scheme for Evaluation of Spoken Dialogue Systems
... as dialogue systems become more success- ful, for example to address the dimension of ”face”, ...a system shows to the user ...a system, it functions to identify the source of the ...
8
A Corpus for a Gesture-Controlled Mobile Spoken Dialogue System
... As well as trying to determine how well gesture recognition works or if users prefer our set of gestures to theirs, another follow-up question was whether users would be willing to execute these gestures in public. ...
8
Evaluation for Darpa Communicator Spoken Dialogue Systems
... the system in order to focus the user on the task of rating the system, as in (Shriberg et ...the system; the user has to state the degree to which they agree with these statements in terms of a 5 ...
7