Alignment effects: Repeatedly it could be observed that users tend to align their wording to the robot’s utterances. For instance, users asked “will you have a look at it?”, as a result of the robot announcing “I will have a look at it” before answering an object recognition request. Similarly, a user replied “Sorry, I can not recognize which object you mean” to the robot’s label request, which is exactly what the robot says in the reverse case. Unfortunately, the system was not prepared to all of these utterances that users adopted from the robot.
Problems with initiative shift: In condition C2 (MixedIni), where the robot takes initiative at the begin of the interaction, but does not explicitly yield initiative later, problems with shifting initiative could be observed. Several participants did not demonstrate any objects on their own initiative, but waited for the robot to ask for the next object, even though this results in fairly long gaps between the teaching episodes sometimes. For this condition, we had the guideline that the time intervals between the robot’s label queries (which were triggered by the WOz ROI selection) should vary, and that some of them should be sufficiently long to allow the user to take over initiative. One participant tried to speed up interaction by explicitly asking the robot to proceed to the next object. Unfortunately, the robot interprets the request as an announcement that the user will now demonstrate the next object, with the consequence that the interaction gets stuck:
F I have learnt the apple. U Next.
F Okay.
[gap of 1:20 min] F What is that?
7.5 Summary
This chapter has described the Curious Flobi system, which can be seen as a proof- of-concept for the proposed approach to dialog modeling, and has in turn advanced the approach by providing new use cases on object teaching. Besides the description of the scenario and the system architecture, a detailed account of the design process has been given: Based on experiences from the CeBit scenario, it has been decided to extend the capabilities of the Curious Robot for more user initiative. The analysis of the nonunderstandings in the CeBit scenario led to the decision to exchange the speech understanding component with more robust keyword matching. A WOz study on object teaching served as a guideline to design the interaction strategy for the Curious Flobi scenario, including the configuration of the speech recognition.
A large-scale user study has been conducted, in which a wide range of both subjective and objective measures have been calculated from the interactions. The evaluation addresses (i) the relationship between the objective and subjective measures, (ii) the influence of the
The relationship between objective and subjective measures has been investigated by means of a PARADISE-style regression analysis. The resulting performance functions have revealed some unexpected relations, e.g. the influence of turn-taking problems (as indicated by short gaps between utterances) on how cooperative and intelligent users perceived the robot. To a certain extent, the results are generalizable, e.g. the influence of badly designed system feedback on general understandability of the system.
The investigation of the robot’s task initiative has shown differences between the conditions, but some of them were not immediately understandable. Also, some presumed differences could not be shown, e.g. differences in the number of learned objects. Thus, this issue deserves to be addressed in a future study with more participants per condition.
The qualitative analysis explains and supports result from the PARADISE and the between- subjects evaluation. Moreover, it points out several deficiencies of the system that future iterations need to address, e.g. the system must cope with (or prevent) the user replacing objects during the learning.
8 Further Scenarios
This chapter describes briefly a number of additional scenarios that have been implemented with the proposed PaMini framework, but not by the author herself. A variety of platforms and scenario types are represented, ranging from robots to virtual agents, from information- oriented to rather action-oriented interactions, relying on diverse forms of system back-ends. This demonstrates that (i) the suggested approach is comprehensible to developers, and (ii) that it is not restricted to a certain interaction type, but that it is versatile enough to
be applied to very different scenarios.
8.1 Receptionist Vince
The Receptionist Vince scenario is a joint project which several working groups at CITEC Bielefeld are contributing to. Main actor in the scenario is the virtual agent Vince, whose task is to provide visitor information about the CITEC. Vince can generate synchronized multimodal output, as specified based on a multi-modal mark-up language, for which a new output source had to be added to PaMini. In the receptionist scenario, Vince communicates not only with the visitor, but also with the mobile robot BIRON, whose task is to show the visitors around. Figure 8.1 shows the interaction setup. An example dialog is given in figure 8.3.
From a dialog modeling perspective, the implementation still has a couple of shortcomings. Technically, the communication between Vince and BIRON has not (yet) been modeled as an actual multi-party dialog, but BIRON’s part of the interaction is generated by a few of pre-scripted outputs. Also, as can be seen from figure 8.3, Vince instructing BIRON has not been modeled using an Action Request, but rather by combining several Notifications. This implies that failure cases can not be accounted for in a systematic manner based on the Task State Protocol, but are handled based on time-out conditions.
As listed in table 5.1, this scenario makes mainly use of Information Requests (modeling the visitor’s questions), Statements (for modeling small-talk and exchange of pleasantries) and Notifications (for Vince’s communication with BIRON).
Figure 8.1: The Receptionist Vince. Figure 8.2: ToBi at the 2011
RoboCup@Home, performing the FollowMe task.
U Hello! H Interaction Opening
V Hi! I am Vince. How may I help you?
U Who are you? H Information Request 1
V My name is Vince. I’m a virtual agent and I contribute to the
V Dialog Demonstrator project. H Information Request 1
U And what can you do? H Information Request 2
V I can provide information about employees and working groups
V at the CITEC. H Information Request 2
U Oh, good. I am looking for Prof. Ritter. H Information Request 3 V You can find him in Q1-123. H Information Request 3 V Would you like BIRON, the mobile robot, to show you the way? R Suggestion
U Yes, please. R Suggestion
V Okay, please wait a moment. R Suggestion B [is approaching]
B Hello Vince, my friend, what may I do for you?
V This is BIRON. He will take you to the lift. R Notification BIRON, please take our guest to the lift. R Notification B Alright, please follow me.
U Bye bye, Vince! H Interaction Closing
V Good bye. H Interaction Closing
Figure 8.3: An example dialog with the Receptionist Vince.