Key Components

This section discusses the different components of interest referred in Figure 63 including the NPC module, the TextChat module, the Text To Speech component, and the learning object. The details of interaction, in relation to specific learning scenarios are discussed afterwards in Section ‎9.1.3.

Experiments

The NPC Module

The Non-Player Character (NPC) is an available module in Open Wonderland that is a 3D embodiment representation similar to an avatar but with no user control (Open Wonderland, 2012). Therefore, it is appealing for utilization since it has several humanoid functions: appearance, movement, and providing gestures. However, it lacks decision functions. Therefore, the target is to develop functions on top of NPC module in Open Wonderland adding cognitive and decision abilities. This can be achieved by adopting it as an intelligent agent while the NPC takes the appearance and animation in Open Wonderland. Furthermore, the NPC lack of conversation abilities should be considered.

A useful feature available with the NPC module is the proximity event listener that senses objects as becomes in proximity. When the learner gets into proximity of the IPA, the IPA knows and identifies the avatar and tries to initiate conversation and shows attention.

Providing gestures to the learner is important to support the emotional state and has influence on learning ‎ (Lester et al., 1997). Therefore, adding this feature and for evaluation become important. Upon answering a question, the IPA provides a supporting gesture instead or with the text-based answer. This is achieved by forwarding a request to the NPC based on the chatbot answer to issue the gesture. Since the NPC module is based on the avatar implementation, several avatar gestures are also available to the NPC as well according the Open Wonderland implementation (Open Wonderland, 2012). For example a “Yes”

gesture is used to show approval and encouragement and the “No” gesture is used to show an occurring mistake and disapproval.

An interesting aspect is when the IPA senses avatar emotional state in Open Wonderland. An approach is to detect the avatar module gesture generation or in the work of ‎ (Amarakeerthi, Ranaweera, Cohen, & Nagel, 2009) where a text chat module carries Open Wonderland compatible gestures through writing emoticons in the text chat module.

Text-Based Chat Module

To allow textual learner-IPA interaction, a text-chat Head Up Display window (HUD) is developed in Open Wonderland. Text messages sent by the learner to the IPA are typed in this window and IPA text directed to the learner is displayed as well in this window. An example text is the IPA to welcome the learner upon starting a conversation. Further details of interaction in relation to key components and the intelligent agent platform are provided later in Section

‎9.1.3.

In order to allow question and answer conversations, RebeccaAIML is employed for chatbot language support (Rebecca AIML, 2006). The tool relies on the Artificial Intelligence Markup Language (AIML), an XML-compliant language to provide textual natural language processing of questions and answers to virtual characters (AIML, n.a.). The design decision to incorporate AIML is prominent given the non-human entity to give answers back to entered questions.

An example of the populated AIML file is shown in ‎Figure 64. The text-chat module of the IPA is linked to RebeccaAIML that sends questions asked by the learner to its server that will send a reply back of the answer according to pre-defined knowledge base in various scenarios (see Section ‎9.1.3). Processing allows variations of the syntax so as to adapt to different names of the Bot, and

Very simple. First, take control of the panel then select number of charges

</template>

</category>

Figure 64: A Sample AIML file containing a simple question and answer about the experiment.

The AIML based knowledge is populated by different types of questions and answers that include:

 Domain knowledge. Examples are: capacitor knowledge, answers about how to run the experiment, and the level of difficulty of the experiment.

 Questions and answers about the environment. For example, which experiments are available? This is to allow the learner to obtain information about the environment.

 Attention and gesture supporting knowledgebase to emotional state of the learner. For example, “Can I help you?”, “It is not a difficult experiment”. In conversation the answer can be linked to gesture.

 General questions and answers.

An important aspect in considering human question making is the potential big variation of the same question syntax. AIML specification allows extension to context, variation of the question pattern by synonyms, and wildcard usage.

AIML files can be added to the knowledge base by incorporating general or domain AIML files through the management tool of RebeccaAIML. Also new questions and answers can be added into the existing knowledge base files. This feature is used in a training mode the conversation of the IPA to expand its conversation knowledge from Open Wonderland session to allow an educator or administrative user to add to the IPA question / answer knowledge base.

Furthermore, control commands to IPA, by the learner, are given in text. Thus, the interpretation of the commands is also supported by AIML. This design decision, to manipulate learner entered commands by AIML, facilitates variations of syntax in favor of flexibility and for convenience to the learner user for interaction in natural language. Similarly, sensing suitable gestures for the NPC in response to learner questions are triggered accordingly after AIML processing. If the conversation is not triggered by the learner, influence of the agent platform should be expected and thus the details of interaction are led by the IPA through the intelligent agent platform with further details as provided in Section ‎9.1.2.

Specific details of interaction in relation to other key components, and with Jadex are shown in relation to particular scenarios depicted in Section ‎9.1.3.

Adding Voice

The voice feature is added by a Text-To-Speech (TTS) synthesis tool. Mary TTS is a Java based open source (TTS) system that has been developed as a joint effort of the German Research Center for Artificial Intelligence (DFKI) and the Institute of Phonetics at Saarland University and is currently maintained in DFKI‎

(Mary Text To Speech, n.a.). Mary TTS is client-server based allowing sending speech synthesis requests through the HTTP protocol to Mary server. Several other features of Mary TTS are provided in (Mary Text To Speech, n.a.; Schröder

& Trouvain, 2003). An interesting feature is its ability to provide synthesis of emotional speech. When it is needed that the IPA speaks in voice, it sends HTTP requests with the required text to be synthesized to Mary server which will return the speech in voice.

The implementation incorporates adopting simple calls, at needed circumstances, from the IPA module to form HTTP requests that are sent to the TTS server which pronounces the text in the corresponding voice. Details of interaction with other modules are provided in Section ‎9.1.3.

Experiment Object and its Interaction

An experiment has three aspects of interest: control panel, visualization of the result, and its internal operation. The objective is to take an experiment module as a unit of learning that allows the pedagogical agent to manipulate it in learning interaction with the learner avatar. An example learning object is a TealSim Cell (see Chapter 3, Figure 8). While the IPA could instantiate and remove any Open Wonderland cell in the learner scenery, awareness and control of its operation is required to vary its behavior (Soliman, Guetl, 2013b). A mock-up Swing-based module is developed and deployed in Open Wonderland to represent the learning object control panel and to provide necessary information useful for the IPA to provide step-wise guidance and to be able to assess the correct operation, with the following goals:

 It simulates experiment or device operation.