Behavioural Analysis

Besides formal questionnaires given to teachers, the video analysis of the children’s behaviours played an important role and they are the main source of information. The produced videos were analysed using the specialized software The Observer XT from Noldus (Noldus, 1991) to quantify predetermined behaviours performed by children. The selection of the behaviours was done according to its relevance (regarding en- gagement, and interactive behaviour), and feasibility to identify them in the recorded data. Regarding non-verbal communication, the literature was consulted to choose

Chapter 4. Scenarios for Human-Robot Interaction in Children with ASD 107 the expected behaviours to be observed in children with ASD (Mundy et al., 1986). In each study, the coded behaviours from the list below will be presented according to the goals of the study. In this list, state events stand for behaviours that take a period of time and therefore have a duration. Point events stand for a behaviour that only takes an instant in time, or whose duration is not important.

• Eye Gazing Behaviour (State Event):

– Robot: head orientation of the child looking towards the robot;

– Experimenter: head orientation of the child looking towards the experi- menter, except experimenter’s eyes;

– Task’s Material: head orientation of the child looking towards the task’s material;

– Eye Contact: head orientation of the child looking towards the experi- menter, specifically to her eyes;

• Tactile Interaction (Point Event):

– Robot’s Body: from the moment the child touches the robot on its body; – Robot’s Head: from the moment the child touches the robot on its head; – Experimenter’s Body: from the moment the child touches the experimenter

on her body;

– Experimenter’s Head: from the moment the child touches the experimenter on her head;

For all the above behaviours regarding tactile interaction, they were classified into:

∗ Spontaneous: the experimenter did not encourage the child to perform the behaviour;

∗ Prompted: the experimenter encouraged the child to perform the be- haviour;

Touches were also classified into gentle and harsh;

• Non-Verbal Communication (Point Event & State Event (for imitation, smiling and leaning forwards)):

– Following: the child follows with head movement (eye gaze if possible) a pointing gesture (with index finger or hand) of the experimenter (even if the pointing gesture is not being performed any more);

– Pointing: the child points at something with index finger to attract the attention of the experimenter;

– Robot’s Imitation: coded when the child copies movements from the robot; – Experimenter’s Imitation: coded when the child copies movements from

the experimenter;

– Smiling: upward curving of the corners of the child’s mouth; – Clapping Hands: the child joining hands together producing sound;

– Leaning forwards: the child leans forward towards the robot/experimenter or stands up getting closer while either looking at or touching;

• Verbal Communication (State Event):

– Echolalia: words or sentences repeated after the experimenter or the robot said them;

– Vocalisations: speech that becomes broken down, cluttered, or unintelli- gible due to a variety of reasons, and oral sounds made by child without meaning;

– Speech: words and sentences said by the child; • Prompts (Point Events):

– Ins-Happy: prompt made either by the robot or the experimenter to request the answer happy;

– Ins-Sad: prompt made either by the robot or the experimenter to request the answer sad;

– Ins-Surprised: prompt made either by the robot or the experimenter to request the answer surprised;

– Ins-Afraid: prompt made either by the robot or the experimenter to request the answer afraid;

– Ins-Angry: prompt made either by the robot or the experimenter to request the answer angry;

• Answers (Point Events):

– Happy, Sad, Surprised, Afraid, Angry: answer given by the child; – Successful: Right answer to the previous prompt;

Chapter 4. Scenarios for Human-Robot Interaction in Children with ASD 109 – Unsuccessful: Wrong answer to the previous prompt;

– Unanswered Prompt: There is no answer from the child or when the ex- perimenter repeats the previous prompt;

The following list summarises the behaviours coded in the performance task of the pre- and post-test used to assess the knowledge of the child regarding emotion recognition. This task is explained in section 4.6.

• Happy, Sad, Surprised, Afraid, Angry: emotion the child is attempting to match with PECS card;

– Right Answer: If the child puts the picture of the person in the right place; – Wrong Answer: If the child puts the picture of the person in the wrong

place.

• Duration: Duration of execution of the performance task

Independent raters were trained to code the behaviours above. Their rating included the following rules:

• When looking at an interaction, the rater should, first, classify the function; second, decide who initiated the function; to establish if the child’s behaviour is prompted or spontaneous, if applicable; and third, identify the particular be- haviour code;

• If a behaviour is not well-defined, it may not be rateable. It is better not to rate a behaviour than to categorize it without sufficient information;

• Do not code any behaviour that is obscured (e.g., by the experimenter blocking the camera’s view of the child);

• When the child exhibits behaviours that were not specified in the list, they are not coded;

• A behaviour ends if the child stops exhibiting that behaviour or shows another behaviour, directly related (for example, looking at ZECA/looking at the exper- imenter);

• For Eye Gaze, turning away ends the behaviour. Turning back immediately and looking again counts as new behaviour;

• For Tactile Interaction, mark whether the child shows the behaviour sponta- neously or whether the behaviour is prompted by the experimenter. If the child touches the robot for no specific reason, the behaviour is classified as sponta- neous. If the experimenter gives an indication such as “Do you want to touch him?”, the behaviour is considered prompted;

• For Tactile Interaction, if the child hits, pushes, or grabs the robot, the touch is marked as harsh. Poking eyes, nose or mouth is considered harsh touch. If the child taps, tickles or touches the robot, the touch is marked as gentle;

• For Following, the emphasis is on behaviours made after the experimenter directs attention to an object or event, thus establishing a common focus of attention between the child and adult. This behaviour should be coded when the experi- menter starts pointing;

• For Pointing, the emphasis is on behaviours used by the child to request the experimenter’s attention for any objects or events. This behaviour should be coded when the child starts pointing;

• For Imitation, repetition is not coded if the child is performing that particular action previously. Verbal repetitions are not considered imitation;

• An activity starts after the sentence “Push START to begin”, and an activity ends after the robot says “See you soon”;

The onset and offset times of behaviours and coding of events were used to record the behavioural sequences.

Recording the onset and offset times was chosen because, for most of the research questions, time information is necessary (Bakeman & Gottman, 1997). In this way, it is possible to report time-budget information and report different kind of behaviours coordinated with time. To facilitate, it is useful that these codes are mutually exclusive because the offset times do not need to be recorded. In such cases, the offset of a code is implied by the onset of another mutually exclusive code. So, with this recording scheme, it is possible to preserve a complete record of how behaviour unfolds in time, recording the onset (and offset, when it is necessary) times for all events that can be coded.

Chapter 4. Scenarios for Human-Robot Interaction in Children with ASD 111 The use of coding events is useful, when there is a concern with the sequence of be- haviours rather than their duration. So, coding events will also be a recording scheme to be used when analysing data.

After choosing a recording scheme, it is important to decide which observational data representation will be used. Depending on how data is recorded, different represen- tations from the same data can be extracted for different purposes. For the studies presented in this thesis, the appropriate data representation form is time-event se- quences (Bakeman & Gottman, 1997). Once data is represented in this form, it is possible to determine, for instance, how often specific behavioural codes co-occur, or whether certain behavioural codes tend to follow or precede other codes in systematic ways. As mentioned before, event sequences are used to represent observational data from events coded. Event sequences consists simply of codes for the events, ordered as they occurred.