Data collection

This experimental application was aimed at observing both if the developed solution was able to achieve the intended purpose, and if its features were appropriate for that. The initial purpose of promoting active behaviours in play was supplemented by the need for communicating contents through the game, a goal emerged from the co-design sessions with the experts of the educational center for children. The ability of the solution to meet these two purposes was estimated taking onto account different aspects, that are quantity of body movement, efficacy in supporting a week-long activity as a real application, and the 100

Fig. 5.10 - New version of the catching game co- designed with the experts of the educational center.

persistence of interest toward the solution from the experts of the educational center.

Other more specific aspects were addressed for evaluating both the usability and the pleasurability of the solution, which should be two prerequisites of a fun experience (Vieira et al. 2017). Accordingly, at the level of the features, the platform was evaluated taking into account the following aspects:

likeability, learnability, enjoyment, and engagement.

The data collection was carried out adopting two different strategies: direct observations and self- reported data from children. As it is common for wild studies, in fact, this real context posed a series of constraints, such as confidentiality issues (Ros et al 2011) and lower level of control on the study

(Baxter et al 2016). Indeed, the fact of involving real school groups made impossible to have the parents consent for recording the experience. In order to overcome this issues, direct observations, with the support of observational forms, were performed. These forms were developed referring to existing observational tools used in Behavioural, Developmental and Nutritional Sciences to observe children, especially in play and physical activities. In particular, the existing tools taken as reference for observing physical activity during play in real contexts were SOFIT (McKenzie et al 1991) and

SOCARP (Ridgers et al 2010). These two tools, in

fact, provide forms that observers can use to record children activities. In addition to the general

Fig. 5.11 - Schema of the test’s setups at the educational center for children. On the bottom are showed the two setups managed by the research team.

information about the observed, these forms ask to estimate aspects such as the level of activity, engagement, and enjoyment by giving a value in a five points scales. A series of self-reported data about the experience were, instead, asked to children. These were collected through a semi-

structured interview carried out by the conductors,

as an informal conversation. Children were asked to say how much they enjoyed playing; how difficult was to understand the functioning and playing; and, finally, they were invited to say what was the preferred setup and why. They were asked to give a value, from 1 to 5, about enjoyment and difficulty. In order to help children in visualizing the values, the conductors were provided with paper sheet on which was printed a 5 values bar chart. The forms edited for this study, hence, were composed by four main sections: child personal data; 5 points Likert scale questions about observed physical activity, engagement, enjoyment and concentration; semi- structured interview; and free comments areas.

Participants

The participant’s sample is composed of 17 classes of third, fourth and fifth years of primary school. A total of 366 children aged between 6 and 10 years. However, the data analysed refer to a sample of 270 children, 135 of which experienced the game platform with the robot and the other 135 without the robot. The majority of children was male, 54%.

The rest of the sample (96 forms) was excluded from the analysis for the incompleteness of the data.

5.6.4 Results

A first aspect addressed during the test was

enjoyment. About that, the results of the semi-

structured interviews revealed a general appreciation of the game platform.

Almost the totality of participants declared that enjoyed playing with the platform (table 5.4), and an average of about 80% of them were totally amused. In particular, children who experienced the platform with the robot resulted in a higher level of enjoyment.

These data, however, were not completely reaffirmed by the reports of the observations, which focused on facial expression of children for estimating the level of concentration and enjoyment during play. In the forms, the level of enjoyment during play was associated with five main face expressions: bored, when the child never smiles and the face looks apathetic; not much amused, when the child does few hints of smiles; quite amused, when the child has a serene face; very amused, when the child smiles; and enthusiastic, when the child has a cheerful face and laugh. According to the observers (table 5.5), in fact, only less than 40% appeared enjoyed, while another 40% looked neutral and around 17% were not enjoying the activity.

102

Table 5.4 - Enjoyment stated by children, about the Immersive Setup (IS), both with and without the robot

This discrepancy between the data points out that the experience was able to foster a positive valence, although without generating excitement, except in few cases.

However, by looking at the data regarding the concentration, this low level of the observed enjoyment may be motivated by the high level of attention required by the game. During play with the platform, in fact, children appeared mostly concentrated, and around 30% of them were totally concentrated.

In fact, by looking at the data regarding the children who appeared not much amused it is possible to notice that 70% of them was considered also totally concentrated and, another 20%, very concentrated (table 5.6). Therefore, it is possible to presume that a little smiling face is mostly determined by a high level of concentration and not necessarily to a low

level of enjoyment. This somehow reaffirm the generally enthusiastic feedback given by children about the experience and reveal a high level of

engagement.

The high level of concentration is also related to the game difficulty. By looking at the answers about the difficulty of the game given by the children, playing in the immersive setup resulted easier than playing with Sphero in the non-immersive setup. In fact, almost 70% of children stated that controlling the ball with the body was little or not difficult at all.

Catching the projected figures in the immersive setup was also considered easy. Just 17% of participants, in fact, stated that it was difficult or very difficult, while striking the cubes in the non- immersive setup was considered difficult or very difficult by almost 30%.

Table 5.6 - Concentration reported by the observers, about the Immersive Setup (IS), both with and without the robot

Table 5.5 - Enjoyment reported by the observers, about the Immersive Setup (IS), both with and without the robot

Apart from the difficulties in controlling the robot and catching the figures/cubes, the initial phase of the game resulted challenging for children.

In terms of learnability, consisting of the initial difficulty in understanding the game logic, the two setups had analogous feedbacks.

Almost 40% did not find it difficult, while 30% said that it was quite difficult and another 30% stated that it was very difficult.

However, in 14 children’s comments, some difficulties regarding the immersive setup

learnability were explicitly pointed out. The main

factors determining misunderstanding about the game were: tendency of “catching” the projected figures walking over them rather that controlling

the robot to go over them; tilting the breast instead of moving around for controlling the ball; thinking that it is necessary to stay in the location indicated at the beginning of the game for the start; and inattention during the game explanation by the tutor. In some cases, in fact, the explanation was repeated two times. Finally, some children mentioned the precision of the game, especially about the non-immersive setup.

The game difficulty was also pointed out as a factor determining the preference of one setup over another, which was addressed as an indicator of likeability. By looking at the children’s answers and comments, in fact, a significantly higher preference of the non-immersive setup (60%) over

the immersive one was noticed and difficulty was pointed out as a preference factor by more than 10% of children. Some of them stated to prefer the immersive setup over the other because it was easier, and in some cases the ease was recalled to the intuitiveness of the solution. The other setup, instead, was preferred by some because was easier and by others because it was more difficult. These results look conflicting and it is not possible to support a hypothesis according to which the more the game is easy the more it is fun, or its contrary. However, these comments highlight the importance of the relationship between difficulty and game enjoyment, which will have to be addressed for future improvements.

The preference of one setup over another, however, was strongly influenced by a series of other factors. The most influencing was probably the presence of the robot. The feedbacks were more positive with the robot and in the case of the immersive setup, children who expressed a maximum appreciation were 70% in the case of interaction with a projected ball and 86% when playing with the robotic ball. Although the non-immersive setup was preferred only by 40% of participants, the percentage of preference of this setup was higher when experienced with the robot (+4%). Furthermore, in the 7% of children’s comments the preference of the non-immersive setup was motivated by the presence of the robot. In these cases, they 104

Table 5.7 - Preference of the two setups stated by children and some recurring motivations found in children’s comments.

experienced the immersive setup without the robot. The comments revealed also other recurring factors.

Regarding the preference of the non-immersive setup, many children highlighted positively the possibility to control the robot through the smartphone, a play modality that one kid defined more technological. Almost a fifth of participants enjoyed a lot the fact of striking the cubes and making the Sphero jump on the ramps.

Nevertheless, the comments highlighted also positive features that made the immersive setup preferable. In most of the cases, the preference of this was determined by the possibility of controlling the game through the body.

In fact, this feature was pointed out by the 35% of children and the 14% of them has positively remarked the fact that the platform requires them to move.

This factor introduces the last aspect addressed in the observations, that is the movement during play. The data reported by the observers reveal that the platform and the game tested were not producing a great amount of movement in the players.

According to the observations, almost half of participants was doing constant but little movements, and the 28% of them were moving just a minimum. Some children showed more active and energetic behaviours, characterized by constant movement and body gestures (shaking harms, jumping), but these were only around 30%.

No significant differences were found between the data regarding the sample that played with the robot and the one that played without it. On

the contrary, a considerable correlation was found between the quantity of movement and the level of concentration, both reported by the observers. By focusing on the data about the 78 children who appeared performing a minimum quantity of movement, the vast majority of them was also appearing very (41%) or totally concentrated (25%). And also the rest appeared quite concentrated. This data reveal that the platform tested with the “coordinates” game was unable to foster a great amount of movement. This was probably due to the high level of concentration required by the game and some of the learnability issues mentioned earlier.

Nevertheless, lower level of arousal and higher concentration appear to be more suitable conditions for the playful learning activities carried out at the educational center. The activity carried out with the platform, in fact, was easily managed

in parallel with the others, without affecting negatively or distracting children from the general topic. This represents a positive feedback in terms

compatibility of the solution with the context and

the existing practices.