The first experiment examined the possibility of learning the conceptual knowledge of actions between novel manipulable objects by near-minimum amount of observation. The experiment includes a pre-training session, two 20 min-training sessions, and a post-training session.
In the pre- and post-training sessions we manipulated the correctness of co-location (correct vs. incorrect) and the object layout (active objects in the left and passive objects in the right visual field vs. vice versa), and created four conditions comparable to previous studies in which the effects of implied actions on affordance selection were established (Chapter 2). In the incorrect co-location condition, the orientation of the active objects was manipulated while the orientation of the passive object was maintained relative to the correct co-location condition. Examples of the stimuli are shown in Figure 2-1. To produce robust effects of implied between-object actions, in this chapter we only manipulated the orientation of the active, not the passive, objects in the incorrect co-location condition, same as in Chapter 3 and 4. The task in the pre- and post- training sessions was similar to the material evaluation in our previous study (supplementary materials of Chapter 2). In both the pre- and post-training sessions, the participants rated pairs of novel and familiar objects regarding (a) whether the action relations between the objects were familiar and apparent, and (b) whether, by changing orientation of the active objects in the incorrect co-location condition we effectively manipulated the implied actions between objects, (c) whether the objects presented on the left side reliably afford left hand responses and those on the right side right hand responses, and (d) the appropriateness of our assignment of active and passive objects.
In the training sessions only the novel object pairs were used. The participants watched videos showing the active objects being used upon the passive objects. A recognition test followed each video, making sure that the participants paid attention to the videos and they
113
could associate the line drawings of the objects (to be used in the pre- and post-training sessions) to corresponding video representations.
We compared the ratings in the pre- and post-training sessions to examine whether the participants managed to learn the possible interactions between the novel objects by viewing video demonstrations. We were mainly interested in: (a) whether the action relations between novel objects were rated more familiar and apparent in post-training session relative to the pre-training session, (b) in post-training rating, whether, by changing orientation of the active objects in the incorrect co-location condition we effectively manipulated the perceived readiness of the actions between novel objects as we did between familiar objects, and (c) after the training, whether the participants were more likely to identify the active novel objects as “active” in each pair.
5.2.1 Methods
5.2.1.1 Participants
A group of volunteers (one male and 11 females, age range: 18-25y) from the University of Birmingham research participation scheme participated in Experiment 1. All participants are right-handed and have normal or corrected-to-normal vision. Participants gave informed consent and received course credits for their time.
5.2.1.2 Materials and Procedure
In Experiment 1, greyscale clip-art style images of 12 pairs of familiar objects and 11 pairs (originally 12, one discarded because the rating in Question 5 [see below] indicates its inappropriateness. See Appendix 5-C for details of itemwise analysis of the novel objects) of novel objects constructed for the present study. All object images were presented on a light grey background (240, 240, 240 RGB). Each object images subtended 3.2°×3.2° of visual angle. The relative sizes of the objects within each pair matched their relative sizes in real life.
114 Figure 5-1. An exemplar pair of novel objects, the object on the right is the active object.
It is used to rotate the passive objects.
The familiar objects had been used in previous chapters and had been found effective in evoking automatic prioritization of active over passive objects in the paradigm used in following experiments (for a full list, see Appendix 5-A). The novel objects were constructed with Lego blocks and plasticine. They were designed as pairs, with one (active) object designed to be used upon the other (passive) one to carry out one of the four possible functions: rotating, lifting, pushing and fitting in (See Figure 5-1 for exemplar objects, and see Appendix 5-B for a full list). The spatial extent of the novel objects was similar to that of the familiar objects (average size of the objects by pixels: familiar objects: 63×49; novel: 60×55), and for both categories of object pairs their centres overlapped with the centre of the screen.
The pre- and the post-training sessions were conducted on different days, separated by 0-2 days. Each session consisted of five blocks. In each of the first four blocks each object pair was presented in four variations (as shown in Figure 5-1), and each variation was evaluated in one trial, resulting in 96 trials per block. The variations were generated by manipulating orthogonally the object layout (active-left and active-right) and the co-location (correct and incorrect co-location condition). In this way, we replicated all the possible displays of a given pair in the response compatibility task in Experiment 2 and 4. In each trial, the object pair was presented at exactly the same location and of the same size as they would be in Experiment 2 and 4, and the questions were presented below the images. The participants were asked to answer the questions on a five-point scale, ranging from “1:
definitely No” to “5: definitely Yes”. In the last block, the objects were presented always in the correct co-location. Consequently each object pair was presented only twice, once with the active object on the left side and once on the right side of the screen. The participants
115
evaluated the distinction between active and passive objects in each object pair by indicating which one is more likely to be used upon the other one, and the responses were coded as 1 for the left object and 2 for the right object. Through all five blocks, the object pair, the question and the choices remained on the screen until a response was made.
In each block the participants evaluated all object pairs and their variations according to a same question. The order of the questions was constant, but the order of object pairs within each block varied across blocks and participants.
The five questions served four main purposes:
a. Familiarity of the action relation Regarding whether the objects in each pair are typically involved in certain action relation, and whether the action relations between objects were recognized in the incorrect co-location condition :
Are these objects typically used together?
This question was for block 1.
b. Readiness of interaction Regarding whether the co-location between objects is appropriate for an implied between-object action in the correct co-location condition but not in the incorrect co-location condition:
Are these objects appeared to be currently used together? Or, are they positioned properly or likely to be used together?
This question was for block 2.
c. Object affordance Regarding whether the assumption is valid that objects presented on the left side afford left-hand responses while objects presented on the right side afford right-hand responses:
When the pair of objects are located in the way they are currently located on the screen, and you are going to use them together, which hand are you going to use to handle the object on the right side of the screen?
116
When the pair of objects are located in the way they are located on the screen, and you are going to use them together, which hand are you going to use if you are going to handle the object on the left side of the screen?
These two questions were for block 3 and 4 respectively.
d. Distinction between active and passive objects regarding which object in each pair was active. The question was presented as:
When the pair of objects are located in the way they are located on the screen, and you are going to use them together, how will these two objects interact? Please press 1 if the object on the left hand side is going to be used upon the right one, and press 2 if the right object is going to be used upon the left one.
Each question was verbally explained to the participants and the participants only began answering the questions after they confirmed that they understood each question clearly.
Especially, for Question b, it was explained to the participants that the question asked whether they perceive the co-location between objects were conducive to interaction or not, and the “Or,” in Question b leads an alternative expression for the same question rather than a different question.
The two training sessions were identical, with the order of trials varied. In the training sessions the participants watched videos, each depicting a pair of novel objects showing the active object being used upon the passive object when they were presented correctly, or a video showing the objects being picked up and then put down without any interaction between them when the two objects were incorrectly co-located. The participants were informed that a recognition test will follow each video, in which they need to identify which two objects were just presented. Each pair of novel objects was presented four times in each training session, the layout of the objects and their co-location manipulated orthogonally. The average length of videos is 16.7s (SD = 2.1). After each video, four line drawings (consisting of the two objects in the previous video and two novel objects randomly picked from the pool)
117
will be presented on the screen and the participants were required to pick which two just appeared. The participants would receive immediate feedback when the choice was wrong, which rarely happened. One training session will be conducted immediately after the pre-training session and the other one immediately before the post-pre-training session, separated by brief breaks.
5.3.2 Results and Discussion
Considering the nature of our data, we conducted median based non-parametric analysis to test our main hypotheses, i.e. (a) whether the action relations between novel objects will be rated more familiar and apparent in post-training relative to pre-training session, (b) in post-training rating, whether, by changing orientation of the active objects in the incorrect co-location condition we effectively manipulated the implied actions between novel objects as we did between familiar objects, (c) whether the participants reliably consider the objects afford actions by the hand spatially aligned with them, and (d) after training, whether the participants were more likely to identify the active novel objects as
“active” in each pair.
Additionally, we used mean-based parametric (ANOVA) methods to investigate the interaction between factors. The application of parametric method on data from Likert scale has been suggested acceptable (Norman, 2010).
Familiarity of the action relation The familiar objects were evaluated as typically involved in interaction, and this perception persisted when the two objects were presented in the incorrect co-location, while the evaluation of the novel objects did not show such conviction in either co-location condition. In response to the question “Are these objects typically used together?” on a five-point scale ranging from “1: definitely No” to “5: definitely Yes”, the median of response score to the correctly and incorrectly co-located familiar object pairs significantly diverted from the mid-point, in both pre- and post-training sessions. In contrast,
118
for the novel objects, the ratings only significantly differed from the mid-point after training (shown in both non-parametric and parametric tests. For detail and exact statistics, see Table 5-1). For the familiar objects, the rating for the correct and the incorrect co-location conditions did not differ from each other in both pre- and post-training sessions. For the novel objects, both correctly and incorrectly positioned object pairs were rated higher in the post-training session than the pre-post-training session (shown in both non-parametric and parametric tests, for detail and exact statistics, see Table 5-3). To illustrate the distinct patterns of training on the familiar and novel object pairs, we used repeated-measures ANOVA on the ratings of this question. The mean rating scores in each condition of each participant were entered into the analysis, with object type (familiar vs. novel), session (pre- vs. post-training) and co-location (correct vs. incorrect) as within subject factors. We found that there was a main effect of object type, F (1, 11) = 39.00, p < .001, η2= .78, rating for the familiar objects being higher than for novel objects (MD = 0.91). There was a main effect of session, F (1, 11)
= 46.91, p < .001, η 2 = .81, rating in post-training session being higher than in pre-training session (MD = 0.91). There was a main effect of co-location, F (1, 11) = 5.21, p = .043, η 2
= .32, rating for the correct co-location being higher than the incorrect co-location (MD = 0.05). The interaction between object type and session was significant, F (1, 11) = 29.13, p
< .001, η 2 = .73. Simple effect analysis suggested that the difference between pre- and post- training sessions existed only in the rating of novel objects (MD = 1.31, p < .001), but not in those of familiar objects (p = .13). The interaction was illustrated in Figure 5-2.
119 Table 5-1: The median ratings for familiarity of action relations
Mean SD Median
Wilcoxon rank test:
difference between 3 (mid-point of the scale)
Familiar objects
Pre-training Correct co-location 4.87 0.15 5.00 >.001
Incorrect co-location 4.80 0.25 5.00 >.001
Post-training Correct co-location 4.93 0.13 5.00 >.001
Incorrect co-location 4.95 0.13 5.00 >.001
Novel objects
Pre-training Correct co-location 3.45 0.66 3.25 .070
Incorrect co-location 3.36 0.71 3.00 .111
Post-training Correct co-location 4.56 0.55 5.00 .002
Incorrect co-location 4.51 0.55 5.00 .002
Figure 5-2. Rating on action familiarity in Experiment 1. The rating of action relation familiarity increased for novel objects after training, and remained high in both pre- and post-training sessions for familiar objects. The error bars indicate the standard error of each condition following the method proposed by Cousineau (2005). The significance of pairwise comparisons is denoted on the figure (a = .05).
Readiness of interaction Manipulating the orientation of active objects significantly affected the perception of implied interaction for familiar objects in both pre- and post-training sessions and for novel objects in post-training sessions. In response to the question “Are these objects appeared to be currently used together? Or, are they positioned properly or likely to be used together?” on a five-point scale ranging from “1: definitely No” to “5:
definitely Yes”, the median of response score to the correctly co-located familiar object pairs significantly diverted from the mid-point to the direction of Yes, while for the incorrectly co-located familiar object pairs the responses significantly diverted from the mid-point to No, in both pre- and post-training sessions (Table 5-2). In contrast, for the novel objects, the rating
1
120
did not significantly divert from the mid-point when the objects were presented correctly in pre-training session, but significantly diverted to the No direction when incorrectly presented.
In the post-training sessions, novel objects in the correct co-location condition were rated significantly different from mid-point towards the direction of Yes, while for the incorrectly co-located object pairs the responses significantly diverted from the mid-point to No, replicating the rating pattern of familiar objects (shown in both non-parametric and parametric tests, for detail and exact statistics, see Table 5-2). Paired comparison confirmed that the training sessions increased the interaction-readiness rating for novel objects when they were presented in the correct co-location, but not when they were in the incorrect co-location.
Though the novel objects were rated more likely to imply interactions in the correct than in the incorrect co-location condition both before and after training, the mean difference was larger in post-training session than in pre-training session. Training did not affect the ratings of familiar objects in either co-location condition, or in either session (shown in both non-parametric and non-parametric tests, for detail and exact statistics, see Table 5-2). To illustrate the distinct patterns of training effects on the familiar and novel object pairs, we applied repeated-measures ANOVA on the ratings of this question. The mean rating scores in each condition of each participant were entered into the analysis, with object type (familiar vs.
novel), session (pre- vs. post-training) and co-location (correct vs. incorrect) as within subject factors. We found a main effect of object type, F (1, 11) = 36.20, p < .001, η2 = .77, rating for the familiar objects being higher than for novel objects (MD = 0.58). There was a main effect of session, F (1, 11) = 10.63, p = .009, η2 = .49, rating in the post-training session being higher than in pre-training session (MD = 0.37). There was a main effect of co-location, F (1, 11) = 121.33, p < .001, η2 = .92, rating for the correct co-location being higher than the incorrect co-location (MD = 1.20). The interactions were significant between object type and session, F (1, 11) = 10.25, p = .008, η2 = .48, object type and co-location, F (1, 11) = 114.48, p < .001, η 2 = .91, session and co-location, F (1, 11) = 5.55, p = 038, η 2 = .34, and all three factors, F (1, 11) = 12.16, p = .005, η 2 = .53. Simple effect analysis suggested that the
121
difference between pre- and post-training sessions existed only in the ratings of novel objects when their co-locations were correct, with training increasing the rating of interaction readiness (MD = 1.10, p = .003), but not that of familiar objects, or novel objects in the incorrect co-location condition (ps >.066). In addition, the interaction between object type and session was significant in the correct co-location condition, F (1, 11) = 15.72, p = .002, η2
= .59, but not in the incorrect co-location condition (p = .18), suggesting training selectively affected the perception of implied actions in novel object pairs in the correct co-location condition. The interactions were illustrated in Figure 5-3.
Object affordance. The association between object location and its affordance was evident across all conditions. In response to the question “When the pair of objects are located in the way they are currently located on the screen, and you are going to use them together, which hand are you going to use to handle the object on the right side of the screen?” on a five-point scale ranging from “1: definitely left” to “5: definitely right”, the overall likelihood of using the left hand for the object presented on the left side of the screen was significantly larger than for the objects presented on the right side of the screen, and the dissociation persisted in both the correct and the incorrect co-location conditions. The difference between ratings in the correct and the incorrect co-location conditions was not significant regardless of whether the objects were on the left or right side of the screen. The results were replicated by Chi-Square test (See Table 5-4 for detailed statistics).
122 Table 5-2: The median ratings for readiness for interaction
Mean SD Median
Wilcoxon rank test:
difference between 3 (mid-point of the scale)
Familiar objects
Pre-training Correct co-location 4.69 0.25 5.00 .001
Incorrect co-location 1.84 0.45 1.50 .002
Post-training Correct co-location 4.77 0.22 5.00 .001
Incorrect co-location 2.11 0.63 2.00 .018
Novel objects
Pre-training Correct co-location 2.84 0.50 2.75 0.187
Incorrect co-location 2.14 0.46 2.00 0.008
Post-training Correct co-location 3.94 0.83 4.50 .012
Incorrect co-location 2.17 0.70 2.00 .012
Figure 5-3. Rating of perceived readiness of interaction in Experiment 1: Training increased the perceived readiness of interaction between novel objects in the correct co-location condition. The error bars indicate the standard error of each condition following the method proposed by Cousineau (2005). The significance of pairwise comparisons is denoted on the figure (a = .05).
123 Table 5-3: Pairwise comparison between median ratings for familiarity of action relations and
readiness for interaction Familiarity of action relations: effect of co-location
Familiar objects Pre-training 1.10 11 .295 1
Post-training -0.97 11 .352 1
Post-training -0.97 11 .352 1