The role of the experimenter

Having the right procedures in place to guide the delivery of the intervention effectively is as important as having the training game programmed well. Good procedures will remove obstacles to engagement, encourage a committed motivated attitude towards the training, and ensure the difficulty is pitched optimally to

encourage cognitive functions development. It is worth noting that many of the intervention delivery procedures could be automated to some degree with more sophisticated programming. The need for some procedures reflects the limitations of the technology or are in place to support the researcher’s evaluation of the

intervention, and as such do not pertain to the intervention directly. It is anticipated that the full automatisation of the training procedures would be possible in

subsequent iteration of the game and with the use of more advanced eye-tracking technologies.

145

Many variables determine each participant’s engagement with the training and assessment tasks on a given day. Given that individuals with ADHD tend be more disorganised, often have poor sleeping patterns, and tend to have more problems in their work or academic activities etc. we might predict that they will tend to be more variable in their disposition across training sessions. Another important determinant of performance is the dynamic of the experimenter-participant relationship. When the intervention is delivered by the experimenter this relationship is difficult to control for or measure. Given this, in subsequent RCT it may be desirable to fully automated training procedures.

On the first day of training the purpose of the training intervention was discussed with participants. While they had been provided with an information sheet to provide an overview of the intervention, on the first day of training the importance of their need to engage with the training and a need for them to challenge themselves to continually improve on the tasks was emphasised. Attention, timing ability, and inhibitory control were described as “brain muscles” that could be trained and strengthened. The training game was described as an external representation of their internal functional ability and that they should think of improved performance as reflecting the strengthening of internal neural systems.

In developing the intervention I attempted to limit the use of externalised

motivation. While I did pay participants for their time this was a low amount, £40 for completing the 10 sessions. This amounts to approximately £3 per hour (£40/13.5 hours of participation). When initially planning to deliver the intervention to children with ADHD, the idea of giving the children an activity folder with stickers and games, with additional sheets to add across sessions was entertained. The intention was to motivate children to maintain their interest in the training sessions. However, it was felt that the game component of the intervention should in itself be sufficiently motivating if the training was to work. The introduction of an external motivator may have acted to weaken the intrinsic motivation in the game, that is, the reasons for the child’s engagement with the game would be pulled in an more extrinsic direction by the off task rewards. While this did not prove relevant to the delivery of the intervention to the mainly adult population used in the proof-of-concept study this has relevance to future delivery of the intervention.

146

During training sessions participants had access to water and savoury snacks were provided (crisps and nuts). They were encouraged to use the toilet at the start of the session and they were allowed to take breaks at any stage during the session. While the participant was engaged in a task peripheral noise was eliminated. The participants were discouraged from conversing with the experimenter while completing a task except conversing about an issue immediately pertinent to the task.

Each training session for each participant had been planned before they arrived with regard to the choice of tasks to be completed, the difficulty of the tasks, and the order of the tasks. The performance of the participant on the previous training session was reviewed and assessed, and decisions made about whether the difficulty parameters should be increased (or occasionally decreased). While parameter

changes can be made within a training session this was generally avoided as reprogramming takes time away from the training and hurried changes can lead to programming errors.

Participants did not have exposure to the entire set of training tasks from the first session. Dependent on the training session new tasks are introduced. On the first session participants completed the stop-signal task, the anti-saccade task, and the forward timing task. In session two the fixation task was introduced, in session three the jumping bomb task, in session five the backward timing task, and in session six the delayed saccade task. This was done to ensure participants were not

overwhelmed with learning a large amount of tasks during the early session, and to maintain a degree of novelty across the training sessions. When any task was first encountered participants first complete a task learning block which is a simplified or scaffolded version of the task, for example, for the anti-saccade task directional cues are provided, for the timing task visual cues are provided to support the learning of the target temporal interval, and for the stop-signal task multiple salient stop-signal cues are provided in conjunction with a short stop-signal delay. Over the course of the training block these features are eliminated. At the end of the block a results screen summarising the task difficulty, points and reaction time data is explained to participants. This explanation is reiterated on the subsequent block and participants are encouraged to interpret and explain this data to the experimenter.

147

For the first two training sessions extra time was taken to ensure that the participant was seated properly and comfortably at the eye-tracking equipment. While this may seem trivial it can have a large impact on the quality of the data collected (as has been discussed in chapter 7). Time was taken to show the participants how they could vary the chair high, the tilt of the seat, the tilt of the back of the seat, and the height of the eye-tracker and table. In addition, at the start of each session the participant was given the opportunity to take some time to relax and talk with the experimenter. It was felt that this was particularly important for some participants as they tended to arrive at the session in a hurried state.

Each participant’s task preferences were noted. A negative attitude towards a

disliked task tended to became more positive as their performance improved on that task. Participants' preferences with regard to trial order were taken into account if they had any; giving them their favourite task as the first task in a session was found to be good for getting them focused and settled. This task was often the forward timing task, a very good task for settling participants as it requires them to wait and be patient while also being focused. How the tasks are organised reflected participants’ preferences to some degree. Some of the participants liked variety and wanted the tasks to change on each block; others liked to have the same task twice for self-competition to see if they could beat their score just achieved. Where possible I switched between a task that required fast responses such as the anti- saccade task and tasks that had a waiting component such as the timing task. Participants were monitored for boredom and fatigue. Longer inter-block breaks were given if needed. If fatigue appeared to be an issue the session was typically reduced in size (e.g. reduced from 50 minute to 40 minutes).

After the task learning block a performance tracking version of the task was

completed to determine the optimal difficulty setting for subsequent levelling blocks. For subsequent trials participants completed the task with a predetermined level of difficulty. The experimenter altered the level of difficulty (typically upping the difficulty) across sessions based on task performance in the previous training session. Different task performance indicators are described above in section 9.4.2 Training Game Tasks, see table 5 for a summary. In the anti-saccade task, jumping bomb task, and delay saccade task the response window was reduced; in the stop-

148

signal task the SSD was increased; and in the timing task the accuracy dependent performance feedback was altered. A number of other task changes were made across sessions independently of these task difficulty parameters. These changes were based on the number of the training session as opposed to the participant’s performance. For example after two training sessions the inter-trial period was changed from a set value (500ms) to a random period (e.g. 300 to 1300ms). In the stop-signal task and the jumping bomb task additional visual cures that scaffolded performance were removed. For the delayed saccade task the random period range between the appearance of the target cue and distracter-go cue was increased.

Table 4. A summary of the difficulty parameter changed for each of the tasks across training sessions.

Task Main Difficulty Parameter Additional Parameters Stop-Signal Task Stop-Signal Delay Response Window

Salience and modality of stop signal

Anti-Saccade Task Response Window

Timing Task Temporal accuracy needed to receive positive feedback

The target interval changes between training session Jumping Bomb Response Window Duration of the penultimate

Bomb

Salience of Distractor Bombs Delayed Saccade Task Response Window Random period between

visual distractor/temporal cue bomb and target bomb Fixation Number of Distractors Period of central fixation

leading to trials success Period of non- central fixation leading to trial failure

A degree of subjective interpretation was involved when varying the task difficulty parameters. Participants performing at around 70% success rate in a task were considered ready for a more difficult level. If they performed consistently at 50% decreasing the difficulty in the next session was considered unless there was a reason for the poor performance, e.g. suffering from a cold, being very tired etc. If a good performance was seen only near the start of the session but then a drop in

149

performance for that same task was seen later in the session then I tended not increase the difficulty. The success rate criterion of 70% is only a rough

approximation. The difficulty was kept slightly easier early on to build participants' familiarity and confidence, and then progressively the achievable success rate was decreased. These decisions were made at the experimenter’s discretion. It is

possible to automate this procedure to some degree but it would be difficult to take account of all the motivational and frustration issues etc. that were taken into consideration.

If delivering an automated version of the training intervention (where the

experimenter is not present in the training environment) it would still be possible to have the experimenter make covert manual alterations to the task difficulty settings between training sessions. In setting task difficulty is should be noted that early on there will be easier gains but that diminishing returns in improved absolute

performance scores are expected across sessions.

A possible solution might be to allow the participant to set the difficulty level themselves within certain parameters. For this setup it would also be possible to provide information on their improvement rate when the task is set to various levels of difficulty. This might afford a mechanism by which they can learn about their own learning process. However, such additions may also destroy the flow of the game and can detract from the intentioned focus.

Independent of the between session changes to difficulty parameters, the task difficulty within a single block/level automatically increased across the three sub- blocks or “waves”. The first wave had a difficult of 75% of their maximum ability (20 trials), the second wave was set at 85% (20 trials), and the final wave was set at 105% (with the exception of the anti-saccade task set to 100%).

The period between blocks, approximately 2 minutes, provides participants with a short break. This break allows participants a short moment to recover and reflect on their performance. An effort was made to dissuade participants from rushing

between tasks. Between blocks the role of the experimenter was to encourage participants to (i) verbalise their strategy for the task completed; (ii) be explicit about what they are doing when they complete the task; and (iii) reflect on their

150

performance asking themselves why they performed well or poorly. The difficulty of the task was also discussed with participants and they were asked how difficult they found various parameter settings. A brief review of the performance in the session was discussed at the end of a session. Participants were congratulated after good performances and encouraged to set performance targets for subsequent blocks. Setting targets was found to be extremely motivating for some participants. For such participants having the same task type with the same difficulty settings back-to-back was a useful form of motivation. Participants who did not wish to discuss the tasks or their performance were not pursued.

A critical function of the experimenter was to monitor the quality of the eye-tracking image. A good image is essential to ensure the gaze-contingent algorithms are executed correctly. A poor quality image can be the result of a participant’s shift in position, a poorly focused image or occasionally simply due to equipment failure. It is the role of the experimenter to maximise the quality of the eye-tracking data and to ensure that failed or successful trials reflect the participant’s performance. Incorrect automated feedback has the potential to undermine the whole training intervention. It can lead to frustration in participants and lead to the development of strategies focused on circumventing the limitations of the equipment as opposed to improving the target of the training intervention. If it is obvious that automated incorrect feedback has been given to the participants it is important that the

experimenter highlights this immediately. Participants need to have confidence in the feedback they receive in order that it can guide their learning. If they begin to question the feedback the efficiency with which the training program can focus their learning on the critical features of their performance is undermined. In order to maintain their trust in the automated feedback they receive they should be

reassured upon equipment failure that this is an exception. It helps to explain to the participant why the equipment failed (if the reason is known).