Adaptive design

Traditionally, for each trial of a learning stage in the 7-stage task, researchers pre-determine which stimuli will be put into each bowl. Such a fixed trial design is meant to counterbalance across rats the positioning of different stimuli, such that the reward-associated stimulus (e.g.,

M1) appears both in the left and in the right bowls over trials, and the pairing of different stimuli (e.g., O1 and O2) of the other dimension to the correct stimulus. However, this fixed design does not consider the performance of each rat (or human) over trials. For example, a rat may be using an incorrect response pattern to choose bowls and the food reward may happen to appear in the chosen bowls in several consecutive trials. In this case, the rat may strengthen the association between the incorrect response pattern and food reward. Consequently, the rat has to spend more trials in finding the food-associated response pattern. Using Bayesian analysis of a rat’s performance in the previous trial, we can adaptively determine which bowl will contain which stimuli for a given trial. I call such a dynamic process adaptive trial design. Different goals of adaptive trial design require different design principles. One possible goal of adaptive trial design is to potentially speed up rats’ learning in each stage. This goal could be achieved by minimising ambiguity and maximising information available to the rat concerning its most likely hypothesis. For example, suppose Bayesian analysis tells us that the rat is likely to respond to O2, even though M1 is reward- associated, then for the next trial, M1 and O2 can be placed in different bowls. In this case, the rat would not be rewarded if it chooses the bowl with O2. This will have the effect of decreasing the association between the incorrect response pattern (‘O2’) and food reward. In this way, the rat may more quickly find the correct response pattern.

The above example also indicates the principle of adaptive design for the goals of speeding up rats’ learning: put the food-associated stimulus into the bowl that the rat is least likely to choose. In more detail, suppose M1 is associated with food reward, each trial can be adaptively designed as follows:

(1) Determination of stimulus pairs: if the Bayesian probability of O2 is higher than that of O1 based on the last trial’s observation, then put M1 and O1 into one bowl, and put the remaining stimuli (M2 and O2) into the other bowl. Otherwise, put M1 and O2 into one bowl.

(2) Determination of side: If the rat is more likely to choose the left bowl than the right bowl based on Bayesian analysis of all four spatial hypotheses in the last trial, then put the food-associated M1 (and its paired odour stimulus) into the right bowl. Otherwise, put M1 into the left bowl.

Based on the above principles, we can adaptively design each trial based on the Bayesian analysis of previous trials, which will potentially speed up rats’ learning process.

Figure 10.1 shows the brief structure of the adaptive design software implemented in MATLAB. For each trial, the software displays the bowl information through the user interface (see the ‘Left’ and ‘Right’ bowls in Figure 10.2). Based on the displayed bowl design, the user (i.e., experimenter) sets up the real trial, and then observes and inputs the rat’s behaviour through the user interface (Figure 10.2). Then the software performs Bayesian analysis of the current trial to estimate the posterior probability of each hypothesis. Based on the posterior probabilities and the adaptive design principles, the software determines and displays which bowl should contain which stimuli for the next trial. The above process is repeated until each learning stage is finished by satisfying the Bayesian criterion (i.e., Bayesian probability of the correct hypothesis is larger than 0.95 by default). The rat’s behavioural responses and the estimated posterior probabilities for each hypothesis are saved into one Excel file for each rat.

Figure 10.2: a screen-shot of the adaptive design software. The stimuli in the ‘Left’ and the ‘Right’ bowls are determined from the adaptive design process, with the bold green stimulus associated with reward. User can input rat’s behaviour by clicking one of the two bowls, according to whether rat’s response is correct (i.e., got reward) or not and whether it digs the 1st _{or 2}nd _{bowl it encounters, and recording the time spent}

to choose a bowl. Clicking ‘Next trial’ will trigger the Bayesian analysis of the current trial and then adaptively determine and display the pairing of stimuli in each bowl for the next trial. The right column on the user interface displays general information, including rat identity, current stage and trial number, the Bayesian learning criterion, and the estimated posterior probabilities of the four perceptual hypotheses from the previous trial.

Using the adaptive design software, 12 normal rats and 11 mPFC-lesioned rats have performed the 7-stage task with each trial adaptively designed. The initial analysis of these 23 rats’ data showed that both normal and lesioned rats did not show a difference in the number of trials between ID and ED stages. It seems that the adaptive design made both control and lesioned rats learn faster in the ED stage by helping rats disambiguate the relationship between stimuli and reward, such that the ID/ED difference observed in previous studies disappeared. However, considering the following factors, it is too early to draw any convincing conclusion from the initial data analysis:

 Bayesian model issue: the rat behavioural Bayesian model has not been well developed

each trial. The current incomplete Bayesian model may not be able to well predict what hypothesis a rat would use to choose bowls in next trials, and therefore make the adaptive design of learning trials inappropriate. Refinement of the current Bayesian model could solve the ID/ED non-difference issue.

 Data collection issue: the data collection procedure may not be well designed.

Specifically, for each of the first four trials in each learning stage of the 7-stage task, the rat was allowed to dig in the other bowl if it dug first in the unrewarded bowl. This special step was designed to make the rat establish what the stimuli were in the bowls, and implicitly sped up rats’ learning. However, it may also suppress the perseveration errors that the rats could potentially have made. From this perspective, the lack of difference between the ID and ED trials based on adaptive design may be potentially from the special data collection step during the first four trials in the ED stage. To avoid the potential effect of this special step, new rats’ data should be collected and analysed using the adaptive design software with the exclusion of this special step.

Therefore, more study is necessary to explore the adaptive design for attentional set-shifting tasks.

In addition, besides the above goal and principle of adaptive design, we may also explore other design goals. An alternative goal of adaptive design is to maximally distinguish the likelihoods of the competing hypotheses (Liepe et al., 2013). To achieve this goal, the adaptive design should take into account the posteriors from the previous trial and search for the configuration of bowls that, on average across all possible responses to the configuration, results in the highest variability in the posterior probabilities after the current trial. In essence, what the adaptive design typically aims for is a posterior probability of 1 for the correct hypothesis and 0 for all remaining hypotheses. The only concern is whether such an adaptive trial design would work if the conditions are changing, as we expect during learning in each stage of the 7-stage task. One future goal is to explore which design goal and corresponding design principle is more appropriate for the attentional set-shifting task.