Sequential Monte Carlo decoding algorithm

When the state variable is high-dimensional and the posterior density cannot be as-sumed to be approximately Gaussian, another computationally efficient alternative is a sequential Monte Carlo algorithm. Point process adaptive filters using sequential Monte Carlo approximations to the posterior density have been developed in previ-ous literature (Ergun et al., 2007; Meng et al., 2011). Here we provide a pseudo-code description of the algorithm with extension to marked point processes. This is a boot-strap filter, so the proposal distribution is based on the one-step prediction density from the previous time step.

At each time step t, the algorithm produces a collection of weighed samples, or particles, each containing proposed values for the state variable x_t. We construct estimates for the state variable by computing their sample means over all the particles, and construct approximate 95% confidence intervals.

1. Initialization:

Set t = 0 and for i = 1, · · · , n particles, draw the initial states and parameters from an initial probability distribution and set the importance weight of the i^th particle wⁱ₀ = n⁻¹ for all i. Set t=1.

2. Importance sampling:

Using the particles from the previous step which represent the one-step prediction density defined in equation (3.4) as the sampling distribution, update all of the states x_t.

Evaluate the importance weight of the i^th particle

w_tⁱ = w^t_t−1p(∆N_t, ~m_k|H_t),

where p(∆N_t, ~m_k|H_t) is computed by equation (4.5) or (4.13).

Normalize the importance weights

˜

w_tⁱ = wⁱ_t P

jw_t^j. 3. Resampling:

Resampling can be performed at any fixed interval. Draw n particles {˜xⁱ_t : i = 1, · · · , n} from {sⁱ_t : i, · · · , n} using the residual resampling approach. Reset the weights to ˜wⁱ_t= n⁻¹ to obtrain the Monte Carlo estimate of the probability density

p(˜xt|Nt) ≈ n⁻¹

n

X

i=t

δ(˜xt− ˜xⁱ_t),

where δ(·) is the Dirac delta function, indicating a point mass at 0.

4. Repeat steps 2–3.

Chapter 4

Rapid Classification of Hippocampal Replay Content for Real-time

Applications

4.1 Introduction

Because of the profound role memory plays in our lives, the study of the neural mech-anisms that underlie memory processes, such as memory encoding, consolidation and retrieval, has attracted substantial interest among neuroscientists. Memory is thought to depend on the reactivation of patterns of activity related to previous experience, and efforts to identifying these patterns have focused, in part, on hippocampal sharp-wave ripple (SWR) events (Carr et al., 2011; Buzsaki, 2015). During these events, the hippocampus often replays sequences that represent previously experience behavioral trajectories (Redish and Touretzky, 1998; Redish, 1999; Lee and Wilson, 2002; Ji and Wilson, 2007; Foster and Wilson, 2006; Diba and Buzsaki, 2007). Sleep SWRs were originally proposed to support memory consolidation and related processes (Buzsaki, 1989) and during sleep, disrupting SWRs has been shown to lead to subsequent per-formance deficits in a spatial memory task (Girardeau et al., 2009; Ego-Stengel and Wilson, 2010).

Interestingly, replay during SWRs also occurs in the awake state, most prevalently during periods of slow movement and immobility (Foster and Wilson, 2006; Diba and Buzsaki, 2007; Csicsvari et al., 2007; Cheng and Frank, 2008; Karlsson and Frank,

2009; Singer and Frank, 2009; Pfeiffer and Foster, 2013; Wu and Foster, 2014; Silva et al., 2015). Interruption of these awake SWRs leads to a specific learning and per-formance deficit (Jadhav et al., 2012). That result helped establish the importance of SWRs, but does not establish that the specific pattern of spiking activity within SWRs contributes to memory processes. Individual SWRs can contain spiking pat-terns that replay different sequences, and further, these sequences can be replayed both in a temporally forward and reverse order during awake SWRs (Foster and Wilson, 2006; Csicsvari et al., 2007; Diba and Buzsaki, 2007; Wu and Foster, 2014;

Ambrose et al., 2015). In particular, reverse replay is hypothesized to be an integral mechanism for learning about recent events and insights into this type of replay is could be critical to understanding how animals learn from experience (Foster and Wilson, 2006; Pfeiffer and Foster, 2013).

Therefore, a deeper examination of how replayed information contributes to learn-ing and decision-maklearn-ing requires the ability to manipulate SWR events based on their content. This presents a fundamental analysis challenge: to decode and characterize replay content in real-time.

However, studies of SWR events have so far been mostly, if not all, limited to analyses of spiking data that were sorted off-line. Currently, real-time spike-sorting algorithms tend not to be sufficiently accurate to allow for closed-loop interventions.

Additionally, methods for categorizing replay content typically require waiting for the completion of a replay event, for example, to compute a correlation measure to study temporal replay order (Foster and Wilson, 2006; Wu and Foster, 2014). The lack of dynamic estimation methods makes it difficult to further probe any causal relationship between a specific replay sequence and memory function that may happen downstream of the replay event. Among the currently used methods, there is no consensus as to which classifies the replay content most accurately and reliably.

One approach that has been successful in decoding dynamic content from neural data is state-space modeling (Brown et al., 1998; Smith and Brown, 2003; Eden et al., 2004; Truccolo et al., 2005; Srinivasan et al., 2006; Kemere et al., 2008; Wu et al., 2009; Huang et al., 2009; Paninski et al., 2010; Koyama et al., 2010). Here we incorporate state-space methods to develop a dynamic algorithm to determine if the content of a replay event represents a specific sequence, which will then make it possible to interrupt events based on the type of spatial trajectory they represent.

Previously, we and others have developed decoding methods that do not require multiunit spiking waveforms to be sorted into single units based on the theory of marked point processes (Chen et al., 2012; Kloosterman et al., 2014a; Deng et al., 2015). Here we extend the state of our marked point process filter to include a discrete state variable that categorizes the content of hippocampal replays in real time. By including such a “decision state”, we make use of the underlying state process in a content-specific way by selectively including different sources of information. In particular, it allows us to incorporate into our model three fundamental features of the information content of individual replay events: the location where the trajectory begins, whether the sequence of locations represents a trajectory proceeding toward or away from the animal’s current location, and whether the spiking pattern reflects place-field structure for a specific direction of movement. We then use the discrete decision state filter to categorize the replay content and compute our confidence about the classification. We illustrate our approach by decoding experimental data recorded in the hippocampus of a rat performing a spatial memory task.