Particle Filter with memory and feedback for SBP prediction

Particle Filter for visual tracking requires updating a confidence interval by calcu- lating probability with respect to the newly available information, i.e., observation (measurement) to predict the state vector at next time step. This confidence de- creases when there is no measurement available, i.e., the target being tracked is occluded (no target measurement in the image). It becomes lower as the number of frames without an observation increases once the target state is lost or occluded. The

4.4 Overview of proposed SBP tracking

standard Particle Filter will fail under this circumstance. A typical way of avoiding this problem is to restart the tracking algorithm, however, it might not be the best solution. Several researchers have proposed solutions to deal with occlusions without restarting the tracking algorithm [112], [111]. In [112], an adaptive Particle Filter is presented that uses a Rayleigh probability distribution during occlusion, while the memory-based Particle Filter in [111] combines the standard Particle Filter with a memory strategy to handle occlusions.

The proposed Particle Filter has two tracking (operation) modes, i.e., no occlusion and occlusion as shown in Fig. 4.4. In the no occlusion mode, the proposed Particle Filter behaves similar to the standard Particle Filter when the SBP is not occluded or missed by the SBP labelling framework (see Section 3.4.3). In the occlusion mode, the proposed Particle Filter uses a memory based feedback scheme when the SBP is occluded or missed by the SBP labelling framework. The proposed Particle Filter is able to track and predict SBPs in the presence or absence of occlusion, or missed SBPs, as shown in Fig. 4.4.

Given the current observation location, i.e,P(t−1) = (xcv, ycv), of a SBP at

time stept−1, the Particle Filter predicts the locationP(t) = (x′_cv, y_cv′ ) of a SBP at time stept. The state vector Xt−1= [PT VT], where P is the position of a convex

point at timet−1 and V =P(t−2)−P(t−1). A constant-acceleration dynamic modelXt is used to update the state vector, where

Xt=M Xt−1 =M[PT VT] (4.10) M=       1 0 dt 0 0 1 0 dt 0 0 dt 0 0 0 0 dt       (4.11)

wheredt is the time lapse between two frames. The confidence interval is updated using the new weights, i.e.,

π(_tn)= exp[−0.05 v u u t∑N n=1 (xcv−s(tn))2+ (ycv−s(tn))2]. (4.12)

where (xcv −s(tn)) and (ycv−s(tn)) represent the distance of N particles s

(n)

t from

4.4 Overview of proposed SBP tracking

Figure 4.4: Block diagram describing the tracking (operation) modes, i.e., no occlu- sion and occlusion of the proposed Particle Filter with memory and feedback.

4.4 Overview of proposed SBP tracking

For each SBP, a Particle Filter with 100 particles is instantiated for optimum accuracy of prediction with particles≥30 producing good results. During occlusion, the Particle Filter is initialized with the last known observation to predict the next SBP (x′_cv, y′_cv). This is achieved by keeping the temporal information of every previ- ous measurement and observation. In the event of occlusion in consecutive frames, the predicted values in the first frameP(t) and V are fed back as observations to initialize the Particle Filter for the subsequent frames.

The proposed Particle Filter with memory and feedback differs from the memory-based Particle Filter in [111] in the following aspects:

1. The memory consists of both past estimations and observations (measure- ments) while in memory-based Particle Filter only the past estimations are stored.

2. Only the two most recent past estimations and observations, i.e., at timet−2 andt−1 are stored in the memory, and are used to predict the state at timet, while in the memory-based Particle Filter past estimations include a complete history till the current timet.

3. The occlusion condition is established based on the input from SBPs’ labelling framework in Section 3.4.3, instead of a predefined threshold.

4. A combination of memory and feedback is used to predict the state vector in occlusion.

4.4.2 Motion flow for SBP prediction

Motion flow employs the direction of linear displacement, prior knowledge of the activity, temporal information of a SBP, and geometry of the human body to define criteria for locating, labelling and tracking SBPs, i.e., arm points (xA, yA) during occlusion, as detailed in Table 4.1. If the displacement dx between current arm xA

t and previous armxAt−1 point is greater than a threshold ζ =Dseg/6 = 5 (where

Dseg=30, see Section 3.4.1.4), it suggests that the maximum displacement is reached

and direction of the arm swing arm has changed. Only dx is used because the horizontal displacement of arm (pendulum) from equilibrium position to maximum displacement is intuitively more than vertical displacement. The direction of the front arm movement is constrained based on the previously labelled front arm points. The criteria in Table 4.1 are used to predict front and back arm points during Walk,

4.4 Overview of proposed SBP tracking

Table 4.1: Parameters and their value for Motion flow based arm prediction.

Activity |dx| xA_t yA_t xA yA W alk −, < ζ ≤xA_t−1 ≥y_tA₋₁ xA_t ∓dx yA_t +dy/0.4ζ W alk > ζ − − xA_t −0.4ζ yA_t +dy/0.4ζ Run < ζ ≤xA_t−1 ≥y_tA₋₁ xA_t ∓dx yA_t +dy/0.4ζ Run −,≥ζ − − xA t ∓0.8ζ yAt +dy/0.4ζ Skip ≤ζ ≤xA_t−1 − xA_t ∓dx/0.4ζ y_tA Skip − − − xH±1.4ζ yH + 4ζ Side < ζ ≤xA_t−1 − xA_t ∓dx y_tA Side > ζ − − xA_t ∓dx/ζ y_tA J ump < ζ ≤xA_t−1 − xA_t ∓dx y_tA J ump > ζ − − xA_t ∓dx/ζ y_tA P auseJ ump < ζ − ≤y_tA₋₁ xA_t y_tA+dy P auseJ ump > ζ − − xA_t y_tA

Side, Jump-in-place-on-Two-Legs , Jump Left to Right, Run Right to Left, and Skip on the Weizmann data set.

In Table 4.1,xH andyH, andxAandyA, respectively denote the coordinates of the head and predicted arm points. The upper polarity is used for Right to Left, and the lower polarity is used for Left to Right. Front arm and Back arm are distinguished respectively on Right side and Left side based on (Eq. 3.11). For all actions the arm point is predicted at the centre (xc,yc) when no conditions are

satisfied or when more than three points have been predicted consecutively. In the first row of Walk, Side, Skip, Pause Jump , and Run in Table 4.1, the relational operator and polarity of criteria for current arm (xA

t, ytA) and predicted arm (xA, yA)

are respectively reversed for front and back arm prediction in Right to Left and Left to Right. The second row of these actions is used to predict back points when they are not predicted by the first row. For Walk, dx is not used for front arm point prediction (which is denoted by a dash) but is used to predict back arm point only. For Jump, front arm point is predicted at centre (xc, yc) in occlusion, while the back

4.5 Experimental Results

back arm point is predicted at the centre.

4.5

Experimental Results