Output: Plfl(list of parameters to use in tracking)
1: functionFeatSelect(tl)
2: Pl ←The list of all image parameters 3: fl← List of F-stat values∀pi ∈Pl 4: for all pi ∈Pldo
5: dl ←Empty list of distance pairs
6: foreach track hypothesis ti∈tldo in parallel 7: ifti.length > 1then
8: for alldetection pairs di∈ti do
9: dl←pair of histogram distances for
10: (1)pairdi
11: (2)first detection indi and 12: random detection not int
i
13: end for
14: end if
15: end for
16: fl←F-stat for pi
17: end forreturn Plfl .Top−Kparameters fromPl ranked on F-stat infl
18: end function
In order to create the two populations of feature values, we only utilize those trajecto- ries that contain more than one detection. So, each temporally sequential pair in these trajectories is used as a comparison point belonging to the class labeled as “same object.” Then, the first detection of each of these pairs is taken and compared to another detection from the data set that resides in another trajectory. These comparisons are considered as being in the class labeled as “different object.” This process results in a balanced50/50
split between the two classes. Once the two classes of measurements are complete for a particular image parameter on a particular AIA channel, the F-Statistic is calculated and the value is stored for use once all parameters are evaluated. This process repeats for all
90AIA channel and image parameter pair combinations and then all of the features are
ranked by ordering them in descending value of their calculated F-Statistic value. The Top−K(K=20) are then returned as the feature set to use for the solar event type being processed, which was deemed sufficient in Chapter 3. The track estimates are broken
down after the feature selection algorithm is finished, and the original set of observations is passed on to Stage2with the selected feature list from Algorithm4.3.
4.3.2 Tracking Stage2: Iterative Tracking
As can be seen in Figure 4.4, Stage 2 is given an input of the individual observation
reports and the feature set selected by Stage1. The goal of this stage is to find the most
likely trajectories from the input set of observations, which is not a trivial task. In track- ing single objects in the set of observations, object trajectories arise by simply connecting the detections of the single object over time. The addition of multiple detections in any given time step breaks this naïve approach and the likelihood of each path must then be considered. In addition, failures in detection also become an issue, such as false positive detections or detections that were missed.
To solve the additional complexity that is found in tracking multiple objects at once, the problem becomes how to associate multiple detections into multiple trajectories across time. The data association approach of multiple hypothesis tracking is used be- cause it can be quite robust to these occasional failures in detection. For example, in the case of a false positive, erroneous detections are often isolated in time and are there- fore easily discarded. Similarly, recovery from failed detections can be incorporated to produce an uninterrupted trajectory [50].
In order to recover from failed detections, our implementation allows “skipped frames”, or time steps that erroneously do not have a detection report for a given trajectory. This is accomplished by searching for spatiotemporal neighbors of the trajectory currently being processed in a timestamp that is later than what would be expected for the next observation in the same object trajectory. However, this increases the number of trajec- tory hypotheses that need to be evaluated, as we then need to consider all the possible trajectories in each of the intervening time steps.
To illustrate this, let us assume that an object detection report oki (ki denotes the ith frame and the kth object report in said frame) can be associated with any object detection report in a subsequent frame oki+1. Then, when searching m frames, with n
object reports per frame, the number of all possible track paths is bound byθ(nm). This means that finding the optimal global trajectory hypothesisτis an optimization problem that grows in complexity at an exponential rate with the number of reporting frames [61].
To help mitigate this exponential growth problem, we adopt a similar approach to that used in [52]. In this approach, there is a hierarchy of stages in the tracking algorithm
that iteratively grows track fragments as they are passed from one stage to the next. This set of stages reduces the complexity of finding the global optimal trajectory by making the assumption that detections should be linked to temporally closer detections before more temporally distant solutions are even considered.
This assumption first limits the exponentm by creating trajectory fragments that have no gaps or “skipped frames” in them. This is accomplished by only searching the next frame for spatiotemporal neighbors. By doing this, the number of object detection re- ports available for possible association in subsequent frames is limited as a result of the independence assumption mentioned in Section 4.1. This assumption states that detections can belong to one and only one trajectory. So, once a detection belongs to a trajectory, it cannot be considered for association in a subsequent stage, and thus reduces
nfor stages that allowm to grow.
In Figure 4.4, we have changed the naming slightly from [52] in that the processing
of results from previous solutions are simply called iterations of Stage 2. In the first
iteration of Stage2, the input is all single detections, which are treated as trajectory frag-
ments of length one. After the first iteration of Stage2, most of the trajectory fragments
are longer than a single detection. As such, this iteratively growing framework uses the simpler transition probability Plink(oi+1|oi) = Plink,var1 from Equation 4.4 on the first
tories. Then, the more complex transition probabilityPlink,var2 is used in later iterations
where most trajectories are of a length that previous motion can be used for comparison. We utilize these two variants of the transition probability calculation for two reasons: (1.) Because it is not possible to compare movement similarity when trajectory fragments
are length one in the first iteration. (2.) We do not want to exclude movement similarity
altogether, because greater ambiguity is present in the association hypotheses in later iterations due to larger number of skipped frames allowed, and more information about the similarity is helpful. It should be noted that the input observations to the transition probability Plink(oi+1|oi) are trajectory fragments and are treated as individual observa- tions in our notation.
In order to search for a possible next observation to link to, and to calculate the tran- sition probability, a search area must first be created. However, as stated previously, these trajectory fragments are only one detection in length in the first iteration of Stage2.
So, the search area is created by using the differential rotation of the sun from [62] to
estimate how far a detection might move in the elapsed time before the next detection report, and the bounding box of the detection is projected by that estimate. After the first iteration of Stage2, there are now trajectory fragments longer than a single detection. So,
it is now possible to use the previous motion of the trajectory fragments to predict where the next trajectory fragment may be located. In the cases where trajectory fragments are now sufficiently long enough, meaning there are more than two reports, the previous motion is used to predict the next trajectory fragment location. Also, the subsequent iterations of Stage 2 allow the search for candidate track fragments to occur more than
one time period in the future, which is an attempt to allow for missed detections.
Once the spatiotemporal neighbors are found for all of the trajectory fragments in the current dataset, and the transition probabilities are calculated for those neighbors, the most probable trajectory paths need to be found. This is called the data association problem, and is solved as the maximization of thea posterioriprobability. This association
S T u1 u2 v1 v2 u3 u4 u5 u6 v3 v4 v5 v6 u7 u8 u9 v9 v8 v7 (ui ,vi) Observation edges (vi ,uj) Transition edges (S,ui )&(vi,T) Enter/Exit edges
Figure4.5:An example of the cost-flow network used to associate detections into longer track fragments. This graph depicts a set of detections with three time steps and nine
observations, see Section 4.3.3 for a detailed description of the components of this
graph.
is done at each iteration of Stage 2, slowly building an approximation of the global
maximization of the a posteriori probability by using the resultant trajectories from the previous iteration as the starting dataset for the next. In the next section, we describe how this association process is accomplished through the use of a minimum cost-flow problem on a directed acyclic graph.
4.3.3 Data Association Through Track Graphs
In the previous section, we discussed how the tracking framework shown in Figure 4.4 must solve the data association problem at each iteration. In this section, we begin our discussion of the association problem and utilize graphs as a means to solve it. We convert the association problem to a minimum cost-flow problem in a directed acyclic graph. In doing so, we convert the MAP probability problem, presented in Equation4.1, to find the minimum cost flow through a graph. We call this graph the track fragment graph, where the vertices are associated with trajectory fragments; the fragments may
be individual detections of an object or multiple detections linked together in previous steps to form a longer trajectory fragment.
In the track graph, a hypothesis of an object’s movement is a path in the graph, and a global association hypothesis is a set of consistent trajectories. In order to maintain consistency, no two trajectories in the global association hypothesis can share a single object detection reportoi. Edges/links that connect two vertices are created if and only
if the nodes have the possibility of representing the same object trajectory at differing timestamps. To be considered as a possible representation of the same object trajectory, the start time of the successor trajectory fragment must be later than the end time of the predecessor trajectory fragment.
As was mentioned in Section 4.3.2, if it is assumed that any object detection report
oki (ki denotes the i
th frame and the kth object report in said frame) can be associated
with any object detection report in a subsequent frame oki+1, then for m frames with n
object reports per frame, the number of all possible trajectory paths isθ(nm). This means that the optimal hypothesis is found by solving an optimization problem, which grows exponentially in size with the number of frames [61], and explicitly storing each of these
track paths will also grow exponentially. So, we take two approaches to help mitigate this growth in storage requirements. The first of which was discussed in Section 4.3.2, where we iteratively lengthen the allowed skipped frames as trajectories get longer and trajectory fragments get fewer. The second is the graphical formulation of the trajectory association problem in which hypotheses are implicitly represented as paths through the graph, instead of having to store each path explicitly.
The graphical formulation approach to mitigating storage growth will still have the exponential rate of growth in the number of trajectory hypotheses as dictated by the data when the number of potential matches increases. However, it accomplishes this with increasing the number of nodes and edges in the graph at a linear rate with respect to the number of trajectories represented in the data. It is because of this linear rate
Algorithm 4.4Track Graph Construction