Building Adjacencies using Pseudo Storms

The adjacencies in our tracking algorithm can be divided into two classes. The first class of adjacency is the same as in the original storm tracking algorithm for connecting two adjacent disparities. It doesn’t matter what kind of storms the disparity connects: the storms can both be real, one storm could be real and the other a pseudo storm or the storms are both pseudo storms. As long as the two disparities have a common storm in the middle image, then no connection is necessary to create this adjacency. The second class of adjacency is created whenever two disparities are connected through a built connection as in the previous section. On one side of a middle image there is one (and only one) disparity connected to the pseudo storm. On the other side of the middle image there is at least one disparity connecting the real storms that comprise the pseudo storm. A typical data structure of this new adjacency is described as in Figure 7.3a.

It is clear that in the second class adjacency there is only one disparity connected to the pseudo storm in the middle image but there could be more than one disparity connected to

(a)

(b)

(c)

Figure 7.2: Artificial tracking results for a more complicated case: (a) using the original tracking algorithm; (b) using the original tracking algorithm with pseudo storms; (c) using pseudo storm and our new pseudo storm tracking algorithm.

(a)

(b)

(c)

Figure 7.3: (a) The organization of a typical adjacency with pseudo storms; (b) Building an adjacency where the real storms have multiple disparities; (c) Building an adjacency in a more complicated situation.

the real storms that comprise it. As we have discussed before, if a real storm doesn’t have any disparity connected to it, no connection will be created for it. To summarize our dis- cussion, there is no connection between two pseudo storms that share one or more common real storms since this relationship could be very weak and won’t bring any improvement to our tracking results.

It should be noted that a pseudo storm may have a real storm that has more than one disparity connected to it, as in Figure 7.3b. In this case, we have to build an adjacency for each possible combination. As shown in Figure 7.3b, a pseudo storm has two real storms, Storm 1 and Storm 2, in the middle image, Image 2. Each of these real storms has 2 disparities connected to different storms in Image 3. In this case, the number of adjacencies we have to build is 2× 2 = 4. When we build the adjacencies, we have to go through all the possible combinations of disparities and build one adjacency for each of them. This scenario could be even more complicated as shown in Figure 7.3c, where the disparities connect to the same storm in Image 3. Since these two disparities map to the same storm, they cannot co-exist in the final storm tracks, so no adjacency is created for this combination case. Therefore, when we build the adjacencies, we have to delete the conflicting cases whenever the disparities for the real storms are pointing to the same storm in the other image.

Once the disparities for an adjacency are decided, we begin to build the adjacency. We defineAj+1as an adjacency built among three images,Ij, Ij+1andIj+2. ThenAj+1will have three parts: the set of disparities fromIjtoIj+1,SetDisp_j, defined as preceding disparity set,

in which the number of all disparities is num{SetDisp_j}; the set of disparities from Ij+1 to Ij+2, defined as succeeding disparity set, SetDisp_j+1, in which the number of all disparities

isnum{SetDisp_j+1}; and a set of connections to connect these disparities,SetConj+1, in which the number of all connections is num{SetConj+1}. Eithernum{SetDispj} ornum{SetDispj+1}has to be 1, while the other number is at least 1 but can be greater. num{SetConj+1}either equals 0 for the first class of adjacency or equals the largest number among num{SetDisp_j} and num{SetDisp_j+1}for the second class of adjacency.

Since the adjacency structure is changed to have a set of disparities, the calculation of all the partial compatibilities have to be modified as well. For every partial compatibility such as the minimum length, maximum length, size, position, orientation or velocity com- patibilities, we first calculate the average of all the disparities in one set, and then calculate the partial compatibility value based on the two average values. The calculation function can be expressed as:

1. Theminimum length compatibilityCdmin

Cdmin( −−−−−→ SjSj+1)=          1− |d1−d2| max(d1,d2) i f d1, d2> 0, 1, otherwise (7.1)

whered1is the mean of the minimum lengthd1from all the disparities inSetDisp_j and d2is the mean of minimum lengthd2from all the disparities inSetDisp_j+1.

2. Themaximum length compatibilityCdmax:

Cdmax( −−−−−→ SjSj+1)=          1− |d1−d2| max(d1,d2) i f d1, d2 >0, 1, otherwise (7.2)

whered1is the mean of the maximum lengthd1 for all the disparities inSetDisp_j and d2is the mean of the maximum lengthd2from all the disparities inSetDisp_j+1.

3. Thesize compatibilityCs:

Cs(

−−−−−→

SjSj+1)=

fs(SjSj+1)+ fs(Sj+1Sj+2)

2 . (7.3)

4. Theposition compatibilityCp:

Cp( −−−−−→ SjSj+1)= fp( −−−−−→ SjSj+1)+ fp( −−−−−−−→ Sj+1Sj+2) 2 . (7.4)

5. Theorientation compatibilityCo: Co( −−−−−→ SjSj+1)= fo( −−−−−→ SjSj+1)+ fo( −−−−−−−→ Sj+1Sj+2) 2 . (7.5)

6. Thevelocity compatibilityCv:

Cv( −−−−−→ SjSj+1)= fv( −−−−−→ SjSj+1)+ fv( −−−−−−−→ Sj+1Sj+2) 2 . (7.6)

All of the compatibilities above taken together are called the partial compatibility. The overall partial compatibility is calculated as:

Cp =ωdminCdmin +ωdmaxCdmax +ωsCs+ωpCp+ωoCo+ωvCv, (7.7)

where ωdmin, ωdmax, ωs, ωp, ωo and ωv are normalized weights that sum to 1. The partial

compatibility and the dynamically updated certainties from the adjacency’s two disparities will be used to calculate the final overall compatibility in the later relaxation iteration. This value will be then be used to determine whether the disparities are selected for the final storm tracks.

As we discussed in the previous chapter, the velocity compatibility cannot provide ob- vious improvement but badly slows down the calculation; therefore we have disabled it in our following experiments. It can also be noted that there is no suchfuzzy angle compati- bilityhere as in the original tracking algorithm. This is due to the fact that we have changed the structure of adjacency to include more than one disparity on one side of the adjacency. As a pseudo storm breaks into several real storms, it doesn’t make sense to calculate the fuzzy angle between the pseudo storm and the real storms that comprise it. Therefore in pseudo storm tracking algorithm we don’t have fuzzy angle involved by design.