5.4 Position optimization
5.4.3 Occlusion Detection and Classification
Due to occlusions or shape changes, some points (o ∈ O − ˜O) of an object could have no correspondence with the foreground F due to shape changes or occlusions. Unfortunately, these two reasons require two different and conflicting solutions. To keep a memory of the object shape even during an occlusion the object model need to be slowly updated; at the same time a fast updating can better face shape changes. Other proposed techniques that exploit probabilistic appearance models without coping with occlusions explicitly, use a unique threshold to reach a good trade off between the above mentioned behaviors. An example is shown in Fig.5.4, in which, after a few frames, the track model looses any knowledge about the per- son’s legs. This approach has some drawbacks, such as the shift of the centroid and the erroneous estimation of position and trajectory. In our work, the adaptive update function can be enriched by the knowledge of occlusion regions. In partic- ular, if a point is detected as occluded we freeze the color and α value of the point instead of using Eq. (5.16) and Eq. (5.18).
The introduction of a higher level reasoning is necessary in order to discrimi- nate between occlusions and shape changes. The set of non visible points ON V are
the candidate points for occluded regions. After a labeling step over ON V, a set
of not visible regions (of connected points) is created; sparse points or too small regions are pruned and a final set of Non Visible Regions {N V Rj} is created. Each
of them can be classified as one of these three classes:
1) dynamic occlusions RDO: occlusions due to overlap of another object, closer
to the camera; therefore the pixels of this region were assigned to the other object;
2) scene occlusions RSO: due to (still) objects, included in the scene and there-
(a) (b) (c) (d)
Figure 5.5: Example of erroneous track freezing in case of occlusion. (a) Original image; (b) an occlusion causes the percentage of visible pixels to became too low and the model updated is stopped; (c) the track andV O matching is still maintained; (d) the track 20 is lost and the new track 21 is created because of the change of shape.
segmentation algorithm, but actually positioned closer to the camera; 3) apparent occlusions RAO: regions not visible because of shape changes,
silhouette’s motion, shadows, or self-occlusions.
The presence of an occlusion can be inferred exploiting the Confidence value of Eq. 5.12 decreasing below an alerting value, since in case of occlusion the object’s shape changes considerably. The occluded points x of the object model (x ∈ RDOorx ∈ RSO) should not be updated since we do not want to lose the
memory of it. Instead, if the Confidence decreases due to a sudden shape change (apparent occlusion), not updating the object state would create an error. The solu- tion is a selective update according to the region classification.
The detection of the first type of occlusion is straightforward, because we al- ways know the position of the objects, and we can easily detect when two or more of them overlap. RDO regions are composed by the points shared between object
Okand other object Oi but not assigned to Ok. We can mathematically formulate
this check as:
RDO =
n
o ∈ ON V,i| ∃o0 ∈ Oj∧ gO−1i(o) = gO−1j(o0)
o
(5.22) To distinguish between RSO and RAO the position and the shape of the ob-
jects in the background can be helpful, but usually they are not provided. However, most of the segmentation algorithms from fixed camera make a background model available. It is computed at each frame in background suppression segmentation techniques, or can be estimated only when needed [5, 82, 94]. Our proposal to dis- criminate between RSOand RAOexploits the background edges set. This subset of
points of the background model contains all points of high color variation, among which the edges of the objects are usually detected. In case of a RSOwe would ex-
the visible part of the track. From the whole set of not visible points ON Vt defined as in Eq. (5.15), we only keep those with a not negligible value of the probability mask in order to get rid of the noise due to motion. The remaining set of points is segmented into connected regions. Then, for each region, the area weighted with the probability values is calculated, and too small regions are pruned. The remain- ing non visible regions (N V Rj) belonging to an object O, must be discriminated
into background object occlusions and apparent occlusions. We call B(·) the set of border points of a region. At the same time, the edges of the background model are computed by a simple edge detector, reinforced by probability density estimation. We could exploit a more robust segmentation technique, e.g. mean shift [91], to extract the border of the objects in the background image, but in our experiments the edge detection has given good results, requiring much less computation. Given the set of edge points E = {ei}i=1..nin the background image a probability den-
sity estimate for the background edges can be computed using a kernel ϕ(x) and a window h: pn( x| E) = 1 n n X i=1 1 h2ϕ kx − eik h (5.23) The probability density for non-edges p(x| ¯E) can be assumed to be uniform over the same region R. We can then naively compute the a posteriori probability for a pixel x to be an edge point:
P ( E| x) = P ( x| E)
P ( x| E) + P x| ¯E (5.24)
where we assumed equal a priori probability. We can now compute the aver- age a posteriori probability of the set of points o ∈ BON V to be generated by
the background edges. In particular we are interested on the subset ˜B(ON V) =
B(ON V) ∩ B(OV) that is the part of the border of ON V connected to the visible
part OV. The probability estimate allows tolerating a noisy match between BON V
and the edge points. In case this average probability is high enough, meaning that the contour of the occluded region has a good match with the edges, we can infer that another object is hiding a part of the current one, and thus the region is labeled as RSO, otherwise as RAO. In other words, if the visible and the non visible part
of an object are separated by an edge, then plausibly we are facing an occlusion between a still object of the scene and the observed moving object. Otherwise, the shape change is more reasonable cause of the no more visible points.
In Fig. 5.6 an example is shown: a person is occluded for a large part by a stack of boxes that are included in the background image. Two parts of its body are not segmented and two candidate occlusion regions are generated (Fig. 5.6(c)): one of them is a shadow included in the object model but now disappeared. In Fig. 5.6(d) the borders of the N V Rs are shown, with pixels that have a good match with the edges marked in black. In real occlusion due to a background object the percentage of the points that have a match with respect to the set of bounding points is high;
(a) (b) (c)
(d) (e)
Figure 5.6: Example of RSOregions: (a) The input frame. (b) The color and theα repre-
sentation of the object points. (c) The current background model (d) The visible part of the track (blue) and the candidate occlusion regions (red).(e) the borders of the Non Visible Regions. Points that have a good match with edge pixels of the background are marked in black. ThusR1is classified as aRBOregion whileR2as aRAOregion.
thus the region is classified as RSO. On the contrary, for the apparent occlusion
(the shadow) we have no matching pixels, and consequently this region is classified as RAO.
In Fig. 5.7 we see another example of apparent occlusion: a person initially standing suddenly falls onto the floor; since his shape considerably changes, a candidate occlusion region appears in correspondence of his legs. The algorithm correctly classifies this region as apparent occlusion, and thus the model is updated.